Documentation

Technical documentation, integration guides, and resources for the DION-E framework

Getting Started

The DION-E framework is designed to be flexible and easy to integrate with your existing workflows. Use our comprehensive documentation to get started with evaluating LLM outputs across multiple dimensions.

Installation
Set up the framework
Quickstart
Learn the basics
API Reference
Detailed API docs

Installation

The DION-E framework can be installed using pip:

pip install dion-e-framework

For a complete installation with all optional dependencies:

pip install dion-e-framework[full]

System Requirements

  • Python 3.8 or higher
  • 4GB RAM minimum (8GB+ recommended)
  • CUDA-compatible GPU optional but recommended for large-scale evaluations

Quickstart

Here's a simple example to get started with evaluating an LLM response:

from dion_e import LLMEvaluator

# Initialize the evaluator
evaluator = LLMEvaluator()

# Sample LLM response to evaluate
response = """
Linear regression models the relationship between variables 
using a linear equation. It assumes linearity, independence, 
homoscedasticity, and normal distribution of residuals. 
The method uses ordinary least squares estimation.
"""

# Evaluate the response
results = evaluator.evaluate(response)

# Print results
for metric_name, result in results.items():
    print(f"{metric_name}: {result.value}")
    print(f"Details: {result.details}")
    print()

Configuration

The framework can be configured to use specific metrics and parameters:

from dion_e import LLMEvaluator, MetricConfig

# Configure specific metrics and their parameters
config = {
    "CognitiveLoad": {
        "enabled": True,
        "params": {
            "readability_weight": 0.4,
            "syntactic_weight": 0.3,
            "memory_weight": 0.2,
            "vocabulary_weight": 0.1
        }
    },
    "Wordiness": {
        "enabled": True,
        "implementation": "technical",  # Use domain-specific evaluation
        "params": {
            "domain": "scientific"
        }
    }
}

# Initialize with configuration
evaluator = LLMEvaluator(config=config)

# Now only the configured metrics will be used
results = evaluator.evaluate("Your text here...")

API Reference

The core classes and methods provided by the DION-E framework:

LLMEvaluator

The main class for evaluating LLM outputs with multiple metrics.

Methods

  • evaluate(text: str) → Dict[str, MetricResult]: Evaluate a single text with all enabled metrics
  • batch_evaluate(texts: List[str]) → List[Dict[str, MetricResult]]: Evaluate multiple texts efficiently
  • get_metrics() → List[str]: Get a list of available metrics
  • get_metric_config(name: str) → MetricConfig: Get the configuration for a specific metric

MetricConfig

Configuration class for individual metrics.

Properties

  • name: str: The name of the metric
  • enabled: bool: Whether the metric is enabled
  • implementation: str: The implementation type ("lite", "standard", or "full")
  • params: Dict[str, Any]: Metric-specific parameters

MetricResult

Stores the result of a metric evaluation.

Properties

  • value: float: The primary metric score
  • details: Dict[str, Any]: Detailed component scores and analysis
  • metadata: Dict[str, Any]: Additional information about the evaluation
Full API Documentation
DION-E SaaS Platform Coming Soon!

Code Examples

Basic Usage

from dion_e import LLMEvaluator

# Initialize the evaluator with default settings
evaluator = LLMEvaluator()

# Evaluate a single response
response = "Your LLM output text here..."
results = evaluator.evaluate(response)

# Print results for each metric
for metric_name, result in results.items():
    print(f"{metric_name}: {result.value}")

Batch Processing

from dion_e import LLMEvaluator

# Initialize the evaluator
evaluator = LLMEvaluator()

# Multiple responses to evaluate
responses = [
    "First LLM response...",
    "Second LLM response...",
    "Third LLM response..."
]

# Evaluate all responses in batch mode
batch_results = evaluator.batch_evaluate(responses)

# Process results
for i, result_set in enumerate(batch_results):
    print(f"Results for response {i+1}:")
    for metric_name, result in result_set.items():
        print(f"  {metric_name}: {result.value}")
    print()

Specific Metrics

from dion_e import LLMEvaluator
from dion_e.metrics import CognitiveLoadScore, WordinessScore

# Initialize specific metrics directly
cls = CognitiveLoadScore()
wds = WordinessScore(domain="technical")

# Text to evaluate
text = "Your technical document here..."

# Get individual metric scores
cognitive_load = cls.score(text)
wordiness = wds.score(text)

print(f"Cognitive Load: {cognitive_load.value}")
print(f"Readability: {cognitive_load.details['readability']}")
print(f"Syntactic Complexity: {cognitive_load.details['syntactic_complexity']}")

print(f"Wordiness: {wordiness.value}")
print(f"Lexical Density: {wordiness.details['lexical_density']}")
print(f"Information-to-Token Ratio: {wordiness.details['info_token_ratio']}")

Custom Configuration

from dion_e import LLMEvaluator, MetricConfig
from dion_e.config import BenchmarkConfig

# Create benchmark configuration
config = BenchmarkConfig()

# Add specific metric configurations
config.add_metric(
    MetricConfig(
        name="CognitiveLoad",
        enabled=True,
        implementation="standard",
        params={
            "readability_weight": 0.4,
            "syntactic_weight": 0.3,
            "memory_weight": 0.15,
            "vocabulary_weight": 0.15
        }
    )
)

config.add_metric(
    MetricConfig(
        name="EnhancedNovelty",
        enabled=True,
        implementation="full",
        params={
            "reference_corpus": "scientific_papers",
            "semantic_weight": 0.5,
            "topical_weight": 0.3,
            "structural_weight": 0.2
        }
    )
)

# Initialize evaluator with custom configuration
evaluator = LLMEvaluator(config=config)

# Only the configured metrics will be used
results = evaluator.evaluate("Your text here...")

Using Plugins

from dion_e import LLMEvaluator
from dion_e.plugins import initialize, register_plugin
from my_custom_plugin import MyCustomMetric

# Initialize plugin system
initialize()

# Register custom plugin
register_plugin(MyCustomMetric)

# Create evaluator with plugins enabled
evaluator = LLMEvaluator(use_plugins=True)

# Evaluate text with all metrics including plugins
results = evaluator.evaluate("Your text here...")

# Access custom metric results
if "MyCustomMetric" in results:
    custom_result = results["MyCustomMetric"]
    print(f"Custom metric score: {custom_result.value}")
    print(f"Custom details: {custom_result.details}")

Visualization Example

from dion_e import LLMEvaluator
from dion_e.visualization import create_radar_chart, create_model_comparison

# Evaluate multiple models
evaluator = LLMEvaluator()

model_responses = {
    "gpt-4": "Response from GPT-4...",
    "claude-3": "Response from Claude 3...",
    "gemini-pro": "Response from Gemini Pro...",
    "llama-3": "Response from Llama 3..."
}

# Evaluate all responses
results = {model: evaluator.evaluate(response) for model, response in model_responses.items()}

# Create radar chart comparing models
create_radar_chart(results, "model_comparison_radar.png")

# Create bar chart comparison for a specific metric
create_model_comparison(results, metric="CognitiveLoad", "cognitive_load_comparison.png")

DION-E SaaS Platform Coming Soon!

Our comprehensive SaaS platform will include:

📚

Documentation Hub

Complete reference docs, tutorials, and integration guides

🧪

Code Repository

Extensive example code and implementation patterns

👥

Developer Community

Forums, support channels, and contributions from the community

Resources

DION-E SaaS Platform Coming Soon!

We're preparing comprehensive documentation, research papers, and community resources for the DION-E framework. These will be available when we launch our SaaS platform.

📚

Documentation

Comprehensive guides and tutorials

🧪

Code Examples

Ready-to-use implementation samples

📝

Research Papers

Technical details and methodology

👥

Community Forum

Discussion and knowledge sharing

Limited Early Access

Get Notified When Our Platform Launches

Sign up to be the first to know when our SaaS platform and resources are available.

Have Questions?

Want to learn more about the DION-E framework or discuss potential use cases?