ai-ensemble-suite 0.5.1
Python framework for multiple GGUF language models to collaborate on tasks
Requires Python: >=3.10
Keywords: llm, ai, ensemble, language-model, gguf, llama, ai-ensemble

# AI Ensemble Suite

> **Alpha Release (v0.5)**: This library is in active development. While already functional and useful for real-world applications, some APIs may change, and additional cleanup and feature development are in progress. Feedback and contributions are welcome!

[![Python Versions](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue.svg)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT)
[![Status](https://img.shields.io/badge/status-alpha-orange.svg)](https://github.com/StephenGenusa/ai-ensemble-suite)

Python framework for multiple GGUF language models to collaborate on tasks using structured communication patterns, aggregating their outputs into coherent responses. While I chose to support GGUF to begin with, I plan to add OpenAI compatible server support allowing local LLM server and Internet APIs to be called.

## Table of Contents

- [Installation](#installation)
- [Quick Start](#quick-start)
- [Key Features](#key-features)
- [Ensembles Explained](#ai-ensembles-explained)
- [Why Ensembles Work](#why-ensembles-work)
- [Why This Library?](#why-this-library)
- [Examples](#examples)
- [Decision Guide: Choosing Collaboration & Aggregation Patterns](#decision-guide-choosing-collaboration--aggregation-patterns)
- [Collaboration Phases](#collaboration-phases)
- [Aggregation Strategies](#aggregation-strategies)
- [Architecture Diagram](#architecture-diagram)
- [Class Hierarchy and Interaction Diagram](#class-hierarchy-and-interaction-diagram)
- [Project File Structure](#project-file-structure)
- [API Reference](#api-reference)
- [Requirements](#requirements)
- [Contributing](#contributing)
- [Running Tests](#running-tests)
- [Roadmap](#roadmap)
- [License](#license)
- [Special Thanks](#special-thanks)

## Installation

### Prerequisites

- Python >= 3.10
- Sufficient RAM for running multiple language models
- Preferably a GPU with sufficient VRAM to load multiple models, otherwise your llama-cpp will store models in RAM and use your CPU for inference

### Install via pip

For GPU acceleration note the export/set See optional dependencies below before installing

```bash
export CMAKE_ARGS="-DGGML_CUDA=on"
export FORCE_CMAKE=1
pip install ai-ensemble-suite
```

```windows
SET CMAKE_ARGS="-DGGML_CUDA=on"
SET FORCE_CMAKE=1
pip install ai-ensemble-suite
```
### Optional Dependencies

The library has optional dependencies that can be installed with:

```bash
pip install "ai-ensemble-suite[rich]"  # For enhanced console output
pip install "ai-ensemble-suite[dev]"   # For development tools
```


### Installation with GPU acceleration

For NVIDIA GPU acceleration:

```bash
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 pip install llama-cpp-python
pip install ai-ensemble-suite
```

For Apple Metal (macOS):

```bash
CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install llama-cpp-python
pip install ai-ensemble-suite
```

### Development Installation

To install the package with development dependencies:

```bash
git clone https://github.com/StephenGenusa/ai-ensemble-suite.git
cd ai-ensemble-suite
pip install -e ".[dev]"
```

[Back to top](#ai-ensemble-suite)

## Quick Start

You will want to determine which GGUF models you wish to work with. Download them and place in the root folder of the project under a /models directory... or change the YAML to point to the location of your models.

Here's a minimal example to get started with AI Ensemble Suite:

```python
from ai_ensemble_suite import Ensemble

# Initialize the ensemble with a configuration
ensemble = Ensemble(config_path="path/to/config.yaml")

# Initialize models (loads them into memory)
await ensemble.initialize()

# Ask a question
try:
    response = await ensemble.ask("What are the key considerations for sustainable urban planning?")
    print(response)
finally:
    # Release resources when done
    await ensemble.shutdown()

# Alternatively, use with async context manager
async with Ensemble(config_path="path/to/config.yaml") as ensemble:
    response = await ensemble.ask("What are the key considerations for sustainable urban planning?")
    print(response)
```

[Back to top](#ai-ensemble-suite)

## Key Features

- ✅ **Multiple Collaboration Phases**: Choose from over a dozen different collaboration methods to design your ensemble approach
- ✅ **Specialized Model Roles**: Assign different roles (critic, synthesizer, researcher, etc.) to models, allowing them to specialize in different aspects of task completion
- ✅ **Flexible Aggregation Strategies**: Use different methods for combining model outputs into coherent, high-quality responses
- ✅ **Advanced Tracing**: Get detailed traces of model interactions and decision-making processes
- ✅ **Extensible Design**: Easily add custom collaboration phases or aggregation strategies
- ✅ **Built for GGUF Models**: Optimized for running multiple smaller GGUF language models locally
- ✅ **Confidence Estimation**: Token probability analysis and self-evaluation capabilities
- ✅ **Concurrent Model Management**: Efficient loading and execution of multiple models
- ✅ **Async-First Design**: Native async/await support with context manager
- ✅ **YAML Configuration for Models**: YAML configuration files to hold model parameters and queries
- ✅ **Jinja2 Templating**: The (query) "templates" portion of the the YAML configuration file support Jinja2

[Back to top](#ai-ensemble-suite)


## AI Ensembles Explained

Think of AI ensembles like getting advice from a group of experts instead of just one person. Just as you might ask several friends for opinions before making an important decision, AI ensembles combine the strengths of multiple AI systems or approaches to produce better results than any single AI could achieve alone. Instead of relying on one AI that might have blind spots or make certain types of mistakes, ensembles bring together diverse AI perspectives that can check each other's work, complement each other's strengths, and collectively arrive at more reliable answers. It's similar to how a team of doctors with different specialties might collaborate on a difficult medical case—the combined expertise leads to better outcomes than what any individual doctor could provide on their own.

**Brief Explanations of AI Ensemble Techniques**

* **AsyncThinking**: Like a rapid brainstorming session where multiple people jot down ideas independently before sharing, this technique generates diverse initial thoughts quickly without influence from other perspectives.

* **StructuredCritique**: Similar to having your work reviewed by a tough but fair editor, this approach systematically evaluates ideas, identifies logical flaws, and improves the rigor of thinking.

* **SynthesisOriented**: Acts like a skilled mediator who finds common ground between opposing viewpoints, integrating different perspectives into a balanced, comprehensive analysis.

* **RoleBasedDebate**: Resembles a panel discussion with experts from different fields, each contributing specialized knowledge to address complex topics from multiple angles.

* **HierarchicalReview**: Works like a multi-stage editing process, where content is refined layer by layer, with each review focusing on different aspects for progressive improvement.

* **CompetitiveEvaluation**: Functions like a contest where multiple solutions compete, and the strongest approach wins based on objective criteria.

* **PerspectiveRotation**: Similar to walking around a sculpture to view it from all sides, this technique examines issues from different stakeholder perspectives, ethical frameworks, or creative angles.

* **ChainOfThoughtBranching**: Like mapping out a complex maze with multiple possible paths, this method explores different reasoning routes for problems with multiple decision points.

* **AdversarialImprovement**: Acts as a stress-test or devil's advocate, actively looking for weaknesses in a solution to strengthen it against potential problems.

* **RoleBasedWorkflow**: Operates like a production line with specialized stations, creating a structured process where different roles handle specific aspects of a multi-stage analysis.

* **Bagging**: Works like taking the average of multiple poll results to get a more stable prediction, reducing the impact of outliers or unusual patterns.

* **UncertaintyBasedCollaboration**: Similar to how a group might work together on a puzzle with missing pieces, this approach handles ambiguous questions by combining different levels of confidence.

* **StackedGeneralization**: Functions like a team of specialists with a coordinator, where outputs from different AI models are combined to leverage their unique strengths and minimize weaknesses.

[Back to top](#ai-ensemble-suite)

## Why Ensembles Work

Ensemble methods have a strong theoretical and empirical foundation in machine learning, and they're particularly effective with language models for several reasons:

1. **Diverse Knowledge and Perspectives**: Different language models, even when trained on similar data, develop slightly different internal representations and "expertise areas." By combining multiple models, you access a broader knowledge base than any single model contains.

2. **Error Reduction Through Aggregation**: Models tend to make different errors. When their outputs are combined intelligently, errors from one model can be corrected by others, leading to more accurate results.

3. **Specialization Through Roles**: When models adopt specialized roles (like critic, researcher, or synthesizer), they can focus on specific aspects of a task. This division of cognitive labor mirrors effective human teams and leads to more thorough analysis.

4. **Iterative Refinement**: Multi-step collaboration allows initial ideas to be critiqued, refined, and expanded. This resembles human drafting and editing processes, typically producing higher quality results than single-pass generation.

5. **Confidence Calibration**: Ensemble techniques help identify areas of uncertainty or disagreement between models, leading to better-calibrated confidence in the final output.

Research consistently shows that properly designed ensembles outperform even the strongest individual models, often by significant margins. AI Ensemble Suite provides the infrastructure to easily tap into these powerful techniques.

[Back to top](#ai-ensemble-suite)

## Why This Library?

AI Ensemble Suite was created to address two key challenges:

1. **The Need for Human-Friendly Ensemble AI**: After extensive searching for a comprehensive yet easy-to-use library for ensemble AI work that followed a "for humans" philosophy, nothing quite fit the bill. This library makes it easy to harness multiple smaller language models on local machines to produce enhanced AI responses.

2. **Structured Collaboration Patterns**: Rather than just averaging model outputs, AI Ensemble Suite implements sophisticated collaboration patterns where models can critique, refine, and extend each other's work - resulting in higher quality responses that benefit from diverse model strengths.

The framework is designed with simplicity in mind for common use cases while providing comprehensive customization for advanced users.

Additionally, this project served as a meta-challenge to build a medium sized Python library using AI assistance despite context window limitations, developing techniques to work around these constraints.

[Back to top](#ai-ensemble-suite)

## Examples

The library includes several example scripts demonstrating different ensemble techniques:

- **Basic Usage**: Non-ensembled simple usage with default configurations
- **Structured Debate**: Models present opposing viewpoints to refine conclusion
- **Expert Committee**: Specialized models contribute domain expertise
- **Hierarchical Review**: Progressive refinement through layers of specialized models
- **Competitive Evaluation**: Multiple solutions generated and evaluated against criteria
- **Perspective Rotation**: Problem analyzed through different framing lenses
- **Chain-of-Thought Branching**: Reasoning paths that branch and reconverge
- **Adversarial Improvement**: One model finds flaws in another's reasoning
- **Role-based Workflow**: Models adopt complementary roles in a structured process
- **Bagging**: Combines models to reduce prediction variance

> **Note on Examples**: Some example files require additional libraries not included in requirements.txt. Several examples also generate graphic charts to disk, which were implemented to help visualize how the ensemble builds the final result. These visualization features are optional but helpful for testing and understanding the process.

[Back to top](#ai-ensemble-suite)

## Decision Guide: Choosing Collaboration & Aggregation Patterns

### Collaboration Patterns

| When you need... | Use this collaboration pattern | Best for |
|------------------|-------------------------------|----------|
| Quick independent analyses | **AsyncThinking** | Simple questions, brainstorming, diverse initial ideas |
| Critical evaluation of ideas | **StructuredCritique** | Evaluating arguments, finding flaws, improving rigor |
| Balanced perspectives | **SynthesisOriented** | Finding common ground, integrating viewpoints, balanced analysis |
| Multiple specialist perspectives | **RoleBasedDebate** | Complex topics requiring multiple forms of expertise |
| Progressive improvement | **HierarchicalReview** | Content requiring layer-by-layer refinement or fact-checking |
| Competition between solutions | **CompetitiveEvaluation** | Generating multiple solutions and selecting the best one |
| Examining from different angles | **PerspectiveRotation** | Ethical analysis, stakeholder considerations, creative ideation |
| Complex reasoning paths | **ChainOfThoughtBranching** | Mathematical problems, logic puzzles, decision trees |
| Stress-testing solutions | **AdversarialImprovement** | Finding edge cases, improving robustness, anticipating objections |
| Structured workflow process | **RoleBasedWorkflow** | Research projects, content creation, multi-stage analysis |
| Stabilizing volatile outputs | **Bagging** | Reducing variance, improving prediction stability |
| Handling uncertainty | **UncertaintyBasedCollaboration** | Questions with ambiguity, calibrating confidence |
| Model stacking | **StackedGeneralization** | Leveraging strengths of different model types, boosting performance |

### Aggregation Strategies

| When you need... | Use this aggregation strategy | Best for |
|------------------|------------------------------|----------|
| To prioritize some models over others | **WeightedVoting** | When certain models perform better for specific tasks |
| To use the final result of a sequence | **SequentialRefinement** | When using phases that progressively refine content |
| To choose the most confident output | **ConfidenceBased** | When models have reliable confidence estimation |
| To evaluate along multiple criteria | **MultidimensionalVoting** | Complex evaluation requiring different quality dimensions |
| To blend multiple perspectives | **EnsembleFusion** | Creating a coherent synthesis from diverse inputs |
| Dynamic strategy selection | **AdaptiveSelection** | When different queries benefit from different aggregation approaches |

[Back to top](#ai-ensemble-suite)

## Collaboration Phases

AI Ensemble Suite implements various collaboration phases that can be combined or used individually:

### Core Phases

- ✅ **AsyncThinking**: Models work independently on a problem before combining insights
- ✅ **Integration/Refinement**: Models refine responses based on feedback and insights
- ✅ **ExpertCommittee**: Final processing/structuring of model outputs before aggregation

### Structured Debate Patterns

- ✅ **StructuredCritique**: Models evaluate others' responses using structured formats
- ✅ **SynthesisOriented**: Models focus on finding common ground and integrating perspectives
- ✅ **RoleBasedDebate**: Models interact according to assigned specialized roles

### Advanced Collaboration Methods

- ✅ **HierarchicalReview**: Content is progressively reviewed by models in a hierarchical structure
- ✅ **CompetitiveEvaluation**: Models are pitted against each other in a competition
- ✅ **PerspectiveRotation**: Models iterate on a problem by assuming different perspectives
- ✅ **ChainOfThoughtBranching**: Models trace through multiple reasoning paths
- ✅ **AdversarialImprovement**: Models improve a solution by seeking its weaknesses
- ✅ **RoleBasedWorkflow**: Models function in specialized roles like researcher, analyst, and writer
- ✅ **Bagging**: Models process different variations of the same input
- ✅ **UncertaintyBasedCollaboration**: Uncertainty measurements guide model interactions
- ✅ **StackedGeneralization**: Base models process input, then a meta-model combines their outputs

[Back to top](#ai-ensemble-suite)

## Aggregation Strategies

The library provides several strategies for aggregating the outputs from multiple models:

- ✅ **WeightedVoting**: Models are assigned different weights based on performance or expertise
- ✅ **SequentialRefinement**: Assumes phases run in a sequence where later phases refine earlier ones
- ✅ **ConfidenceBased**: Selects output with the highest confidence score
- ✅ **MultidimensionalVoting**: Evaluates outputs along multiple dimensions
- ✅ **EnsembleFusion**: Uses a model to synthesize multiple outputs into one
- ✅ **AdaptiveSelection**: Dynamically selects and executes another aggregation strategy

[Back to top](#ai-ensemble-suite)

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                              Ensemble                                                   │
│                                    (Main user-facing interface)                                         │
│ ┌─────────────────────────┐ ┌───────────────────────────┐ ┌────────────────────────────────────────────┐│
│ │ Attributes:             │ │ Core Methods:             │ │ Interaction Methods:                       ││
│ │ ● config_manager        │ │ ● __init__()              │ │ ● initialize() → ModelManager.initialize() ││
│ │ ● model_manager         │ │ ● ask()                   │ │ ● shutdown() → ModelManager.shutdown()     ││
│ │ ● template_manager      │ │ ● configure()             │ │ ● _execute_collaboration_phases()          ││
│ │ ● _initialized          │ │ ● __aenter__(),           │ │ ● _aggregate_results()                     ││
│ │ ● _initialization_lock  │ │   __aexit__()             │ │                                            ││
│ └─────────────────────────┘ └───────────────────────────┘ └────────────────────────────────────────────┘│
└─────────────────┬───────────────────────┬────────────────────────────┬──────────────────────────────────┘
                  │                       │                            │
                  │                       │                            │
          ┌───────▼──────┐       ┌────────▼─────────┐        ┌─────────▼────────┐
          │ ConfigManager│       │   ModelManager   │        │  TemplateManager │
          │ (YAML Config │       │   (GGUF Models)  │        │                  │
          └───────┬──────┘       └────────┬─────────┘        └──────────────────┘
                  │                       │
                  │                       │
                  │                       ▼
                  │            ┌────────────────────┐
                  │            │   Model Registry   │
                  │            └───────────┬────────┘
                  │                        │
         ┌────────▼──────────┐      ┌─────▼───────────────────────────────────┐
         │                   │      │                                         │
         ▼                   ▼      ▼                                         ▼
┌─────────────────┐  ┌──────────────────┐                   ┌─────────────────────────────────┐
│ BaseAggregator  │  │BaseCollaboration │                   │          TraceCollector         │
│  (Abstract)     │  │Phase (Abstract)  │                   │          Tracing System         │
└───┬──────┬──────┘  └────┬──────┬──────┘                   │(Records all collaboration steps)│
    │      │              │      │                          └─────────────────────────────────┘ 
    ▼      ▼              ▼      ▼                                       
┌─────────────────────────────┐  ┌────────────────────────────────────────────────────────────┐
│                             │  │                                                            │
│ Aggregation Implementations │  │              Collaboration Phase Implementations           │
│ ┌─────────────────────────┐ │  │ ┌────────────────┐ ┌─────────────────┐ ┌──────────────────┐│
│ │ WeightedVoting          │ │  │ │ AsyncThinking  │ │ChainOfThought   │ │BaseDebate        ││
│ ├─────────────────────────┤ │  │ ├────────────────┤ ├─────────────────┤ ├──────────────────┤│
│ │ SequentialRefinement    │ │  │ │ Integration    │ │CompetitiveEval  │ │StructuredCritique││
│ ├─────────────────────────┤ │  │ ├────────────────┤ ├─────────────────┤ ├──────────────────┤│
│ │ ConfidenceBased         │ │  │ │ ExpertCommittee│ │PerspectiveRot   │ │SynthesisOriented ││
│ ├─────────────────────────┤ │  │ ├────────────────┤ ├─────────────────┤ ├──────────────────┤│
│ │ MultidimensionalVoting  │ │  │ │ HierarchicalRev│ │AdversarialImp   │ │RoleBasedDebate   ││
│ ├─────────────────────────┤ │  │ └────────────────┘ ├─────────────────┤ └──────────────────┘│
│ │ EnsembleFusion          │ │  │                    │RoleBasedWorkflow│                     │
│ ├─────────────────────────┤ │  │                    └─────────────────┘                     │
│ │ AdaptiveSelection       │ │  │ ┌────────────────┐ ┌─────────────────┐ ┌────────────────┐  │
│ └─────────────────────────┘ │  │ │ Bagging        │ │UncertaintyBased │ │StackedGen      │  │
│                             │  │ └────────────────┘ └─────────────────┘ └────────────────┘  │
└─────────────────────────────┘  └────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                    Exception Hierarchy                                                  │
│                                                                                                         │
│          ┌─────────────────────────────────────────────────────────────────────────────┐                │
│          │                        AiEnsembleSuiteError (Base)                          │                │
│          └──────┬──────────────┬─────────────────┬────────────────┬────────────────────┘                │
│                 │              │                 │                │                                     │
│                 ▼              ▼                 ▼                ▼                                     │
│          ┌────────────┐ ┌─────────────┐ ┌───────────────┐ ┌─────────────┐                               │
│          │ModelError  │ │ConfigError  │ │CollabError    │ │AggrError    │                               │
│          └────────────┘ └──────┬──────┘ └───────────────┘ └─────────────┘                               │
│                                │                                                                        │
│                                ▼                                                                        │
│                         ┌────────────────┐                                                              │
│                         │ValidationError │                                                              │
│                         └────────────────┘                                                              │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

[Back to top](#ai-ensemble-suite)


## Class Hierarchy and Interaction Diagram

```
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                         │
│                                     ENSEMBLE CLASS                                      │
│                        (Main user-facing interface & orchestrator)                      │
│                                                                                         │
├─────────────────┬─────────────────────────────────────────┬─────────────────────────────┤
│                 │                                         │                             │
│   Public API:   │           Core Control Flow:            │      Context Management:    │
│   ● initialize()│   ● _execute_collaboration_phases()     │      ● __aenter__()         │
│   ● shutdown()  │   ● _aggregate_results()                │      ● __aexit__()          │
│   ● ask()       │   ● _get_phase_class()                  │      ● _initialization_lock │
│   ● configure() │                                         │                             │
│                 │                                         │                             │
└─────────┬───────┴─────────────────────┬───────────────────┴─────────────┬───────────────┘
          │                             │                                 │
          ▼                             ▼                                 ▼
┌─────────────────────────────┐  ┌─────────────────────────┐   ┌─────────────────────────────┐
│                             │  │                         │   │                             │
│       ConfigManager         │  │     ModelManager        │   │     TemplateManager         │
│      (Config handling)      │  │(Model loading/inference)│   │    (Prompt template         │
│                             │  │                         │   │     management)             │
│ ● load()                    │  │  ● initialize()         │   │  ● get_template()           │
│ ● update()                  │  │  ● shutdown()           │   │  ● render_template()        │
│ ● validate()                │  │  ● get_model()          │   │                             │
│ ● get_collaboration_mode()  │  │  ● run_inference()      │   └─────────────────────────────┘
│ ● get_aggregation_strategy()│  │                         │
│                             │  │                         │
└─────────┬───────────────────┘  └───────────┬─────────────┘
          │                                  │
          │                                  ▼
          │              ┌────────────────────────────────────────────────┐
          │              │                                                │
          │              │              Model Registry                    │
          │              │         (Loaded GGUF Models)                   │
          │              │                                                │
          └──────────────┼━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━┓
                         └────────────────────────────────────────────────┘         │
                                                                                    │
┌────────────────────────────────────────────────────────────────────┐              │
│                                                                    │              │
│                    COLLABORATION PHASES                            │              │
│                                                                    │              │
├──────────────────────┬─────────────────────┬───────────────────────┤              │
│                      │                     │                       │              │
│   Simple Phases:     │   Complex Phases:   │     Debate Types:     │              │
│   ● AsyncThinking    │   ● CompetitiveEval │   ● StructuredCritique│              │
│   ● Integration      │   ● PerspectiveRot  │   ● SynthesisOriented │              │
│   ● ExpertCommittee  │   ● ChainOfThought  │   ● RoleBasedDebate   │              │
│   ● HierarchicalRev  │   ● AdversarialImp  │                       │              │
│                      │   ● RoleBasedWork   │                       │              │
│                      │                     │                       │              │
├──────────────────────┴─────────────────────┴───────────────────────┤              │
│                                                                    │              │
│   Machine Learning-Oriented Phases:                                │              │
│   ● Bagging                                                        │              │
│   ● UncertaintyBasedCollaboration                                  │              │
│   ● StackedGeneralization                                          │              │
│                                                                    │              │
└────────────────────────────────┬───────────────────────────────────┘              │
                                 │                                                  │
                                 │                                                  │
                                 ▼                                                  │
┌────────────────────────────────────────────────────────────────────┐              │
│                                                                    │              │
│                     AGGREGATION STRATEGIES                         │◀─────────────┘
│                                                                    │
├──────────────────────┬─────────────────────┬───────────────────────┤
│                      │                     │                       │
│   ● WeightedVoting   │   ● ConfidenceBased │   ● EnsembleFusion    │
│   ● SequentialRef    │   ● MultiDimVoting  │   ● AdaptiveSelection │
│                      │                     │                       │
└──────────────────────┴─────────────────────┴───────────────────────┘

┌────────────────────────────────────────────────────────────────────┐
│                                                                    │
│                    UTILITY COMPONENTS                              │
│                                                                    │
├──────────────────────────────────┬─────────────────────────────────┤
│                                  │                                 │
│   ● TraceCollector               │   ● Exceptions:                 │
│     (Execution tracing)          │     - AiEnsembleSuiteError      │
│                                  │     - ConfigurationError        │
│   ● Logger                       │     - ModelError                │
│     (Structured logging)         │     - CollaborationError        │
│                                  │     - AggregationError          │
│                                  │     - ValidationError           │
│                                  │                                 │
└──────────────────────────────────┴─────────────────────────────────┘
```

[Back to top](#ai-ensemble-suite)

#