cortex-intelligence 0.1.0
A three-layer intelligence engine with ONNX integration for Open Agent Spec
Requires Python: >=3.8
Keywords: ai, intelligence, onnx, open-agent-spec, machine-learning, neural-networks

# Cortex Intelligence Engine 🧠⚡

[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Tests](https://img.shields.io/badge/tests-20%20passed-brightgreen.svg)](tests/)

A **three-layer intelligence engine** designed for integration with [Open Agent Spec](https://github.com/openagents/openagents), providing advanced sensory processing, interpretation, and reasoning capabilities with **intelligent filtering and cost optimization**.

## 🎯 What is Cortex?

Cortex acts as an **intelligent filtering layer** between your OAS agents and expensive external LLMs. Instead of routing every request directly to external LLMs, Cortex:

1. **Processes input through Layer 1** (sensory processing)
2. **Analyzes with Layer 2** (internal intelligence - rule-based or ONNX models)
3. **Decides whether to trigger Layer 3** (external LLM) based on complexity and importance
4. **Provides intelligent responses** with suggested actions

**Result**: 60-80% cost reduction while maintaining intelligence! 💰🧠

## 🏗️ Architecture

Cortex implements a three-layer architecture:

```
OAS Agent → Cortex Intelligence Engine → External LLM (if needed)
                ↓
            Layer 1: Sensory Processing (Text, Image, Audio)
                ↓
            Layer 2: Internal Intelligence (Rule-based/ONNX/Local LLM)
                ↓
            Layer 3: External LLM (OpenAI, Claude, etc.)
```

### Layer Breakdown

1. **Layer 1 - Sensory Processing**: Raw data processing for vision, audio, and text inputs
2. **Layer 2 - Interpretation**: Onboard ONNX/LLM for base-level data interpretation and reaction decisions
3. **Layer 3 - Reasoning**: External LLM vendor integration (OpenAI, Claude, etc.) for complex reasoning

## ✨ Features

- **🧠 Intelligent Filtering**: Automatically decides when to use expensive external LLMs
- **🤖 ONNX Integration**: Local neural network models for intelligent decision-making
- **💰 Cost Optimization**: 60-80% reduction in external API calls
- **⚡ Performance**: Sub-second responses for simple queries
- **🔄 Multi-modal Processing**: Handles text, images, and audio data
- **🎯 Smart Routing**: Determines when complex reasoning is needed
- **🔌 Flexible LLM Integration**: Supports multiple LLM providers with fallback mechanisms
- **🤖 Open Agent Spec Compatible**: Drop-in replacement for standard OAS intelligence engines
- **📊 Performance Monitoring**: Built-in metrics and status tracking
- **📦 Batch Processing**: Efficient processing of multiple inputs
- **⚙️ Configurable Processing Modes**: Reactive, proactive, and selective processing

## 🚀 Installation

```bash
# Clone the repository
git clone https://github.com/yourusername/cortex.git
cd cortex

# Install in development mode
pip install -e .
```

### 📋 Dependencies

- **Python 3.8+**
- **numpy** - Numerical computing
- **Pillow** - Image processing
- **onnxruntime** - Layer 2 ONNX models
- **aiohttp** - Async HTTP requests
- **pydantic** - Data validation

## ⚡ Quick Start

### Basic Usage

```python
import asyncio
from cortex import create_simple_cortex, create_cortex_function

async def main():
    # Create a basic Cortex instance
    cortex = create_simple_cortex(
        openai_api_key="your_openai_key",  # Optional
        claude_api_key="your_claude_key",  # Optional
        enable_layer3=True
    )
    
    # Process some input
    result = await cortex.process_input(
        data="Analyze this urgent system alert!",
        context="System monitoring detected anomaly"
    )
    
    print(f"Response: {result.final_response}")
    print(f"Suggested actions: {result.suggested_actions}")

asyncio.run(main())
```

### 🤖 Open Agent Spec Integration

```python
from cortex import create_cortex_oas_intelligence, create_cortex_oas_function

# Configure like standard OAS intelligence
config = {
    "type": "cortex",
    "config": {
        "processing_mode": "reactive",
        "layer2_threshold": 0.6,
        "enable_layer3": True,
        "external_engine": "openai",
        "external_model": "gpt-4"
    }
}

# Create OAS-compatible intelligence engine
intelligence = create_cortex_oas_intelligence(config)
oas_function = create_cortex_oas_function(intelligence)

# Use in your OAS agent
response = await oas_function(
    prompt="URGENT: System failure detected!",
    context={"user_id": "123", "session_id": "456"}
)

print(f"Triggered Layer 3: {response['metadata']['triggered_layer3']}")
```

📖 **For detailed OAS integration guide, see [docs/OAS_INTEGRATION.md](docs/OAS_INTEGRATION.md)**

### 🧠 ONNX Integration

```python
from cortex import create_cortex_oas_intelligence, CortexOASConfig

# Configure with ONNX models for intelligent filtering
config = CortexOASConfig(
    layer2_engine="onnx",  # Enable ONNX models
    layer2_threshold=0.6,  # Confidence threshold
    enable_layer3=True,    # Allow external LLM calls
    external_engine="openai",
    external_model="gpt-4"
)

# Create intelligence engine with ONNX-powered Layer 2
intelligence = create_cortex_oas_intelligence(config)

# Process input - ONNX models will decide if Layer 3 is needed
response = await intelligence.process("URGENT: System failure detected!")
print(f"Triggered Layer 3: {response['metadata']['triggered_layer3']}")
```

📖 **For detailed ONNX integration guide, see [docs/ONNX_INTEGRATION.md](docs/ONNX_INTEGRATION.md)**

## 📚 Usage Examples

### 📝 Text Processing

```python
result = await cortex.process_input(
    data="Emergency: Database server is down!",
    sense_type="text",
    task_type="incident_response"
)
```

### 🖼️ Image Processing

```python
from PIL import Image

image = Image.open("photo.jpg")
result = await cortex.process_input(
    data=image,
    sense_type="vision",
    context="Security camera feed analysis"
)
```

### 🎵 Audio Processing

```python
import numpy as np

# Audio data as numpy array
audio_data = np.load("audio_sample.npy")
result = await cortex.process_input(
    data=audio_data,
    sense_type="audio",
    task_type="sound_classification"
)
```

### 📦 Batch Processing

```python
inputs = [
    {"data": "Message 1", "task_type": "general"},
    {"data": "URGENT: Alert message", "task_type": "alert"},
    {"data": image_data, "sense_type": "vision"}
]

results = await cortex.batch_process(inputs)
```

## ⚙️ Configuration

### 🔄 Processing Modes

| Mode | Behavior | Use Case |
|------|----------|----------|
| **REACTIVE** | Only Layer 3 when Layer 2 decides | Cost-effective, intelligent filtering |
| **PROACTIVE** | Always use all layers | Maximum intelligence, higher cost |
| **SELECTIVE** | Custom logic for Layer 3 | Custom filtering rules |

```python
from cortex.core import CortexConfig, ProcessingMode

config = CortexConfig(
    processing_mode=ProcessingMode.REACTIVE,
    layer2_threshold=0.6,
    enable_layer3=True,
    max_processing_time=30.0
)

cortex = Cortex(config)
```

### 🤖 LLM Provider Configuration

```python
from cortex.layers.layer3 import LLMConfig, LLMProvider

# OpenAI Configuration
openai_config = LLMConfig(
    provider=LLMProvider.OPENAI,
    api_key="your_key",
    model="gpt-3.5-turbo",
    max_tokens=1000,
    temperature=0.7
)

# Claude Configuration
claude_config = LLMConfig(
    provider=LLMProvider.CLAUDE,
    api_key="your_key",
    model="claude-3-sonnet-20240229",
    max_tokens=1000,
    temperature=0.7
)

cortex.add_llm_provider(openai_config, is_primary=True)
cortex.add_llm_provider(claude_config, is_primary=False)  # Fallback
```

## 🤖 Open Agent Spec Integration

Cortex is designed as a **drop-in replacement** for standard OAS intelligence engines:

### Basic OAS Configuration

```yaml
name: "my_cortex_agent"
intelligence:
  type: "cortex"
  engine: "cortex-hybrid"
  config:
    processing_mode: "reactive"
    layer2_threshold: 0.6
    enable_layer3: true
    external_engine: "openai"
    external_model: "gpt-4"
    external_endpoint: "https://api.openai.com/v1"
    temperature: 0.7
    max_tokens: 150
    trigger_keywords: ["help", "urgent", "error", "assist"]
```

### Response Format

Cortex returns OAS-compatible responses:

```json
{
  "success": true,
  "response": "Analysis and response from Cortex",
  "actions": ["suggested_action_1", "suggested_action_2"],
  "metadata": {
    "layers_used": ["layer1", "layer2", "layer3"],
    "processing_time": 1.23,
    "confidence": 0.85,
    "cortex_stats": {
      "total_requests": 100,
      "layer2_only": 70,
      "layer3_triggered": 30,
      "average_response_time": 0.5
    },
    "triggered_layer3": true
  },
  "error": null
}
```

📖 **For comprehensive OAS integration guide, see [docs/OAS_INTEGRATION.md](docs/OAS_INTEGRATION.md)**

## 📊 Monitoring and Metrics

### Performance Statistics

```python
# Get current status
status = cortex.get_status()
print(f"Performance metrics: {status['cortex']['performance_metrics']}")

# Get processing history
history = cortex.get_recent_history(limit=10)

# Reset metrics
cortex.reset_metrics()
```

### 📈 Example Output

```
Performance Statistics:
  Total Requests: 100
  Layer 2 Only: 70 (70.0%)
  Layer 3 Triggered: 30 (30.0%)
  Average Response Time: 0.5s
```

## 🔔 Callback System

```python
async def on_high_priority(data):
    print(f"High priority reaction detected: {data}")

cortex.add_callback("on_reaction_decision", on_high_priority)
```

## 🛠️ Development

### 🧪 Running Tests

```bash
pip install pytest pytest-asyncio
python -m pytest tests/ -v
```

### 🚀 Running Examples

```bash
# Basic usage examples
python examples/basic_usage.py

# OAS integration examples
python examples/oas_integration_example.py
```

### 📁 Project Structure

```
cortex/
├── cortex/
│   ├── __init__.py
│   ├── core.py              # Main Cortex class
│   ├── oas_integration.py   # OAS integration module
│   └── layers/
│       ├── __init__.py
│       ├── layer1.py        # Sensory processing
│       ├── layer2.py        # Interpretation and reaction decisions
│       └── layer3.py        # External LLM reasoning
├── examples/
│   ├── basic_usage.py       # Basic usage examples
│   └── oas_integration_example.py  # OAS integration examples
├── docs/
│   └── OAS_INTEGRATION.md   # Detailed OAS integration guide
├── tests/
│   └── test_basic.py        # Test suite
├── pyproject.toml           # Project configuration
└── README.md
```

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run the test suite
6. Submit a pull request

## 📄 License

MIT License - see LICENSE file for details.

## 📚 API Reference

### 🧠 Core Classes

- `Cortex`: Main intelligence engine
- `SenseLayer`: Layer 1 sensory processing
- `InterpretationLayer`: Layer 2 interpretation and reaction decisions
- `ReasoningLayer`: Layer 3 external LLM reasoning

### 🛠️ Utility Functions

- `create_simple_cortex()`: Quick Cortex setup
- `create_cortex_function()`: Create Open Agent Spec compatible function
- `create_cortex_oas_intelligence()`: Create OAS intelligence engine
- `create_cortex_oas_function()`: Create OAS-compatible function

### ⚙️ Configuration Classes

- `CortexConfig`: Main configuration
- `CortexOASConfig`: OAS-specific configuration
- `LLMConfig`: LLM provider configuration
- `ProcessingMode`: Processing mode enumeration

## 🎯 Use Cases

### 💰 Cost Optimization
- **Problem**: High LLM API costs for simple queries
- **Solution**: Cortex filters out simple requests
- **Result**: 60-80% cost reduction

### ⚡ Response Speed
- **Problem**: Slow responses for simple questions
- **Solution**: Layer 2 handles simple queries instantly
- **Result**: Sub-second responses for basic requests

### 🧠 Intelligent Routing
- **Problem**: All requests treated equally
- **Solution**: Cortex analyzes complexity and urgency
- **Result**: Appropriate resource allocation

### 🔄 Multi-modal Support
- **Problem**: Text-only intelligence engines
- **Solution**: Cortex handles text, images, and audio
- **Result**: Rich multi-modal interactions

---

**Cortex** - Intelligent filtering for Open Agent Spec agents. 🧠⚡