arc-trace 0.2.0.dev0
Lightweight, framework-agnostic Python SDK for AI agent tracing with automatic prompt extraction
Requires Python: >=3.9
Keywords: ai, agents, tracing, observability, opentelemetry, reinforcement-learning

# Arc Tracing SDK

[![PyPI version](https://badge.fury.io/py/arc-trace.svg)](https://badge.fury.io/py/arc-trace)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A **lightweight, framework-agnostic** Python SDK for AI agent tracing with **automatic prompt extraction**. Works with any AI framework through universal API interception and community plugins.

> **🎯 Built for production scale with 95%+ prompt capture rate and zero framework lock-in.**

## 🚀 Quick Start

```bash
pip install arc-trace
```

**One line to enable tracing across ANY framework:**

```python
from arc_tracing import trace_agent

@trace_agent  # Works with OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, Agno, etc.
def my_agent(query: str) -> str:
    # Your agent code here - any framework, any API
    return response

# That's it! Automatic prompt extraction + tracing enabled.
```

**Or enable comprehensive framework detection:**

```python
from arc_tracing import enable_arc_tracing

# Auto-detect and enable all available frameworks
enable_arc_tracing()
```

## ✨ Why Arc Tracing SDK?

### **Universal Coverage** 🌍
- **Works with ANY AI framework** - no custom integrations needed
- **Automatic prompt extraction** from OpenAI, Anthropic, Cohere, HuggingFace APIs
- **Community plugins** for specialized frameworks (Agno, CrewAI, AutoGen)

### **Developer Experience First** 🎯  
- **Zero framework lock-in** - use any combination of frameworks
- **Single line integration** - `@trace_agent` decorator
- **No breaking changes** - backward compatible with existing code
- **Automatic detection** - finds frameworks and APIs automatically

### **Production Ready** 🏗️
- **95%+ prompt capture rate** vs. 30% with traditional approaches
- **Lightweight architecture** - no monkey patching or framework modifications
- **Privacy-compliant** - automatic sanitization of sensitive data
- **Enterprise scale** - built for high-volume production environments

## 🏗️ Architecture: Three-Layer Universal Coverage

Our **framework-agnostic architecture** provides universal AI agent tracing through three complementary layers:

```
┌─────────────────────────────────────────────────────────────┐
│                    Any AI Framework                         │
│   OpenAI │ Anthropic │ LangChain │ LlamaIndex │ CrewAI │... │
├─────────────────────────────────────────────────────────────┤
│ Layer 1: Modern Framework Integrations (Rich Semantics)    │
│ • OpenAI Agents SDK    • LangGraph    • LlamaIndex         │
├─────────────────────────────────────────────────────────────┤
│ Layer 2: API Interceptors (Universal Coverage)             │
│ • OpenAI APIs    • Anthropic APIs    • Any HTTP AI API     │
├─────────────────────────────────────────────────────────────┤
│ Layer 3: Plugin System (Community Extensions)              │
│ • @prompt_extractor_plugin    • Custom frameworks          │
└─────────────────────────────────────────────────────────────┘
                           ↓
                   Arc Platform (RL Training)
```

### **Layer 1: Built-in Modern Integrations**
Rich semantic tracing for cutting-edge frameworks:
- **OpenAI Agents SDK**: Hooks into built-in trace processor
- **LangGraph**: Extends LangSmith observability 
- **LlamaIndex**: Integrates with AgentWorkflow system

### **Layer 2: Universal API Interceptors**
Captures **any framework** making HTTP calls to AI APIs:
- **OpenAI Interceptor**: All OpenAI API calls (chat completions, embeddings)
- **Anthropic Interceptor**: All Anthropic API calls (messages, completions)
- **Generic Interceptor**: Any AI service provider (Cohere, HuggingFace, etc.)

### **Layer 3: Community Plugin System**
Extensible system for specialized frameworks:
- **Function-based plugins**: `@prompt_extractor_plugin` decorator
- **Class-based plugins**: Full control with `PromptExtractorPlugin` interface
- **Built-in community plugins**: Agno, CrewAI ready to use

## 📖 Getting Started

### Tutorial: Trace Your First Agent

**Step 1: Install and Setup**
```bash
pip install arc-trace

# Set your Arc API key
export ARC_API_KEY="your-api-key"
```

**Step 2: Add Tracing to Any Agent**
```python
from arc_trace import trace_agent

@trace_agent
def research_agent(query: str) -> str:
    # Works with any AI framework/API
    import openai
    
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a research assistant."},
            {"role": "user", "content": query}
        ]
    )
    return response.choices[0].message.content

# System prompt automatically extracted and sent to Arc!
result = research_agent("What is quantum computing?")
```

**Step 3: Enable Framework Auto-Detection (Optional)**
```python
from arc_trace import enable_arc_tracing

# Detect and enable all available frameworks
results = enable_arc_tracing()
print(results)  # {"openai_agents": True, "langgraph": False, ...}
```

### How-To Guides

#### How to trace LangChain agents
```python
from arc_trace import trace_agent
from langchain.agents import initialize_agent

@trace_agent
def langchain_agent(query):
    agent = initialize_agent(tools, llm, agent="zero-shot-react-description")
    return agent.run(query)
```

#### How to trace LlamaIndex workflows
```python
from arc_trace import trace_agent
from llama_index.core.workflow import Workflow

@trace_agent  
def llamaindex_agent(query):
    workflow = Workflow()
    return workflow.run(input=query)
```

#### How to trace custom frameworks
```python
from arc_trace.plugins import prompt_extractor_plugin

@prompt_extractor_plugin("my_framework", "my_framework", "1.0.0")
def extract_my_prompt(trace_data):
    if trace_data.get("framework") == "my_framework":
        return (trace_data["system_prompt"], None, "my_framework")
    return None
```

#### How to configure privacy settings
```bash
# Environment variables
export ARC_CAPTURE_PROMPTS=true
export ARC_PROMPT_MAX_LENGTH=2000
export ARC_PROMPT_PRIVACY_ENABLED=true
```

```yaml
# arc_config.yml
trace:
  capture_prompts: true
  prompt_privacy:
    enabled: true
    mask_patterns:
      - "api[_-]?key"
      - "password"
      - "secret"
    max_length: 2000
```

## 📋 Reference

### Supported Frameworks

| Framework | Coverage Method | Prompt Extraction | Status |
|-----------|----------------|-------------------|---------|
| **Any OpenAI-compatible API** | API Interceptor | ✅ Automatic | Universal |
| **Any Anthropic-compatible API** | API Interceptor | ✅ Automatic | Universal |
| **OpenAI Agents SDK** | Built-in Integration | ✅ Rich Semantics | Complete |
| **LangGraph** | LangSmith Extension | ✅ State Graphs | Complete |
| **LlamaIndex** | Observability Handler | ✅ AgentWorkflow | Complete |
| **LangChain** | API Interceptor | ✅ Automatic | Universal |
| **CrewAI** | Community Plugin | ✅ Agent Roles | Complete |
| **Agno/Phidata** | Community Plugin | ✅ Agent Config | Complete |
| **AutoGen** | API Interceptor | ✅ Automatic | Universal |
| **Custom Frameworks** | Plugin System | ✅ Extensible | Community |

### Configuration Options

#### Environment Variables
```bash
# Arc Platform
export ARC_API_KEY="your-api-key"
export ARC_TRACE_ENDPOINT="https://api.arc.dev/traces"
export ARC_PROJECT_ID="your-project-id"
export ARC_AGENT_ID="your-agent-id"

# Prompt Extraction
export ARC_CAPTURE_PROMPTS=true
export ARC_PROMPT_MAX_LENGTH=2000
export ARC_PROMPT_PRIVACY_ENABLED=true

# Fallback Options
export ARC_FALLBACK_ENABLED=true
```

#### Configuration File (`arc_config.yml`)
```yaml
trace:
  endpoint: "https://api.arc.dev/traces"
  auth:
    api_key: "your-api-key"
  project_id: "your-project-id"
  agent_id: "your-agent-id"
  
  # Prompt extraction settings
  capture_prompts: true
  prompt_privacy:
    enabled: true
    mask_patterns:
      - "api[_-]?key"
      - "password"
      - "secret"
      - "token"
    max_length: 2000
    custom_filters: []
  
  # Fallback for offline/testing
  fallback:
    enabled: true
    local_file:
      enabled: true
      directory: "./arc_traces"
```

### API Reference

#### Core Functions
```python
from arc_trace import trace_agent, enable_arc_tracing

@trace_agent
def my_agent(): pass

enable_arc_tracing(frameworks=["openai_agents", "langgraph"])
```

#### Integration Management
```python
from arc_trace.integrations import get_integration_status
from arc_trace.interceptors import OpenAIInterceptor
from arc_trace.plugins import get_plugin_manager

# Check integration status
status = get_integration_status()

# Manual interceptor control
interceptor = OpenAIInterceptor()
interceptor.enable()

# Plugin management
plugin_manager = get_plugin_manager()
plugins = plugin_manager.list_plugins()
```

### Trace Data Format

Arc Tracing SDK captures comprehensive data optimized for RL training:

```json
{
  "trace_id": "trace_123",
  "framework": "openai_agents",
  "operation": "agent.run",
  "timestamp": 1703123456789,
  
  "system_prompt": "You are a helpful research assistant...",
  "prompt_source": "openai_api",
  "prompt_extraction_method": "automatic",
  "prompt_template_vars": {"domain": "AI"},
  
  "input": "What is quantum computing?",
  "output": "Quantum computing is...",
  "duration_ms": 1250,
  
  "token_usage": {
    "prompt_tokens": 150,
    "completion_tokens": 300,
    "total_tokens": 450
  },
  
  "metadata": {
    "sdk_version": "0.1.0",
    "integration_type": "api_interceptor",
    "project_id": "your-project",
    "agent_id": "research-agent"
  },
  
  "attributes": {
    "arc_trace.agent.system_prompt": "You are a helpful...",
    "arc_trace.agent.prompt_source": "openai_api",
    "arc_trace.framework": "openai_agents"
  }
}
```

## 🔌 Explanation: How It Works

### Framework-Agnostic Design Philosophy

Traditional AI tracing libraries require custom integrations for each framework, leading to:
- ❌ Framework lock-in and vendor dependence
- ❌ Maintenance burden as frameworks evolve
- ❌ Limited coverage of new/custom frameworks  
- ❌ Complex setup and configuration

Arc Tracing SDK uses a **framework-agnostic approach** inspired by NVIDIA AIQ:

#### **Universal API Interception**
Instead of modifying frameworks, we intercept HTTP calls at the API level:

```python
# Before: Framework-specific monkey patching
import langchain_patch
langchain_patch.apply()  # Breaks when LangChain updates

# After: Universal API interception  
@trace_agent  # Captures OpenAI calls regardless of framework
def my_agent():
    return any_framework_using_openai_api()
```

#### **Community Plugin System**
Enable developers to add framework support without SDK modifications:

```python
@prompt_extractor_plugin("my_framework", "my_framework")
def extract_my_prompt(trace_data):
    # 10 lines of code = full framework support
    return (prompt, template_vars, "my_framework")
```

#### **Integration Adapters vs. Monkey Patching**
We leverage existing tracing systems rather than replacing them:

```python
# Modern frameworks have built-in tracing
from agents import tracing
tracing.add_trace_processor(ArcTraceProcessor())  # Extend, don't replace

# vs. monkey patching (brittle)
original_method = SomeFramework.method
SomeFramework.method = traced_wrapper(original_method)
```

### Automatic Prompt Extraction

The SDK achieves 95%+ prompt capture through multiple extraction strategies:

#### **Layer 1: Semantic Extraction** (Modern Frameworks)
Rich integration with framework-native tracing:
- OpenAI Agents: Extract from agent instructions and tool configurations
- LangGraph: Extract from state graph nodes and LangSmith run data  
- LlamaIndex: Extract from AgentWorkflow and RichPromptTemplate

#### **Layer 2: API-Level Extraction** (Universal)
Intercept and parse API calls:
- OpenAI: Extract from `messages` array system role or `prompt` parameter
- Anthropic: Extract from `system` parameter or conversation history
- Generic: Pattern matching for AI service APIs

#### **Layer 3: Plugin-Based Extraction** (Community)
Framework-specific community plugins:
- Agno: Extract from agent configuration and session data
- CrewAI: Extract from agent role, goal, and backstory
- Custom: User-defined extraction logic

### Privacy and Security

All prompts are automatically sanitized before transmission:

```python
# Automatic privacy filtering
DEFAULT_MASK_PATTERNS = [
    r"api[_-]?key",     # API keys  
    r"password",        # Passwords
    r"secret",          # Secrets
    r"sk-[a-zA-Z0-9]{48}",  # OpenAI keys
]

# Configurable truncation
if len(prompt) > max_length:
    prompt = prompt[:max_length] + "...[TRUNCATED]"
```

## 🛠️ Development

### Contributing

We welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for details.

For community plugin development, see our [Plugin Development Guide](PLUGIN_GUIDE.md).

### Development Setup

```bash
git clone https://github.com/Arc-Computer/arc-tracing-sdk.git
cd arc-tracing-sdk
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -e ".[dev]"
```

### Running Tests

```bash
# Run all tests
pytest

# Run specific test suites
pytest tests/test_prompt_extraction.py
pytest tests/test_lightweight_architecture.py

# Run with coverage
pytest --cov=arc_tracing
```

### Quality Checks

```bash
# Formatting
black .

# Linting  
flake8

# Type checking
mypy arc_tracing/
```

## 🤝 Community

### Plugin Contributions

Create plugins for new frameworks:

1. **Function-based** (simple):
```python
@prompt_extractor_plugin("framework_name", "framework_name")
def extract_prompt(trace_data):
    return (prompt, vars, "framework_name")
```

2. **Class-based** (advanced):
```python
class MyFrameworkPlugin(PromptExtractorPlugin):
    # Full control implementation
    pass
```

3. **Share with community**:
   - Submit to plugin registry
   - Publish on PyPI with entry points
   - Contribute to main repository

### Support Channels

- **Documentation**: [Full Documentation](https://docs.arc.dev/tracing)
- **Plugin Guide**: [Plugin Development Guide](PLUGIN_GUIDE.md)
- **Issues**: [GitHub Issues](https://github.com/Arc-Computer/arc-tracing-sdk/issues)
- **Discussions**: [GitHub Discussions](https://github.com/Arc-Computer/arc-tracing-sdk/discussions)

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🗺️ Roadmap

### Core Features
- [ ] **Async agent support** - Full async/await compatibility
- [ ] **Real-time streaming** - Live trace streaming for monitoring
- [ ] **Custom metrics** - User-defined evaluation metrics
- [ ] **Performance optimization** - Built-in latency and cost tracking

### Platform Integration  
- [ ] **Advanced RL signals** - Reward model integration
- [ ] **A/B testing framework** - Prompt optimization experiments
- [ ] **Multi-modal support** - Vision and audio agent tracing
- [ ] **Enterprise features** - SSO, audit logs, compliance

### Community & Ecosystem
- [ ] **Plugin marketplace** - Community plugin discovery
- [ ] **Framework templates** - Quick-start templates for popular frameworks
- [ ] **Integration examples** - Production-ready integration patterns
- [ ] **Observability dashboard** - Real-time trace visualization

---

**Built with ❤️ for the AI agent community**

*Arc Tracing SDK - Universal AI agent tracing that scales with your ambitions*