agentdrift 0.4.3
Detect when your AI agents get dumber. Open-source quality drift monitoring.
License: MIT
Requires Python: >=3.10
Keywords: agent, ai, drift, evaluation, monitoring, observability

# AgentDrift

[![CI](https://github.com/agentdrift-ai/agentdrift/actions/workflows/ci.yml/badge.svg)](https://github.com/agentdrift-ai/agentdrift/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/agentdrift.svg)](https://pypi.org/project/agentdrift/)
[![Python](https://img.shields.io/pypi/pyversions/agentdrift.svg)](https://pypi.org/project/agentdrift/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Detect when your AI agents get dumber.**

Open-source quality drift monitoring for AI agents. Track agent outputs over time, detect quality degradation, and get alerts before your agents go off the rails.

> **[Live Demo](https://agentdrift-demo.vercel.app)** | [API Docs](docs/API.md) | [Roadmap](ROADMAP.md) | [Contributing](CONTRIBUTING.md)

## The Problem

AI agents degrade over time. Context pollution, error accumulation, prompt drift, and model updates silently erode output quality. By the time you notice, your users already have.

## Quick Start

```bash
pip install agentdrift
```

```python
from agentdrift import track_agent

@track_agent("my-agent")
def my_agent(query: str) -> str:
    return llm.call(query)
```

Check drift status:

```bash
agentdrift status
```

```
┌──────────────────────────────────────┐
│         Agent Drift Status           │
├──────────┬──────────┬───────┬───────┤
│ Agent    │ Status   │ Score │ Change│
├──────────┼──────────┼───────┼───────┤
│ my-agent │ STABLE   │ 0.847 │ +2.1% │
│ qa-bot   │ DRIFTING │ 0.612 │-18.3% │
└──────────┴──────────┴───────┴───────┘
```

## How It Works

```
Agent Call → Log Trace → Score Quality → Detect Drift → Alert
                                              ↓
                                    Dashboard / OTel Export
```

1. **Track** — Log every agent input/output with the SDK
2. **Score** — LLM-as-Judge rates output quality (0.0 - 1.0)
3. **Compare** — Recent scores vs baseline period
4. **Alert** — Notify when quality drops below threshold

## Key Features

- **1-line integration** — `@track_agent` decorator or manual `tracker.log()`
- **Web dashboard** — Real-time quality trends, trace trees, golden tests management
- **Golden tests** — Regression testing from your best agent outputs
- **Alert webhooks** — Slack, Discord, Email on drift detection
- **Multi-agent tracing** — Parent/child trace relationships
- **OpenTelemetry export** — Jaeger, Zipkin, any OTLP backend
- **REST API** — Language-agnostic trace ingestion (`POST /api/traces`)
- **TypeScript SDK** — `@agentdrift/sdk` for Node.js agents
- **PostgreSQL** — Production-ready storage (`pip install agentdrift[postgres]`)
- **Docker + Helm** — Kubernetes deployment ready

## Usage

### Web Dashboard

```bash
pip install agentdrift[dashboard]
agentdrift serve        # API server on :8888
cd web && npm run dev   # Dashboard on :3000
```

### Manual Logging

```python
from agentdrift import AgentTracker

tracker = AgentTracker("my-agent")
tracker.log(input=query, output=result, latency_ms=1200)
```

### Multi-Agent Tracing

```python
orchestrator = AgentTracker("orchestrator")
worker = AgentTracker("worker")

parent = orchestrator.log(input="plan task", output="delegating")
worker.log(input="subtask", output="result", parent_trace_id=parent.trace_id)
```

### HTTP API

```bash
curl -X POST http://localhost:8888/api/traces \
  -H "Content-Type: application/json" \
  -d '{"agent_name": "my-agent", "input": "hello", "output": "world"}'
```

### TypeScript SDK

```typescript
import { AgentDriftClient, AgentTracker } from "@agentdrift/sdk";

const client = new AgentDriftClient({ baseUrl: "http://localhost:8888" });
const tracker = new AgentTracker("my-agent", client);
await tracker.log({ input: "hello", output: "world" });
```

### Golden Tests

```bash
agentdrift test init my-agent --count 10 --min-score 0.8
agentdrift test run my-agent --module my_app:my_agent --threshold 0.7
```

### Configuration

```bash
agentdrift init --config   # Generate agentdrift.toml
```

See [`examples/`](examples/) for more integration patterns.

## Deployment

```bash
# Docker
docker compose up

# Kubernetes
helm install agentdrift chart/agentdrift/

# Vercel (dashboard only)
cd web && vercel
```

## Documentation

| Resource | Description |
|----------|-------------|
| [API Reference](docs/API.md) | REST API endpoints |
| [Roadmap](ROADMAP.md) | Completed features & future plans |
| [Contributing](CONTRIBUTING.md) | Development setup & guidelines |
| [Changelog](https://github.com/agentdrift-ai/agentdrift/releases) | Release notes |

## License

MIT