langchain-oso 0.1.0
Oso observability integration for LangChain agents
License: Apache-2.0
Requires Python: <4.0,>=3.10
Keywords: langchain, observability, oso, monitoring

# langchain-oso

Oso observability integration for LangChain agents.

Callback handler that automatically captures and sends all LangChain agent events to Oso's observability platform for monitoring, debugging, and security analysis.

## Installation

```bash
pip install langchain-oso
```

Or with poetry:

```bash
poetry add langchain-oso
```

## Quick Start

```python
from langchain_oso import OsoObservabilityCallback
from langchain.agents import create_openai_tools_agent, AgentExecutor
from langchain_openai import ChatOpenAI

# Create the callback (reads OSO_AUTH_TOKEN from environment)
callback = OsoObservabilityCallback(
    agent_id="my-support-agent"
)

# Add to your agent
llm = ChatOpenAI(model="gpt-4")
agent = create_openai_tools_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, callbacks=[callback])

# Use your agent - all events are automatically captured
result = await agent_executor.ainvoke({"input": "Hello, how can I help?"})

# Clean up
await callback.aclose()
```

### Using Context Manager (Recommended)

```python
async with OsoObservabilityCallback(agent_id="my-agent") as callback:
    agent_executor = AgentExecutor(agent=agent, tools=tools, callbacks=[callback])
    result = await agent_executor.ainvoke({"input": "Hello"})
    # Automatic cleanup when context exits
```

## Configuration

### Environment Variables

Set these in your environment or `.env` file:

```bash
# Required: Your Oso authentication token
OSO_AUTH_TOKEN=your-token-here

# Optional: Custom Oso endpoint (defaults to https://cloud.osohq.com/api/events)
OSO_ENDPOINT=https://cloud.osohq.com/api/events

# Optional: Enable/disable observability (defaults to true)
OSO_OBSERVABILITY_ENABLED=true
```

### Constructor Parameters

```python
OsoObservabilityCallback(
    endpoint="https://cloud.osohq.com/api/events",  # Oso endpoint URL
    auth_token="your-token",                         # Oso auth token
    enabled=True,                                    # Enable/disable sending events
    session_id="unique-session-id",                  # Group related conversations
    metadata={"user_id": "123", "env": "prod"},     # Custom metadata for all events
    agent_id="my-agent"                              # Agent identifier
)
```

All parameters are optional and fall back to environment variables or defaults.

## What Gets Captured

The callback automatically captures all LangChain events:

### LLM Events
- Model name and configuration
- Prompts sent to the LLM
- Generated responses
- Token usage (prompt, completion, total)
- Errors and failures

### Tool Events
- Tool name and description
- Input parameters
- Output/results
- Execution duration (milliseconds)
- Errors and stack traces

### Agent Events
- Agent reasoning and thought process
- Tool selection decisions
- Tool input parameters
- Final outputs
- Complete execution flow

### Chain Events
- Chain type and name
- Input parameters
- Output values
- Nested chain execution

### Execution Summary
At the end of each agent execution, a summary event is sent with:
- Total execution duration
- Number of LLM calls, tool calls, and agent steps
- Total token usage
- Error count
- Complete execution trace

## Event Structure

Every event sent to Oso has this structure:

```json
{
  "event_type": "tool.completed",
  "execution_id": "unique-execution-id",
  "session_id": "conversation-session-id",
  "timestamp": "2024-02-15T10:30:45.123Z",
  "agent_id": "my-agent",
  "data": { /* event-specific data */ },
  "metadata": { /* your custom metadata */ }
}
```

### Event Types

- `llm.started` / `llm.completed` / `llm.error`
- `tool.started` / `tool.completed` / `tool.error`
- `agent.action` / `agent.finished`
- `chain.started` / `chain.completed` / `chain.error`
- `execution.summary` - Final summary with all accumulated data

## Error Handling

The callback is designed to fail gracefully:

- Network errors or timeouts won't crash your agent
- Failed event sends are logged but don't interrupt execution
- 5-second timeout for all HTTP requests
- Comprehensive error logging for debugging

## Logging

The callback uses Python's standard logging. Configure it in your application:

```python
import logging

# See debug info about events being sent
logging.basicConfig(level=logging.DEBUG)

# Or just warnings and errors
logging.basicConfig(level=logging.WARNING)

# Or configure just this library
logging.getLogger("langchain_oso").setLevel(logging.DEBUG)
```

## Examples

### Basic Agent with Tools

```python
from langchain_oso import OsoObservabilityCallback
from langchain.agents import create_openai_tools_agent, AgentExecutor
from langchain_openai import ChatOpenAI
from langchain.tools import tool

@tool
def search_orders(customer_id: str) -> str:
    """Search for customer orders."""
    return f"Orders for {customer_id}: ORD001, ORD002"

async def main():
    async with OsoObservabilityCallback(agent_id="support-agent") as callback:
        llm = ChatOpenAI(model="gpt-4o-mini")
        tools = [search_orders]

        agent = create_openai_tools_agent(llm, tools, prompt)
        agent_executor = AgentExecutor(
            agent=agent,
            tools=tools,
            callbacks=[callback]
        )

        result = await agent_executor.ainvoke({
            "input": "Find orders for customer CUST001"
        })

        print(result["output"])
```

### With Custom Metadata

```python
callback = OsoObservabilityCallback(
    agent_id="support-agent",
    session_id="user-session-123",
    metadata={
        "user_id": "user-456",
        "environment": "production",
        "version": "1.2.3"
    }
)
```

### Multiple Agents in Same Session

```python
session_id = str(uuid.uuid4())

# Agent 1
async with OsoObservabilityCallback(
    agent_id="agent-1",
    session_id=session_id
) as callback1:
    result1 = await agent1_executor.ainvoke(...)

# Agent 2 - same session
async with OsoObservabilityCallback(
    agent_id="agent-2",
    session_id=session_id
) as callback2:
    result2 = await agent2_executor.ainvoke(...)
```

## Requirements

- Python 3.10+
- langchain-core >= 0.1.0
- httpx >= 0.24.0

## License

Apache License 2.0

## Support

- Documentation: https://www.osohq.com/docs
- Issues: https://github.com/osohq/langchain-oso/issues
- Website: https://www.osohq.com