ashr-labs 0.1.0
Python SDK for the Ashr Labs voice agent testing platform
Requires Python: >=3.10
Keywords: voice-agent, testing, sdk, ashr, ai, quality-assurance

# Ashr Labs Python SDK

A Python client library for interacting with the Ashr Labs API.

## Documentation

- [Quick Start Guide](docs/quickstart.md)
- [Installation](docs/installation.md)
- [Authentication](docs/authentication.md)
- [API Reference](docs/api-reference.md)
- [Error Handling](docs/error-handling.md)
- [Examples](docs/examples.md)

## Installation

```bash
pip install ashr-labs
```

Or install from source:

```bash
cd sdk
pip install -e .
```

## Quick Start

```python
from ashr_labs import AshrLabsClient

# Initialize the client with your API key
client = AshrLabsClient(
    api_key="tp_your_api_key_here",
    base_url="https://api.ashr.io/testing-platform-api"
)

# Initialize session and get user/tenant info
session = client.init()
print(f"Logged in as: {session['user']['email']}")
print(f"Tenant: {session['tenant']['name']}")

# List datasets
response = client.list_datasets(tenant_id=1)
for dataset in response["datasets"]:
    print(f"Dataset: {dataset['name']} (ID: {dataset['id']})")

# Get a specific dataset with signed URLs for media files
dataset = client.get_dataset(
    dataset_id=42,
    include_signed_urls=True,
    url_expires_seconds=3600
)

# Build a test run incrementally as the agent executes
from ashr_labs import RunBuilder

run_builder = RunBuilder()
run_builder.start()

test = run_builder.add_test("bank_analysis")
test.start()
test.add_user_text(text="Analyze this statement", description="User prompt")
test.add_tool_call(
    expected={"tool_name": "extract_pdf", "arguments": {"file": "a.pdf"}},
    actual={"tool_name": "extract_pdf", "arguments": {"file": "a.pdf"}},
    match_status="exact",
)
test.add_agent_response(
    expected_response={"summary": "Expected..."},
    actual_response={"summary": "Actual..."},
    match_status="similar",
    semantic_similarity=0.89,
)
test.complete()
run_builder.complete()

# Deploy the run to the API
run_builder.deploy(client, tenant_id=1, dataset_id=42)

# Create a request
request = client.create_request(
    tenant_id=1,
    requestor_id=5,
    request_name="Generate Audio",
    request={
        "text": "Hello world",
        "voice": "alloy"
    }
)
print(f"Created request: {request['id']}")
```

## Available Methods

### Datasets

| Method | Description |
|--------|-------------|
| `get_dataset(dataset_id, include_signed_urls, url_expires_seconds)` | Get a dataset by ID |
| `list_datasets(tenant_id, limit, offset, include_signed_urls)` | List datasets for a tenant |

### Runs

| Method | Description |
|--------|-------------|
| `create_run(tenant_id, dataset_id, result, runner_id)` | Create a new test run |
| `get_run(run_id)` | Get a run by ID |
| `list_runs(tenant_id, dataset_id, limit, offset)` | List runs |
| `delete_run(run_id)` | Delete a run |

### RunBuilder

| Method | Description |
|--------|-------------|
| `RunBuilder()` | Create a new run builder |
| `run.start()` | Mark the run as started |
| `run.add_test(test_id)` | Add a test and get a `TestBuilder` |
| `run.complete(status)` | Mark the run as completed |
| `run.build()` | Serialize to a result dict |
| `run.deploy(client, tenant_id, dataset_id)` | Build and submit via the API |

### TestBuilder

| Method | Description |
|--------|-------------|
| `test.start()` | Mark the test as started |
| `test.add_user_file(file_path, description)` | Record a user file upload |
| `test.add_user_text(text, description)` | Record a user text input |
| `test.add_tool_call(expected, actual, match_status)` | Record an agent tool call |
| `test.add_agent_response(expected_response, actual_response, match_status)` | Record an agent response |
| `test.complete(status)` | Mark the test as completed |

### Requests

| Method | Description |
|--------|-------------|
| `create_request(tenant_id, requestor_id, request_name, request, request_input_schema)` | Create a new request |
| `get_request(request_id)` | Get a request by ID |
| `list_requests(tenant_id, status, limit, offset)` | List requests for a tenant |

### API Keys

| Method | Description |
|--------|-------------|
| `list_api_keys(include_inactive)` | List API keys for your tenant |
| `revoke_api_key(api_key_id)` | Revoke an API key |

### Session

| Method | Description |
|--------|-------------|
| `init()` | Initialize session and get user/tenant info |

### Utility

| Method | Description |
|--------|-------------|
| `health_check()` | Check if the API is reachable |

## Error Handling

The SDK raises specific exceptions for different error types:

```python
from ashr_labs import (
    AshrLabsClient,
    AuthenticationError,
    AuthorizationError,
    NotFoundError,
    ValidationError,
    RateLimitError,
    ServerError,
)

client = AshrLabsClient(api_key="tp_...", base_url="https://...")

try:
    dataset = client.get_dataset(dataset_id=999)
except AuthenticationError:
    print("Invalid API key")
except AuthorizationError:
    print("You don't have access to this resource")
except NotFoundError:
    print("Dataset not found")
except ValidationError as e:
    print(f"Invalid request: {e.message}")
except RateLimitError:
    print("Rate limit exceeded, please retry later")
except ServerError:
    print("Server error, please try again")
```

## API Key Permissions

API keys have access to a limited set of operations:

- **Datasets**: Read-only access (get, list)
- **Runs**: Full access (create, get, list, delete)
- **Requests**: Create and read access
- **API Keys**: List and revoke (cannot create new keys via API)

To create new API keys, use the web interface with OAuth authentication.

## Configuration

### Timeout

You can configure the request timeout (default: 30 seconds):

```python
client = AshrLabsClient(
    api_key="tp_...",
    base_url="https://...",
    timeout=60  # 60 seconds
)
```

## Requirements

- Python 3.10+
- No external dependencies (uses only standard library)

## License

MIT