npm package: @llmtracer/sdk
Description: See where your AI budget goes. Lightweight LLM cost tracking SDK.
Author: LLM Tracer
Latest version: 2.5.0

# @llmtracer/sdk

Track cost, latency, and token usage across OpenAI, Anthropic, and Google Gemini — in one line of code.

## Install

```bash
npm install @llmtracer/sdk
```

## Quick Start

```typescript
import llmtracer from '@llmtracer/sdk';

llmtracer.init({ apiKey: 'lt_...' });

// All OpenAI, Anthropic, and Google Gemini calls are now tracked automatically.
```

No wrappers, no manual instrumentation, no code changes. The SDK auto-patches your provider clients at import time.

View your dashboard at [llmtracer.dev](https://llmtracer.dev).

## What Gets Captured

Every LLM call is automatically tracked with:

- **Provider, model, tokens** (input + output), latency, cost
- **Google Gemini**: thinking tokens (2.5 models), tool tokens, cached tokens
- **Anthropic**: cache creation + read tokens
- **OpenAI**: reasoning tokens (o1/o3/o4), cached tokens
- **Caller file, function, and line number**
- **Auto-flush on process exit** via `process.on('beforeExit')`, `SIGINT`, and `SIGTERM`

## Environment Variable Pattern

```typescript
import llmtracer from '@llmtracer/sdk';

llmtracer.init({
  apiKey: process.env.LLMTRACER_API_KEY,
  debug: true, // prints token counts to console
});
```

## Multi-App Tracking

If you have multiple services sharing an API key, set `appName` to filter by application in the dashboard:

```typescript
llmtracer.init({ apiKey: 'lt_...', appName: 'billing-service' });
```

Or via environment variable:

```bash
export LLMTRACER_APP_NAME=billing-service
```

## Trace Context and Tags

```typescript
await llmtracer.trace({ tags: { feature: 'chat', user_id: 'u_sarah' } }, async () => {
  const response = await openai.chat.completions.create({ ... });
});
```

Tags appear in the dashboard's Breakdown page and Top Tags card. Use them to answer questions like "which user costs the most?" or "which feature should I optimize?"

### Tagging Patterns

| Pattern | Tag | Example |
|---------|-----|---------|
| Track cost by feature | `feature` | `"chat"`, `"search"`, `"summarize"` |
| Track cost by user | `user_id` | `"u_sarah"`, `"u_mike"` |
| Track cost by customer (B2B) | `customer` | `"acme-corp"`, `"initech"` |
| Track cost by conversation | `conversation_id` | `"conv_abc123"` |
| Track environment | `env` | `"production"`, `"staging"` |

## Supported Providers

| Provider | Package | Auto-patched |
|----------|---------|-------------|
| OpenAI | `openai` | ✅ |
| Anthropic | `@anthropic-ai/sdk` | ✅ |
| Google Gemini | `@google/genai` | ✅ |

## Serverless Usage

Auto-flush handles graceful shutdown in long-running processes. For serverless environments (Lambda, Cloud Functions, etc.), flush before returning:

```typescript
export async function handler(event) {
  const response = await openai.chat.completions.create({ ... });
  await llmtracer.flush();
  return response;
}
```

## Debug Mode

Enable `debug: true` to print token counts to the console:

```typescript
llmtracer.init({ apiKey: 'lt_...', debug: true });
```

```
[llmtracer] openai gpt-4o | 1,247 in → 384 out | $0.0094 | 1.2s
[llmtracer] anthropic claude-sonnet-4-5 | 2,100 in → 512 out (cache_read: 1,800) | $0.0031 | 0.8s
[llmtracer] google gemini-2.5-pro | 900 in → 280 out (thinking: 1,420) | $0.0067 | 2.1s
```

## Streaming Support

Streaming calls are instrumented automatically. Token counts are captured from the final chunk:

```typescript
const stream = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
  stream: true,
});

for await (const chunk of stream) {
  // use chunk as normal
}
```

## Configuration

| Option | Type | Default | Range | Description |
|---|---|---|---|---|
| `apiKey` | `string` | *required* | — | Your LLM Tracer API key (starts with `lt_`) |
| `appName` | `string` | `undefined` | — | Application name for multi-app filtering. Falls back to `LLMTRACER_APP_NAME` env var |
| `endpoint` | `string` | Production URL | — | Ingestion endpoint URL |
| `maxBatchSize` | `number` | `50` | 1–500 | Max events per HTTP request |
| `flushIntervalMs` | `number` | `5000` | 1000–60000 | Auto-flush interval in milliseconds |
| `maxQueueSize` | `number` | `1000` | 100–10000 | Max events in queue before dropping oldest |
| `maxRetries` | `number` | `3` | 0–10 | Max retry attempts for failed flushes |
| `sampleRate` | `number` | `1.0` | 0.0–1.0 | Sampling rate. `0.5` captures ~50% of events |
| `debug` | `boolean` | `false` | — | Enable debug logging to console |
| `skipExitHandlers` | `boolean` | `false` | — | Skip process exit handlers (for plugins/serverless) |

All numeric options are validated on `init()`. Out-of-range values are replaced with the default, and a warning is logged when `debug: true`.

## Reliability

The SDK is designed to never interfere with your application:

- **Never throws** — all internal errors are swallowed silently (enable `debug: true` for visibility)
- **Batching** — events are queued and sent in batches of `maxBatchSize`
- **Retry with backoff** — failed flushes are retried up to `maxRetries` times with exponential backoff (`min(1000 * 2^attempt, 30000)`) plus random jitter (0–1000ms)
- **Drop after retries** — after `maxRetries` consecutive failures, the batch is dropped to prevent unbounded memory growth
- **Queue overflow** — drops oldest events when the queue exceeds `maxQueueSize`
- **Sampling** — set `sampleRate` below 1.0 to reduce volume in high-throughput environments

## License

MIT