npm package: @gizmo-ai/model-providers
Description: LLM provider implementations for Gizmo (OpenAI, Anthropic, Mistral)
Latest version: 0.2.1

# @gizmo-ai/model-providers

Model provider implementations for Gizmo agents.

## Overview

This package provides LLM provider implementations that integrate with `@gizmo-ai/agent-plugin`. Each provider transforms between Gizmo's internal conversation format and the provider's API format, handling both synchronous and streaming responses.

## Installation

```bash
bun add @gizmo-ai/model-providers
```

## Available Providers

### OpenAI

Full support for OpenAI models including GPT-4, GPT-4o, and o1 reasoning models.

```typescript
import { openAI } from '@gizmo-ai/model-providers';

// Simple usage with model name
const provider = openAI('gpt-4o-mini');

// Full configuration
const provider = openAI({
  model: 'gpt-4o',
  apiKey: 'sk-...', // defaults to process.env.OPENAI_API_KEY
  temperature: 0.7,
  maxTokens: 1000,
  baseUrl: 'https://api.openai.com/v1', // optional custom endpoint
  contextWindowSize: 128000, // default: 128000
});

// For o1 models with reasoning
const provider = openAI({
  model: 'o1-preview',
  reasoning: 'high', // 'low' | 'medium' | 'high'
});
```

**Features:**
- ✅ Synchronous generation (`generate()`)
- ✅ Streaming generation (`generateStream()`)
- ✅ Function/tool calling
- ✅ Reasoning output (o1 models)
- ✅ Token usage tracking
- ✅ Full test coverage (11 tests)

### Anthropic

Full support for Anthropic Claude models including Claude 3.5 Sonnet, Claude 3 Opus/Sonnet/Haiku, and Claude 4 models with extended thinking.

```typescript
import { anthropic } from '@gizmo-ai/model-providers';

// Simple usage with model name
const provider = anthropic('claude-3-5-sonnet-20241022');

// Full configuration
const provider = anthropic({
  model: 'claude-3-5-sonnet-20241022',
  apiKey: 'sk-ant-...', // defaults to process.env.ANTHROPIC_API_KEY
  temperature: 0.7,
  maxTokens: 4096,
  contextWindowSize: 200000, // default: 200000
});

// For models with extended thinking
const provider = anthropic({
  model: 'claude-sonnet-4-5-20250929',
  thinking: {
    enabled: true,
    budgetTokens: 10000, // minimum: 1024
  },
});
```

**Features:**
- ✅ Synchronous generation (`generate()`)
- ✅ Streaming generation (`generateStream()`)
- ✅ Function/tool calling
- ✅ Extended thinking support (Claude 3.7+/4+)
- ✅ Token usage tracking
- ✅ Full test coverage (12 tests)

### Grok (Coming Soon)

xAI Grok support is planned.

## Provider Interface

All providers implement the `ModelProvider` interface from `@gizmo-ai/agent-plugin`:

```typescript
interface ModelProvider {
  /**
   * Generate a response synchronously
   */
  generate(
    conversation: ConversationItem[],
    tools: Tool[]
  ): Promise<ModelResponse>;

  /**
   * Generate a response with streaming (optional)
   *
   * Providers that support streaming should implement this method.
   * The middleware will automatically use streaming when available.
   */
  generateStream?(
    conversation: ConversationItem[],
    tools: Tool[],
    onToken: (chunk: { type: "content" | "function_call"; contentChunk: string }) => void,
    onReasoning: (chunk: { type: "reasoning"; reasoningChunk: string }) => void,
    signal?: AbortSignal
  ): Promise<ModelResponse>;
}
```

## Conversation Format

Providers transform between Gizmo's unified conversation format and provider-specific APIs:

```typescript
type ConversationItem =
  | { type: "message"; role: "user" | "assistant" | "system"; content: string }
  | { type: "function_call"; id: string; name: string; args: Record<string, unknown> }
  | { type: "function_call_output"; call_id: string; output: string }
  | { type: "reasoning"; summary: string };
```

## Streaming

When a provider implements `generateStream()`, the agent middleware automatically uses it and dispatches streaming delta actions:

- `AGENT_MODEL_STREAM_STARTED` - Streaming begins
- `AGENT_MODEL_STREAM_CONTENT_DELTA` - Content chunk received
- `AGENT_MODEL_STREAM_REASONING_DELTA` - Reasoning chunk received (o1 models)
- `AGENT_MODEL_STREAM_COMPLETED` - Streaming finished

**Example: Observing streaming state**

```typescript
import { createRuntime } from '@gizmo-ai/runtime';
import { agentPlugin } from '@gizmo-ai/agent-plugin';
import { openAI } from '@gizmo-ai/model-providers';

const agent = agentPlugin({
  model: openAI('gpt-4o-mini')
});

const runtime = createRuntime({ plugins: [agent] });

// Subscribe to state changes to see streaming updates
runtime.subscribe(() => {
  const state = runtime.getState();

  if (state.agent.isStreaming) {
    console.log('Current message:', state.agent.currentMessage);
    console.log('Current reasoning:', state.agent.currentReasoning);
  }
});

runtime.execute("Write a haiku about programming");
```

See `packages/examples/src/streaming-response.ts` for a full example.

## Usage with Agent

```typescript
import { createRuntime } from '@gizmo-ai/runtime';
import { agentPlugin } from '@gizmo-ai/agent-plugin';
import { openAI } from '@gizmo-ai/model-providers';

const agent = agentPlugin({
  model: openAI('gpt-4o-mini'),
  tools: [/* your tools */],
  systemMessage: "You are a helpful assistant",
  maxLoops: 10,
});

const runtime = createRuntime({
  plugins: [agent],
});

const handle = runtime.execute("Hello!");
await handle.completed;

console.log(runtime.getState().agent.conversation);
```

## Testing

Run the test suite:

```bash
bun test
```

Current coverage:
- ✅ OpenAI provider (11 tests)
- ✅ Anthropic provider (12 tests)
- ⏳ Grok provider (not yet implemented)

## Architecture

### Helper Functions

The OpenAI provider uses extracted helper functions to eliminate code duplication:

- `transformConversationToOpenAI()` - Converts internal format to OpenAI messages
- `transformToolsToOpenAI()` - Converts Tool[] to OpenAI function definitions
- `buildRequestParams()` - Builds request configuration
- `transformOpenAIResponseToConversation()` - Converts OpenAI response to internal format

### Streaming Implementation

Streaming providers:
1. Accept `onToken` and `onReasoning` callbacks
2. Call callbacks for each chunk as it arrives
3. Return the final `ModelResponse` when complete

The agent middleware:
1. Detects if `generateStream()` exists
2. Dispatches delta actions when callbacks fire
3. Updates `state.agent.currentMessage` and `state.agent.currentReasoning`
4. Clears streaming state on completion

## Roadmap

See [TODO.md](./TODO.md) for the complete roadmap including:
- Missing providers (Anthropic, Grok)
- Error handling improvements (retries, rate limits, timeouts)
- Advanced features (prompt caching, vision/multimodal, cost tracking)
- Provider abstraction (fallback, registry, orchestration)

## Contributing

When implementing a new provider:

1. Create `src/<provider>.ts` implementing `ModelProvider`
2. Add factory function (e.g., `anthropic()`)
3. Export from `src/index.ts`
4. Add tests in `src/__tests__/<provider>.test.ts`
5. Update this README
6. Update TODO.md

See `src/openai.ts` as a reference implementation.

## License

MIT