npm package: @layr-labs/ai-gateway-provider
Description: Vercel AI SDK provider for EigenCloud AI Gateway with JWT authentication
Author: Layr Labs
Latest version: 1.0.1

# EigenCloud AI Gateway Provider for Vercel AI SDK

A custom provider for the [Vercel AI SDK](https://sdk.vercel.ai) that connects to the EigenCloud AI Gateway with JWT authentication.

## Features

- **Configured via env vars**: BaseURL read from env vars, no configuration required
- **TEE Attestation Auth**: Automatic JWT via TEE attestation when `KMS_SERVER_URL` and `KMS_PUBLIC_KEY` are set
- **OpenAI-Compatible**: Works with any OpenAI-compatible API
- **Full AI SDK Support**: Compatible with all Vercel AI SDK features (streaming, tool calls, structured output, etc.)
- **Image & Video Generation**: Built-in support for image and video generation models
- **TypeScript**: Fully typed for excellent developer experience

## Installation

```bash
npm install @layr-labs/ai-gateway-provider ai
```

## Quick Start

```typescript
import { eigen } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

// Use with AI SDK functions
const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'What is the capital of France?',
});

console.log(result.text);
```

## Usage

### Basic Text Generation

```typescript
import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const { text } = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Explain quantum computing in simple terms.',
});
```

### Streaming Responses

```typescript
import { streamText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await streamText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Write a short story about a robot.',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}
```

### Structured Output / JSON Mode

Request structured JSON responses with a schema:

```typescript
import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Extract the name and age from: "John is 30 years old."',
  responseFormat: {
    type: 'json',
    schema: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        age: { type: 'number' },
      },
    },
  },
});

console.log(JSON.parse(result.text)); // { name: "John", age: 30 }
```

### Using Multiple Models

```typescript
import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

// Use different models
const sonnet = eigen('anthropic/claude-sonnet-4.6');
const gpt4 = eigen('openai/gpt-4o');

const [result1, result2] = await Promise.all([
  generateText({ model: sonnet, prompt: 'Hello' }),
  generateText({ model: gpt4, prompt: 'Hello' }),
]);
```

### Image Generation

```typescript
import { generateImage } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await generateImage({
  model: eigen.image('dall-e-3'),
  prompt: 'A cat wearing a space helmet',
});

console.log(result.image); // base64 image data
```

### Video Generation

```typescript
import { eigen } from '@layr-labs/ai-gateway-provider';

const model = eigen.video('google/veo-3.0-generate-001');
// Video generation uses SSE streaming and returns video data
```

### Tool Calls

```typescript
import { generateText, tool } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';
import { z } from 'zod';

const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'What is the weather in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get the weather for a location',
      parameters: z.object({
        location: z.string().describe('The location to get weather for'),
      }),
      execute: async ({ location }) => {
        // Your weather API call here
        return { temperature: 72, condition: 'sunny' };
      },
    }),
  },
});
```

### Without Authentication

If your gateway doesn't require authentication, you can omit the JWT:

```typescript
import { createEigenGateway } from '@layr-labs/ai-gateway-provider';

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
});
```

### Custom Headers

Add custom headers to all requests:

```typescript
import { createEigenGateway } from '@layr-labs/ai-gateway-provider';

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  headers: {
    'X-Custom-Header': 'value',
    'X-Organization-Id': 'org-123',
  },
});
```

### Debug Mode

Enable debug logging to see request details:

```typescript
const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  debug: true, // Logs requests and responses
});
```

You can also enable debug mode globally via the `DEBUG=true` environment variable.

### Custom Timeout

Set a custom request timeout:

```typescript
const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  timeout: 60000, // 60 seconds
});
```

Default timeouts: 30s for chat, 60s for images, 120s for video.

## Authentication

In a TEE environment, authentication is automatic. The provider uses attestation to obtain and refresh JWTs when `KMS_SERVER_URL` and `KMS_PUBLIC_KEY` are set (these are exported automatically by the compute environment).

You can also provide a JWT manually via the `jwt` config option or `KMS_AUTH_JWT` env var to bypass attestation.

### Using Environment Variables

The pre-configured `eigen` export automatically reads from environment variables:

```bash
# .env
EIGEN_GATEWAY_URL=https://ai-gateway-dev.eigencloud.xyz
# KMS_AUTH_JWT=optional-jwt-override
# DEBUG=true
```

```typescript
import { eigen } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

// Automatically configured from environment
const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Hello!',
});
```

## Configuration Options

### `EigenGatewayProviderConfig`

| Option | Type | Required | Default | Description |
|--------|------|----------|---------|-------------|
| `baseURL` | `string` | Yes | - | Base URL of your Eigen Gateway |
| `jwt` | `string` | No | - | Optional JWT override (bypasses attestation) |
| `attestConfig` | `AttestClientConfig` | No | - | Config for attestation-based JWT auth (auto-configured from env in TEE) |
| `headers` | `Record<string, string>` | No | `{}` | Additional headers for all requests |
| `timeout` | `number` | No | `30000` | Request timeout in milliseconds |
| `fetch` | `typeof fetch` | No | `globalThis.fetch` | Custom fetch implementation |
| `debug` | `boolean` | No | `false` | Enable debug logging |

## Examples

### Next.js API Route

```typescript
// app/api/chat/route.ts
import { createEigenGateway } from '@layr-labs/ai-gateway-provider';
import { streamText } from 'ai';

const eigenGateway = createEigenGateway({
  baseURL: process.env.EIGEN_GATEWAY_URL!,
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: eigenGateway('anthropic/claude-sonnet-4.6'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

### React Component

```typescript
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: '/api/chat',
  });

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          <strong>{m.role}:</strong> {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

### Node.js Script

```typescript
import { createEigenGateway } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

async function main() {
  const eigenGateway = createEigenGateway({
    baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  });

  const result = await generateText({
    model: eigenGateway('anthropic/claude-sonnet-4.6'),
    prompt: 'Hello, world!',
  });

  console.log(result.text);
}

main().catch(console.error);
```

## Provider Class

For reusable configuration across multiple model types, use the `EigenGatewayProvider` class:

```typescript
import { EigenGatewayProvider } from '@layr-labs/ai-gateway-provider';

const provider = new EigenGatewayProvider({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
});

const claude = provider.model('anthropic/claude-sonnet-4.6');
const dalle = provider.image('dall-e-3');
const veo = provider.video('google/veo-3.0-generate-001');
```

## TypeScript Support

This package is written in TypeScript and provides full type definitions:

```typescript
import type {
  EigenGatewayProviderConfig,
  ModelRequestOptions,
} from '@layr-labs/ai-gateway-provider';
```

## Error Handling

The provider throws descriptive errors for common issues:

```typescript
try {
  const result = await generateText({
    model: eigen('anthropic/claude-sonnet-4.6'),
    prompt: 'Hello',
  });
} catch (error) {
  if (error.message.includes('401')) {
    console.error('Authentication failed - check your JWT token');
  } else if (error.message.includes('429')) {
    console.error('Rate limit exceeded');
  } else {
    console.error('Request failed:', error);
  }
}
```

## Related

- [Vercel AI SDK Documentation](https://sdk.vercel.ai)