npm package: @mdb-devx/ai-gateway
Description: The AI Gateway Client is a universal (browser & Node.js) TypeScript module that provides seamless integration with the AI Gateway. Quickly and securely integrate Generative AI models from OpenAI (and soon to be others) into your client and server applicat
Latest version: 0.3.3

# AI Gateway Client

The AI Gateway Client is a universal (browser & Node.js) TypeScript module that provides seamless integration with the AI Gateway. Quickly and securely integrate Generative AI models from OpenAI (and soon to be others) into your client and server applications.

The following OpenAI endpoints and models are supported:

- [Chat completions](https://platform.openai.com/docs/api-reference/chat)
- [Embeddings](https://platform.openai.com/docs/api-reference/embeddings)
- [Legacy completions](https://platform.openai.com/docs/api-reference/completions)

Chat Models:

- `gpt-3.5-turbo`
- `gpt-3.5-turbo-0613`
- `gpt-3.5-turbo-16k`
- `gpt-3.5-turbo-16k-0613`
- `gpt-4`
- `gpt-4-0613`
- `gpt-4-32k`
- `gpt-4-32k-0613`

Embedding Models:

- `text-embedding-ada-002`

Legacy Completion Models:

- `gpt-3.5-turbo`

## Installation

To install the AI Gateway Client package:

```bash
npm install @mdb-devx/ai-gateway
```

## Usage

### Creating the client:

```typescript
import { AIGatewayClient } from "@mdb-devx/ai-gateway";
const client = new AIGatewayClient({
  url: "https://example.com",
  jwt: "SOME_OKTA_JWT_TOKEN",
});
```

### Using the client:

#### Chat Completions:

```typescript
import { Chat, RequestOpts } from '@mdb-devx/ai-gateway';

const chat: Chat = await client.chat({
  model: "gpt-3.5-turbo-0613",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the captial of Canada?" },
  ],
  max_token: 10,
  temperature: 0.2,
}, {}: RequestOpts);
const result = completion.choices[0].message.content;
```

#### Embeddings:

```typescript
import { Embedding, RequestOpts } from '@mdb-devx/ai-gateway';

const embedding: Embedding = await client.embedding({
  input: "The food was delicious and the waiter...",
  model: "text-embedding-ada-002",
}, {}: RequestOpts);
```

#### Legacy Completions:

```typescript
import { Completion, RequestOpts } from '@mdb-devx/ai-gateway';

const completion: Completion = await client.completion({
  model: "gpt-3.5-turbo", // Older legacy completion models (e.g. text-davinci-003) are unsupported.
  prompt: "The capital of Canada is",
  max_token: 10,
  temperature: 0.2,
}, {}: RequestOpts);
const result = completion.choices[0].text;
```

#### Stream completions:

Both chat and legacy completions support streaming responses as the model produces tokens, so the client provides separate methods (`client.chatStream()` and `client.completionStream()`) to deeply integrate with the stream lifecycle.

```typescript
import {ChatChunk, EventSourceMessage, StreamOpts} from '@mdb-devx/ai-gateway'; // or CompletionChunk

const ctrl = new AbortController();
await client.chatStream(
  {
    model: "gpt-3.5-turbo-0613",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "What is the captial of Canada?" },
    ],
    stream: true,
    max_token: 10,
    temperature: 0.2,
  },
  {
    onopen(response: Response) {
      // Called when a response is received. If not provided, will default to
      // a basic validation to ensure the content-type is text/event-stream.
    },
    onmessage(data: ChatChunk, event: EventSourceMessage) {
      // Called on each message. Response data is parsed from JSON event.data
    },
    onerror(error: unknown) {
      // Rethrow to stop the operation.
      // Do nothing to automatically retry.
      // You can also return a specific retry interval here.
    },
    onclose() {
      // Called when a response finishes. If you don't expect the server to kill
      // the connection, you can throw an exception here and retry using onerror.
    },
    signal: ctrl.signal, // Stream can be cancelled elsewhere with ctrl.abort()
  }: StreamOpts
);
```