npm package: @inference/sdk
Description: Thin OpenAI, Anthropic, and Vercel AI SDK adapters for routing requests through Inference.net observability and proxy infrastructure
Latest version: 1.0.4

# @inference/sdk

`@inference/sdk` is a thin adapter around the official `openai`,
`@anthropic-ai/sdk`, and Vercel AI SDK provider factories for calling the
Inference proxy with provider-native APIs.

The SDK does not implement transport, streaming, retries, or provider resource
internals. It instantiates the caller's official SDK class or AI SDK provider
factory with Inference defaults, adds Inference routing headers, patches the
methods that benefit from Inference-specific request options, and returns the
configured official client or AI SDK provider instance.

## Current scope

- OpenAI and Anthropic adapters
- Returns the configured official client directly
- Patched OpenAI methods:
  - `chat.completions.create`
  - `responses.create`
  - `completions.create`
  - `embeddings.create`
- Patched Anthropic methods:
  - `messages.create`
  - `messages.stream`
  - `messages.parse`
  - `messages.countTokens`
- AI SDK v6 provider adapters:
  - OpenAI-compatible language models and embeddings
  - Anthropic language models
  - per-call `providerOptions.inference` support for `task`, `metadata`, and
    `environment`
- All other official client methods remain available directly on the returned
  client

## Usage

### OpenAI-compatible providers

```ts
import OpenAI from "openai";
import { createInferenceClient } from "@inference/sdk";

const inference = createInferenceClient({
  openai: OpenAI,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
  environment: process.env.NODE_ENV,
  options: {
    maxRetries: 0,
  },
  proxy: {
    baseUrl: "https://openrouter.ai/api",
    apiKey: process.env.OPENROUTER_API_KEY!,
  },
});

const response = await inference.chat.completions.create(
  {
    model: "z-ai/glm-5-turbo",
    messages: [{ role: "user", content: "Hello" }],
  },
  {
    task: "contact-research",
    metadata: {
      userId: "123",
      workflow: "lead-enrichment",
    },
  },
);

const models = await inference.models.list();
```

### Anthropic

```ts
import Anthropic from "@anthropic-ai/sdk";
import { createInferenceClient } from "@inference/sdk";

const inference = createInferenceClient({
  anthropic: Anthropic,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
  environment: process.env.NODE_ENV,
  options: {
    maxRetries: 0,
  },
  proxy: {
    baseUrl: "https://api.anthropic.com",
    apiKey: process.env.ANTHROPIC_API_KEY!,
  },
});

const response = await inference.messages.create(
  {
    model: "claude-sonnet-4-6",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello" }],
  },
  {
    task: "contact-research",
    metadata: {
      userId: "123",
      workflow: "lead-enrichment",
    },
  },
);

const tokenCount = await inference.messages.countTokens(
  {
    model: "claude-sonnet-4-6",
    messages: [{ role: "user", content: "Hello" }],
  },
  {
    task: "contact-research",
  },
);
```

### Vercel AI SDK v6

```ts
import { createAnthropic } from "@ai-sdk/anthropic";
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { embed, generateObject, generateText, streamText } from "ai";
import { z } from "zod";

import {
  createInferenceProvider,
} from "@inference/sdk";

const inferenceOpenAI = createInferenceProvider({
  createOpenAICompatible,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
  environment: process.env.NODE_ENV,
  proxy: {
    baseUrl: "https://api.openai.com",
    apiKey: process.env.OPENAI_API_KEY!,
  },
});

const inferenceAnthropic = createInferenceProvider({
  createAnthropic,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
  environment: process.env.NODE_ENV,
  options: {},
  proxy: {
    baseUrl: "https://api.anthropic.com",
    apiKey: process.env.ANTHROPIC_API_KEY!,
  },
});

const textResult = await generateText({
  model: inferenceOpenAI.chatModel("gpt-4o-mini"),
  prompt: "Reply with exactly OK.",
  providerOptions: {
    inference: {
      task: "contact-research",
      metadata: {
        userId: "123",
      },
    },
  },
});

const embeddingResult = await embed({
  model: inferenceOpenAI.embeddingModel("text-embedding-3-small"),
  value: "sunny day at the beach",
  providerOptions: {
    inference: {
      task: "semantic-indexing",
    },
    openai: {
      dimensions: 512,
    },
  },
});

const objectResult = await generateObject({
  model: inferenceAnthropic.messages("claude-sonnet-4-6"),
  prompt: "Return an object with status set to OK.",
  providerOptions: {
    inference: {
      task: "structured-extraction",
      metadata: {
        userId: "user_123",
      },
    },
  },
  schema: z.object({
    status: z.string(),
  }),
});

const streamed = streamText({
  model: inferenceAnthropic("claude-sonnet-4-6"),
  prompt: "Reply with exactly OK.",
  providerOptions: {
    inference: {
      task: "streaming-chat",
    },
  },
});

for await (const chunk of streamed.textStream) {
  process.stdout.write(chunk);
}

console.log(textResult.text, embeddingResult.embedding.length, objectResult.object.status);
```

Use `providerOptions.inference.metadata` for Inference observability metadata.
Use `providerOptions.openai` or `providerOptions.anthropic` only for
provider-native request options that should be forwarded to that upstream SDK.
OpenAI-compatible AI SDK providers default `includeUsage` to `true` so streamed
responses include usage information for observability. Set
`options: { includeUsage: false }` if a provider rejects `stream_options`.

### Vercel AI SDK provider registry

```ts
import { createAnthropic } from "@ai-sdk/anthropic";
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { createProviderRegistry, embedMany, streamObject } from "ai";
import { z } from "zod";

import {
  createInferenceProvider,
} from "@inference/sdk";

const registry = createProviderRegistry({
  anthropic: createInferenceProvider({
    createAnthropic,
    observabilityApiKey: process.env.INFERENCE_API_KEY!,
    proxy: {
      baseUrl: "https://api.anthropic.com",
      apiKey: process.env.ANTHROPIC_API_KEY!,
    },
  }),
  openai: createInferenceProvider({
    createOpenAICompatible,
    observabilityApiKey: process.env.INFERENCE_API_KEY!,
    proxy: {
      baseUrl: "https://api.openai.com",
      apiKey: process.env.OPENAI_API_KEY!,
    },
  }),
});

const embeddings = await embedMany({
  model: registry.embeddingModel("openai:text-embedding-3-small"),
  values: ["sunny day at the beach", "rainy afternoon in the city"],
  providerOptions: {
    inference: {
      task: "semantic-indexing",
      metadata: {
        batchId: "embeddings-123",
      },
    },
  },
});

const streamedObject = streamObject({
  model: registry.languageModel("anthropic:claude-sonnet-4-6"),
  prompt: "Return an object with status set to OK.",
  schema: z.object({
    status: z.string(),
  }),
  providerOptions: {
    inference: {
      task: "structured-extraction",
      metadata: {
        requestId: "req_123",
      },
    },
  },
});

for await (const partialObject of streamedObject.partialObjectStream) {
  console.log(partialObject);
}

console.log(embeddings.embeddings.length);
```

### Provider cookbook

The SDK currently documents and tests these provider routes:

- OpenAI
- OpenRouter
- Anthropic
- Inference.net
- Cerebras
- Gemini (via the OpenAI-compatible path)

Any other OpenAI-compatible provider can use the same OpenAI/OpenAI-compatible
pattern by swapping `proxy.baseUrl` and the provider API key.

#### Official SDK adapters

```ts
import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";

import { createInferenceClient } from "@inference/sdk";

const common = {
  environment: process.env.NODE_ENV,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
};

const providers = {
  anthropic: createInferenceClient({
    ...common,
    anthropic: Anthropic,
    proxy: {
      apiKey: process.env.ANTHROPIC_API_KEY!,
      baseUrl: "https://api.anthropic.com",
    },
  }),
  cerebras: createInferenceClient({
    ...common,
    openai: OpenAI,
    proxy: {
      apiKey: process.env.CEREBRAS_API_KEY!,
      baseUrl: "https://api.cerebras.ai",
    },
  }),
  gemini: createInferenceClient({
    ...common,
    openai: OpenAI,
    proxy: {
      apiKey: process.env.GEMINI_API_KEY!,
      baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
    },
  }),
  inferenceNet: createInferenceClient({
    ...common,
    openai: OpenAI,
    proxy: {
      apiKey: process.env.INFERENCE_NET_API_KEY!,
      baseUrl: "https://api.inference.net",
    },
  }),
  openai: createInferenceClient({
    ...common,
    openai: OpenAI,
    proxy: {
      apiKey: process.env.OPENAI_API_KEY!,
      baseUrl: "https://api.openai.com",
    },
  }),
  openrouter: createInferenceClient({
    ...common,
    openai: OpenAI,
    proxy: {
      apiKey: process.env.OPENROUTER_API_KEY!,
      baseUrl: "https://openrouter.ai/api",
    },
  }),
};

const task = "provider-cookbook";

const openaiResult = await providers.openai.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Reply with exactly OK." }],
  },
  { task },
);

const openrouterResult = await providers.openrouter.responses.create(
  {
    input: "Reply with exactly OK.",
    model: "z-ai/glm-5-turbo",
  },
  { task },
);

const anthropicResult = await providers.anthropic.messages.create(
  {
    model: "claude-sonnet-4-6",
    max_tokens: 256,
    messages: [{ role: "user", content: "Reply with exactly OK." }],
  },
  { task },
);

const inferenceNetResult = await providers.inferenceNet.chat.completions.create(
  {
    model: "google/gemma-3-27b-instruct/bf-16",
    messages: [{ role: "user", content: "Reply with exactly OK." }],
  },
  { task },
);

const cerebrasResult = await providers.cerebras.chat.completions.create(
  {
    model: "llama-4-scout-17b-16e-instruct",
    messages: [{ role: "user", content: "Reply with exactly OK." }],
  },
  { task },
);

const geminiResult = await providers.gemini.chat.completions.create(
  {
    model: "gemini-2.5-flash",
    messages: [{ role: "user", content: "Reply with exactly OK." }],
  },
  { task },
);

console.log({
  anthropic: anthropicResult.content[0],
  cerebras: cerebrasResult.choices[0]?.message.content,
  gemini: geminiResult.choices[0]?.message.content,
  inferenceNet: inferenceNetResult.choices[0]?.message.content,
  openai: openaiResult.choices[0]?.message.content,
  openrouter: openrouterResult.output_text,
});
```

#### Vercel AI SDK adapters

```ts
import { createAnthropic } from "@ai-sdk/anthropic";
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

import {
  createInferenceProvider,
} from "@inference/sdk";

const common = {
  environment: process.env.NODE_ENV,
  observabilityApiKey: process.env.INFERENCE_API_KEY!,
};

const aiProviders = {
  anthropic: createInferenceProvider({
    ...common,
    createAnthropic,
    proxy: {
      apiKey: process.env.ANTHROPIC_API_KEY!,
      baseUrl: "https://api.anthropic.com",
    },
  }),
  cerebras: createInferenceProvider({
    ...common,
    createOpenAICompatible,
    proxy: {
      apiKey: process.env.CEREBRAS_API_KEY!,
      baseUrl: "https://api.cerebras.ai",
    },
  }),
  gemini: createInferenceProvider({
    ...common,
    createOpenAICompatible,
    proxy: {
      apiKey: process.env.GEMINI_API_KEY!,
      baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
    },
  }),
  inferenceNet: createInferenceProvider({
    ...common,
    createOpenAICompatible,
    proxy: {
      apiKey: process.env.INFERENCE_NET_API_KEY!,
      baseUrl: "https://api.inference.net",
    },
  }),
  openai: createInferenceProvider({
    ...common,
    createOpenAICompatible,
    proxy: {
      apiKey: process.env.OPENAI_API_KEY!,
      baseUrl: "https://api.openai.com",
    },
  }),
  openrouter: createInferenceProvider({
    ...common,
    createOpenAICompatible,
    proxy: {
      apiKey: process.env.OPENROUTER_API_KEY!,
      baseUrl: "https://openrouter.ai/api",
    },
  }),
};

const task = "provider-cookbook";

const openaiText = await generateText({
  model: aiProviders.openai.chatModel("gpt-4o-mini"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

const openrouterText = await generateText({
  model: aiProviders.openrouter.chatModel("z-ai/glm-5-turbo"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

const anthropicText = await generateText({
  model: aiProviders.anthropic.messages("claude-sonnet-4-6"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

const inferenceNetText = await generateText({
  model: aiProviders.inferenceNet.chatModel("google/gemma-3-27b-instruct/bf-16"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

const cerebrasText = await generateText({
  model: aiProviders.cerebras.chatModel("llama-4-scout-17b-16e-instruct"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

const geminiText = await generateText({
  model: aiProviders.gemini.chatModel("gemini-2.5-flash"),
  prompt: "Reply with exactly OK.",
  providerOptions: { inference: { task } },
});

console.log({
  anthropic: anthropicText.text,
  cerebras: cerebrasText.text,
  gemini: geminiText.text,
  inferenceNet: inferenceNetText.text,
  openai: openaiText.text,
  openrouter: openrouterText.text,
});
```

## Design constraints

- Uses the caller's installed `openai`, `@anthropic-ai/sdk`, AI SDK provider
  factories, and AI SDK Core package
- Treats Inference routing headers as adapter-owned
- Keeps unsupported endpoints available through the returned official client
- Keeps unsupported AI SDK model methods available through the returned provider
- Defaults to `https://api.inference.net`
- Expects upstream provider URLs in `proxy.baseUrl` root form (for example,
  `https://api.openai.com` or `https://api.anthropic.com`); a trailing `/v1`
  is normalized away for OpenAI-compatible and Anthropic overrides
- AI SDK adapters use `providerOptions.inference` for per-call `task`,
  `metadata`, and `environment`
- AI SDK image models are not wrapped in this release

## Installation

- Install `@inference/sdk`
- Install the provider SDK you plan to use:
  - `openai`
  - `@anthropic-ai/sdk`
- Install the AI SDK packages you plan to use:
  - `ai`
  - `@ai-sdk/openai-compatible`
  - `@ai-sdk/anthropic`
- Both provider SDKs are optional peer dependencies; you only need the one you
  initialize the adapter with

## Next milestones

- Add more wrapped OpenAI and Anthropic resource methods as backend support
  grows
- Add image-model wrapping once local and hosted image support is verified for
  the AI SDK adapters
- Add a CI matrix that validates compatibility across supported provider SDK
  versions