---
title: AI SDK by Vercel
description: Welcome to the AI SDK documentation!
---

# AI SDK

The AI SDK is the TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more.

## Why use the AI SDK?

Integrating large language models (LLMs) into applications is complicated and heavily dependent on the specific model provider you use.

- **[AI SDK Core](/docs/ai-sdk-core):** A unified API for generating text, structured objects, and tool calls with LLMs.
- **[AI SDK UI](/docs/ai-sdk-ui):** A set of framework-agnostic hooks for quickly building chat and generative user interface.

## Model Providers

The AI SDK supports [multiple model providers](/providers).

<OfficialModelCards />

## Templates

We've built some [templates](https://vercel.com/templates?type=ai) that include AI SDK integrations for different use cases, providers, and frameworks. You can use these templates to get started with your AI-powered application.

### Starter Kits

<Templates type="starter-kits" />

### Feature Exploration

<Templates type="feature-exploration" />

### Frameworks

<Templates type="frameworks" />

### Generative UI

<Templates type="generative-ui" />

### Security

<Templates type="security" />

## Join our Community

If you have questions about anything related to the AI SDK, you're always welcome to ask our community on [GitHub Discussions](https://github.com/vercel/ai/discussions).

---
title: Overview
description: An overview of foundational concepts critical to understanding the AI SDK
---

# Overview

<Note>
  This page is a beginner-friendly introduction to high-level artificial
  intelligence (AI) concepts. To dive right into implementing the AI SDK, feel
  free to skip ahead to our [quickstarts](/docs/getting-started) or learn about
  our [supported models and providers](/docs/foundations/providers-and-models).
</Note>

The AI SDK standardizes integrating artificial intelligence (AI) models across [supported providers](/docs/foundations/providers-and-models). This enables developers to focus on building great AI applications, not waste time on technical details.

For example, here’s how you can generate text with various models using the AI SDK:

<PreviewSwitchProviders />

To effectively leverage the AI SDK, it helps to familiarize yourself with the following concepts:

## Generative Artificial Intelligence

**Generative artificial intelligence** refers to models that predict and generate various types of outputs (such as text, images, or audio) based on what’s statistically likely, pulling from patterns they’ve learned from their training data. For example:

- Given a photo, a generative model can generate a caption.
- Given an audio file, a generative model can generate a transcription.
- Given a text description, a generative model can generate an image.

## Large Language Models

A **large language model (LLM)** is a subset of generative models focused primarily on **text**. An LLM takes a sequence of words as input and aims to predict the most likely sequence to follow. It assigns probabilities to potential next sequences and then selects one. The model continues to generate sequences until it meets a specified stopping criterion.

LLMs learn by training on massive collections of written text, which means they will be better suited to some use cases than others. For example, a model trained on GitHub data would understand the probabilities of sequences in source code particularly well.

However, it's crucial to understand LLMs' limitations. When asked about less known or absent information, like the birthday of a personal relative, LLMs might "hallucinate" or make up information. It's essential to consider how well-represented the information you need is in the model.

## Embedding Models

An **embedding model** is used to convert complex data (like words or images) into a dense vector (a list of numbers) representation, known as an embedding. Unlike generative models, embedding models do not generate new text or data. Instead, they provide representations of semantic and syntactic relationships between entities that can be used as input for other models or other natural language processing tasks.

In the next section, you will learn about the difference between models providers and models, and which ones are available in the AI SDK.

---
title: Providers and Models
description: Learn about the providers and models available in the AI SDK.
---

# Providers and Models

Companies such as OpenAI and Anthropic (providers) offer access to a range of large language models (LLMs) with differing strengths and capabilities through their own APIs.

Each provider typically has its own unique method for interfacing with their models, complicating the process of switching providers and increasing the risk of vendor lock-in.

To solve these challenges, AI SDK Core offers a standardized approach to interacting with LLMs through a [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1) that abstracts differences between providers. This unified interface allows you to switch between providers with ease while using the same API for all providers.

Here is an overview of the AI SDK Provider Architecture:

<MDXImage
  srcLight="/images/ai-sdk-diagram.png"
  srcDark="/images/ai-sdk-diagram-dark.png"
  width={800}
  height={800}
/>

## AI SDK Providers

The AI SDK comes with several providers that you can use to interact with different language models:

- [OpenAI Provider](/providers/ai-sdk-providers/openai) (`@ai-sdk/openai`)
- [Azure OpenAI Provider](/providers/ai-sdk-providers/azure) (`@ai-sdk/azure`)
- [Anthropic Provider](/providers/ai-sdk-providers/anthropic) (`@ai-sdk/anthropic`)
- [Amazon Bedrock Provider](/providers/ai-sdk-providers/amazon-bedrock) (`@ai-sdk/amazon-bedrock`)
- [Google Generative AI Provider](/providers/ai-sdk-providers/google-generative-ai) (`@ai-sdk/google`)
- [Google Vertex Provider](/providers/ai-sdk-providers/google-vertex) (`@ai-sdk/google-vertex`)
- [Mistral Provider](/providers/ai-sdk-providers/mistral) (`@ai-sdk/mistral`)
- [xAI Grok Provider](/providers/ai-sdk-providers/xai) (`@ai-sdk/xai`)
- [Cohere Provider](/providers/ai-sdk-providers/cohere) (`@ai-sdk/cohere`)
- [Groq](/providers/ai-sdk-providers/groq) (`@ai-sdk/groq`)

You can also use the OpenAI provider with OpenAI-compatible APIs:

- [Perplexity](/providers/ai-sdk-providers/perplexity)
- [Fireworks](/providers/ai-sdk-providers/fireworks)
- [LM Studio](/providers/openai-compatible-providers/lmstudio)
- [Baseten](/providers/openai-compatible-providers/baseten)

Our [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1) is published as an open-source package, which you can use to create [custom providers](/providers/community-providers/custom-providers).

The open-source community has created the following providers:

- [LLamaCpp Provider](/providers/community-providers/llama-cpp) (`nnance/llamacpp-ai-provider `)
- [Ollama Provider](/providers/community-providers/ollama) (`sgomez/ollama-ai-provider`)
- [ChromeAI Provider](/providers/community-providers/chrome-ai) (`jeasonstudio/chrome-ai`)
- [Portkey Provider](/providers/community-providers/portkey) (`@portkey-ai/vercel-provider`)
- [AnthropicVertex Provider](/providers/community-providers/anthropic-vertex-ai) (`nalaso/anthropic-vertex-ai`)

## Model Capabilities

The AI providers support different language models with various capabilities.
Here are the capabilities of popular models:

| Provider                                                                 | Model                        | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------------------------------------------------------------ | ---------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4o`                     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4o-mini`                | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4-turbo`                | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4`                      | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `o1-preview`                 | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `o1-mini`                    | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-sonnet-20241022` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-sonnet-20240620` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-haiku-20241022`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `pixtral-large-latest`       | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `mistral-large-latest`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `mistral-small-latest`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `pixtral-12b-2409`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Generative AI](/providers/ai-sdk-providers/google-generative-ai) | `gemini-1.5-flash`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Generative AI](/providers/ai-sdk-providers/google-generative-ai) | `gemini-1.5-pro`             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex)               | `gemini-1.5-flash`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex)               | `gemini-1.5-pro`             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [xAI Grok](/providers/ai-sdk-providers/xai)                              | `grok-beta`                  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [xAI Grok](/providers/ai-sdk-providers/xai)                              | `grok-vision-beta`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-405b-reasoning`   | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-70b-versatile`    | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-8b-instant`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `mixtral-8x7b-32768`         | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `gemma2-9b-it`               | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  This table is not exhaustive. Additional models can be found in the provider
  documentation pages and on the provider websites.
</Note>

---
title: Prompts
description: Learn about the Prompt structure used in the AI SDK.
---

# Prompts

Prompts are instructions that you give a [large language model (LLM)](/docs/foundations/overview#large-language-models) to tell it what to do.
It's like when you ask someone for directions; the clearer your question, the better the directions you'll get.

Many LLM providers offer complex interfaces for specifying prompts. They involve different roles and message types.
While these interfaces are powerful, they can be hard to use and understand.

In order to simplify prompting, the AI SDK support text, message, and system prompts.

## Text Prompts

Text prompts are strings.
They are ideal for simple generation use cases,
e.g. repeatedly generating content for variants of the same prompt text.

You can set text prompts using the `prompt` property made available by AI SDK functions like [`streamText`](/docs/reference/ai-sdk-core/stream-text) or [`generateObject`](/docs/reference/ai-sdk-core/generate-object).
You can structure the text in any way and inject variables, e.g. using a template literal.

```ts highlight="3"
const result = await generateText({
  model: yourModel,
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

You can also use template literals to provide dynamic data to your prompt.

```ts highlight="3-5"
const result = await generateText({
  model: yourModel,
  prompt:
    `I am planning a trip to ${destination} for ${lengthOfStay} days. ` +
    `Please suggest the best tourist activities for me to do.`,
});
```

## System Prompts

System prompts are the initial set of instructions given to models that help guide and constrain the models' behaviors and responses.
You can set system prompts using the `system` property.
System prompts work with both the `prompt` and the `messages` properties.

```ts highlight="3-6"
const result = await generateText({
  model: yourModel,
  system:
    `You help planning travel itineraries. ` +
    `Respond to the users' request with a list ` +
    `of the best stops to make in their destination.`,
  prompt:
    `I am planning a trip to ${destination} for ${lengthOfStay} days. ` +
    `Please suggest the best tourist activities for me to do.`,
});
```

<Note>
  When you use a message prompt, you can also use system messages instead of a
  system prompt.
</Note>

## Message Prompts

A message prompt is an array of user, assistant, and tool messages.
They are great for chat interfaces and more complex, multi-modal prompts.
You can use the `messages` property to set message prompts.

Each message has a `role` and a `content` property. The content can either be text (for user and assistant messages), or an array of relevant parts (data) for that message type.

```ts highlight="3-7"
const result = await streamUI({
  model: yourModel,
  messages: [
    { role: 'user', content: 'Hi!' },
    { role: 'assistant', content: 'Hello, how can I help?' },
    { role: 'user', content: 'Where can I buy the best Currywurst in Berlin?' },
  ],
});
```

Instead of sending a text in the `content` property, you can send an array of parts that includes a mix of text and other content parts.

<Note type="warning">
  Not all language models support all message and content types. For example,
  some models might not be capable of handling multi-modal inputs or tool
  messages. [Learn more about the capabilities of select
  models](./providers-and-models#model-capabilities).
</Note>

### User Messages

#### Text Parts

Text content is the most common type of content. It is a string that is passed to the model.

If you only need to send text content in a message, the `content` property can be a string,
but you can also use the `parts` property to send multiple parts of content.

```ts highlight="7"
const result = await generateText({
  model: yourModel,
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'Where can I buy the best Currywurst in Berlin?',
        },
      ],
    },
  ],
});
```

#### Image Parts

User messages can include image parts. An image can be one of the following:

- base64-encoded image:
  - `string` with base-64 encoded content
  - data URL `string`, e.g. `data:image/png;base64,...`
- binary image:
  - `ArrayBuffer`
  - `Uint8Array`
  - `Buffer`
- URL:
  - http(s) URL `string`, e.g. `https://example.com/image.png`
  - `URL` object, e.g. `new URL('https://example.com/image.png')`

##### Example: Binary image (Buffer)

```ts highlight="8-11"
const result = await generateText({
  model,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'image',
          image: fs.readFileSync('./data/comic-cat.png'),
        },
      ],
    },
  ],
});
```

##### Example: Base-64 encoded image (string)

```ts highlight="8-11"
const result = await generateText({
  model: yourModel,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'image',
          image: fs.readFileSync('./data/comic-cat.png').toString('base64'),
        },
      ],
    },
  ],
});
```

##### Example: Image URL (string)

```ts highlight="8-12"
const result = await generateText({
  model: yourModel,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'image',
          image:
            'https://github.com/vercel/ai/blob/main/examples/ai-core/data/comic-cat.png?raw=true',
        },
      ],
    },
  ],
});
```

#### File Parts

<Note type="warning">
  Only a few providers and models currently support file parts: [Google
  Generative AI](/providers/ai-sdk-providers/google-generative-ai), [Google
  Vertex AI](/providers/ai-sdk-providers/google-vertex),
  [OpenAI](/providers/ai-sdk-providers/openai) (for `wav` and `mp3` audio with
  `gpt-4o-audio-preview`), [Anthropic](/providers/ai-sdk-providers/anthropic)
  (for `pdf`).
</Note>

User messages can include file parts. A file can be one of the following:

- base64-encoded file:
  - `string` with base-64 encoded content
  - data URL `string`, e.g. `data:image/png;base64,...`
- binary data:
  - `ArrayBuffer`
  - `Uint8Array`
  - `Buffer`
- URL:
  - http(s) URL `string`, e.g. `https://example.com/some.pdf`
  - `URL` object, e.g. `new URL('https://example.com/some.pdf')`

You need to specify the MIME type of the file you are sending.

##### Example: PDF file from Buffer

```ts highlight="12-14"
import { google } from '@ai-sdk/google';
import { generateText } from 'ai';

const result = await generateText({
  model: google('gemini-1.5-flash'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the file about?' },
        {
          type: 'file',
          mimeType: 'application/pdf',
          data: fs.readFileSync('./data/example.pdf'),
        },
      ],
    },
  ],
});
```

##### Example: mp3 audio file from Buffer

```ts highlight="12-14"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const result = await generateText({
  model: openai('gpt-4o-audio-preview'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the audio saying?' },
        {
          type: 'file',
          mimeType: 'audio/mpeg',
          data: fs.readFileSync('./data/galileo.mp3'),
        },
      ],
    },
  ],
});
```

### Assistant Messages

Assistant messages are messages that have a role of `assistant`.
They are typically previous responses from the assistant and can contain text and tool call parts.

#### Example: Assistant message with text

```ts highlight="5"
const result = await generateText({
  model: yourModel,
  messages: [
    { role: 'user', content: 'Hi!' },
    { role: 'assistant', content: 'Hello, how can I help?' },
  ],
});
```

#### Example: Assistant message with tool call

```ts highlight="5-10"
const result = await generateText({
  model: yourModel,
  messages: [
    { role: 'user', content: 'How many calories are in this block of cheese?' },
    {
      type: 'tool-call',
      toolCallId: '12345',
      toolName: 'get-nutrition-data',
      args: { cheese: 'Roquefort' },
    },
  ],
});
```

### Tool messages

<Note>
  [Tools](/docs/foundations/tools) (also known as function calling) are programs
  that you can provide an LLM to extend it's built-in functionality. This can be
  anything from calling an external API to calling functions within your UI.
  Learn more about Tools in [the next section](/docs/foundations/tools).
</Note>

For models that support [tool](/docs/foundations/tools) calls, assistant messages can contain tool call parts, and tool messages can contain tool result parts.
A single assistant message can call multiple tools, and a single tool message can contain multiple tool results.

```ts highlight="14-42"
const result = await generateText({
  model: yourModel,
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'How many calories are in this block of cheese?',
        },
        { type: 'image', image: fs.readFileSync('./data/roquefort.jpg') },
      ],
    },
    {
      role: 'assistant',
      content: [
        {
          type: 'tool-call',
          toolCallId: '12345',
          toolName: 'get-nutrition-data',
          args: { cheese: 'Roquefort' },
        },
        // there could be more tool calls here (parallel calling)
      ],
    },
    {
      role: 'tool',
      content: [
        {
          type: 'tool-result',
          toolCallId: '12345', // needs to match the tool call id
          toolName: 'get-nutrition-data',
          result: {
            name: 'Cheese, roquefort',
            calories: 369,
            fat: 31,
            protein: 22,
          },
        },
        // there could be more tool results here (parallel calling)
      ],
    },
  ],
});
```

#### Multi-modal Tool Results

<Note type="warning">
  Multi-part tool results are experimental and only supported by Anthropic.
</Note>

Tool results can be multi-part and multi-modal, e.g. a text and an image.
You can use the `experimental_content` property on tool parts to specify multi-part tool results.

```ts highlight="20-32"
const result = await generateText({
  model: yourModel,
  messages: [
    // ...
    {
      role: 'tool',
      content: [
        {
          type: 'tool-result',
          toolCallId: '12345', // needs to match the tool call id
          toolName: 'get-nutrition-data',
          // for models that do not support multi-part tool results,
          // you can include a regular result part:
          result: {
            name: 'Cheese, roquefort',
            calories: 369,
            fat: 31,
            protein: 22,
          },
          // for models that support multi-part tool results,
          // you can include a multi-part content part:
          content: [
            {
              type: 'text',
              text: 'Here is an image of the nutrition data for the cheese:',
            },
            {
              type: 'image',
              data: fs.readFileSync('./data/roquefort-nutrition-data.png'),
              mimeType: 'image/png',
            },
          ],
        },
      ],
    },
  ],
});
```

### System Messages

System messages are messages that are sent to the model before the user messages to guide the assistant's behavior.
You can alternatively use the `system` property.

```ts highlight="4"
const result = await generateText({
  model: yourModel,
  messages: [
    { role: 'system', content: 'You help planning travel itineraries.' },
    {
      role: 'user',
      content:
        'I am planning a trip to Berlin for 3 days. Please suggest the best tourist activities for me to do.',
    },
  ],
});
```

---
title: Tools
description: Learn about tools with the AI SDK.
---

# Tools

While [large language models (LLMs)](/docs/foundations/overview#large-language-models) have incredible generation capabilities,
they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather).

Tools are actions that an LLM can invoke.
The results of these actions can be reported back to the LLM to be considered in the next response.

For example, when you ask an LLM for the "weather in London", and there is a weather tool available, it could call a tool
with London as the argument. The tool would then fetch the weather data and return it to the LLM. The LLM can then use this
information in its response.

## What is a tool?

A tool is an object that can be called by the model to perform a specific task.
You can use tools with [`generateText`](/docs/reference/ai-sdk-core/generate-text)
and [`streamText`](/docs/reference/ai-sdk-core/stream-text) by passing one or more tools to the `tools` parameter.

A tool consists of three properties:

- **`description`**: An optional description of the tool that can influence when the tool is picked.
- **`parameters`**: A [Zod schema](/docs/foundations/tools#schema-specification-and-validation-with-zod) or a [JSON schema](/docs/reference/ai-sdk-core/json-schema) that defines the parameters. The schema is consumed by the LLM, and also used to validate the LLM tool calls.
- **`execute`**: An optional async function that is called with the arguments from the tool call.

<Note>
  `streamUI` uses UI generator tools with a `generate` function that can return
  React components.
</Note>

If the LLM decides to use a tool, it will generate a tool call.
Tools with an `execute` function are run automatically when these calls are generated.
The results of the tool calls are returned using tool result objects.

You can automatically pass tool results back to the LLM
using [multi-step calls](/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls) with `streamText` and `generateText`.

## Schemas

Schemas are used to define the parameters for tools and to validate the [tool calls](/docs/ai-sdk-core/tools-and-tool-calling).

The AI SDK supports both raw JSON schemas (using the `jsonSchema` function) and [Zod](https://zod.dev/) schemas.
[Zod](https://zod.dev/) is the most popular JavaScript schema validation library.
You can install Zod with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add zod" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install zod" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add zod" dark />
  </Tab>
</Tabs>

You can then specify a Zod schema, for example:

```ts
import z from 'zod';

const recipeSchema = z.object({
  recipe: z.object({
    name: z.string(),
    ingredients: z.array(
      z.object({
        name: z.string(),
        amount: z.string(),
      }),
    ),
    steps: z.array(z.string()),
  }),
});
```

<Note>
  You can also use schemas for structured output generation with
  [`generateObject`](/docs/reference/ai-sdk-core/generate-object) and
  [`streamObject`](/docs/reference/ai-sdk-core/stream-object).
</Note>

## Toolkits

When you work with tools, you typically need a mix of application specific tools and general purpose tools.
There are several providers that offer pre-built tools as **toolkits** that you can use out of the box:

- **[agentic](https://github.com/transitive-bullshit/agentic)** - A collection of 20+ tools. Most tools connect to access external APIs such as [Exa](https://exa.ai/) or [E2B](https://e2b.dev/).
- **[browserbase](https://github.com/browserbase/js-sdk?tab=readme-ov-file#vercel-ai-sdk-integration)** - Browser tool that runs a headless browser
- **[Stripe agent tools](https://docs.stripe.com/agents)** - Tools for interacting with Stripe.

<Note>
  Do you have open source tools or tool libraries that are compatible with the
  AI SDK? Please [file a pull request](https://github.com/vercel/ai/pulls) to
  add them to this list.
</Note>

## Learn more

The AI SDK Core [Tool Calling](/docs/ai-sdk-core/tools-and-tool-calling)
and [Agents](/docs/ai-sdk-core/agents) documentation has more information about tools and tool calling.

---
title: Streaming
description: Why use streaming for AI applications?
---

# Streaming

Streaming conversational text UIs (like ChatGPT) have gained massive popularity over the past few months. This section explores the benefits and drawbacks of streaming and blocking interfaces.

[Large language models (LLMs)](/docs/foundations/overview#large-language-models) are extremely powerful. However, when generating long outputs, they can be very slow compared to the latency you're likely used to. If you try to build a traditional blocking UI, your users might easily find themselves staring at loading spinners for 5, 10, even up to 40s waiting for the entire LLM response to be generated. This can lead to a poor user experience, especially in conversational applications like chatbots. Streaming UIs can help mitigate this issue by **displaying parts of the response as they become available**.

<div className="grid lg:grid-cols-2 grid-cols-1 gap-4 mt-8">
  <Card
    title="Blocking UI"
    description="Blocking responses wait until the full response is available before displaying it."
  >
    <BrowserIllustration highlight blocking />
  </Card>
  <Card
    title="Streaming UI"
    description="Streaming responses can transmit parts of the response as they become available."
  >
    <BrowserIllustration highlight />
  </Card>
</div>

## Real-world Examples

Here are 2 examples that illustrate how streaming UIs can improve user experiences in a real-world setting – the first uses a blocking UI, while the second uses a streaming UI.

### Blocking UI

<InlinePrompt
  initialInput="Come up with the first 200 characters of the first book in the Harry Potter series."
  blocking
/>

### Streaming UI

<InlinePrompt initialInput="Come up with the first 200 characters of the first book in the Harry Potter series." />

As you can see, the streaming UI is able to start displaying the response much faster than the blocking UI. This is because the blocking UI has to wait for the entire response to be generated before it can display anything, while the streaming UI can display parts of the response as they become available.

While streaming interfaces can greatly enhance user experiences, especially with larger language models, they aren't always necessary or beneficial. If you can achieve your desired functionality using a smaller, faster model without resorting to streaming, this route can often lead to simpler and more manageable development processes.

However, regardless of the speed of your model, the AI SDK is designed to make implementing streaming UIs as simple as possible. In the example below, we stream text generation from OpenAI's `gpt-4-turbo` in under 10 lines of code using the SDK's [`streamText`](/docs/reference/ai-sdk-core/stream-text) function:

```ts
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

const { textStream } = streamText({
  model: openai('gpt-4-turbo'),
  prompt: 'Write a poem about embedding models.',
});

for await (const textPart of textStream) {
  console.log(textPart);
}
```

For an introduction to streaming UIs and the AI SDK, check out our [Getting Started guides](/docs/getting-started).

---
title: Foundations
description: A section that covers foundational knowledge around LLMs and concepts crucial to the AI SDK
---

# Foundations

<IndexCards
  cards={[
    {
      title: 'Overview',
      description: 'Learn about foundational concepts around AI and LLMs.',
      href: '/docs/foundations/overview',
    },
    {
      title: 'Providers and Models',
      description:
        'Learn about the providers and models that you can use with the AI SDK.',
      href: '/docs/foundations/providers-and-models',
    },
    {
      title: 'Prompts',
      description:
        'Learn about how Prompts are used and defined in the AI SDK.',
      href: '/docs/foundations/prompts',
    },
    {
      title: 'Tools',
      description: 'Learn about tools in the AI SDK.',
      href: '/docs/foundations/tools',
    },
    {
      title: 'Streaming',
      description: 'Learn why streaming is used for AI applications.',
      href: '/docs/foundations/streaming',
    },
  ]}
/>

---
title: Navigating the Library
description: Learn how to navigate the AI SDK.
---

# Navigating the Library

the AI SDK is a powerful toolkit for building AI applications. This page will help you pick the right tools for your requirements.

Let’s start with a quick overview of the AI SDK, which is comprised of three parts:

- **[AI SDK Core](/docs/ai-sdk-core/overview):** A unified, provider agnostic API for generating text, structured objects, and tool calls with LLMs.
- **[AI SDK UI](/docs/ai-sdk-ui/overview):** A set of framework-agnostic hooks for building chat and generative user interfaces.
- [AI SDK RSC](/docs/ai-sdk-rsc/overview): Stream generative user interfaces with React Server Components (RSC). Development is currently experimental and we recommend using [AI SDK UI](/docs/ai-sdk-ui/overview).

## Choosing the Right Tool for Your Environment

When deciding which part of the AI SDK to use, your first consideration should be the environment and existing stack you are working with. Different components of the SDK are tailored to specific frameworks and environments.

| Library                                   | Purpose                                                                                                                                                                                                  | Environment Compatibility                                              |
| ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| [AI SDK Core](/docs/ai-sdk-core/overview) | Call any LLM with unified API (e.g. [generateText](/docs/reference/ai-sdk-core/generate-text) and [generateObject](/docs/reference/ai-sdk-core/generate-object))                                         | Any JS environment (e.g. Node.js, Deno, Browser)                       |
| [AI SDK UI](/docs/ai-sdk-ui/overview)     | Build streaming chat and generative UIs (e.g. [useChat](/docs/reference/ai-sdk-ui/use-chat))                                                                                                             | React & Next.js, Vue & Nuxt, Svelte & SvelteKit, Solid.js & SolidStart |
| [AI SDK RSC](/docs/ai-sdk-rsc/overview)   | Stream generative UIs from Server to Client (e.g. [streamUI](/docs/reference/ai-sdk-rsc/stream-ui)). Development is currently experimental and we recommend using [AI SDK UI](/docs/ai-sdk-ui/overview). | Any framework that supports React Server Components (e.g. Next.js)     |

## Environment Compatibility

These tools have been designed to work seamlessly with each other and it's likely that you will be using them together. Let's look at how you could decide which libraries to use based on your application environment, existing stack, and requirements.

The following table outlines AI SDK compatibility based on environment:

| Environment           | [AI SDK Core](/docs/ai-sdk-core/overview) | [AI SDK UI](/docs/ai-sdk-ui/overview) | [AI SDK RSC](/docs/ai-sdk-rsc/overview) |
| --------------------- | ----------------------------------------- | ------------------------------------- | --------------------------------------- |
| None / Node.js / Deno | <Check size={18} />                       | <Cross size={18} />                   | <Cross size={18} />                     |
| Vue / Nuxt            | <Check size={18} />                       | <Check size={18} />                   | <Cross size={18} />                     |
| Svelte / SvelteKit    | <Check size={18} />                       | <Check size={18} />                   | <Cross size={18} />                     |
| Solid.js / SolidStart | <Check size={18} />                       | <Check size={18} />                   | <Cross size={18} />                     |
| Next.js Pages Router  | <Check size={18} />                       | <Check size={18} />                   | <Cross size={18} />                     |
| Next.js App Router    | <Check size={18} />                       | <Check size={18} />                   | <Check size={18} />                     |

## When to use AI SDK UI

AI SDK UI provides a set of framework-agnostic hooks for quickly building **production-ready AI-native applications**. It offers:

- Full support for streaming chat and client-side generative UI
- Utilities for handling common AI interaction patterns (i.e. chat, completion, assistant)
- Production-tested reliability and performance
- Compatibility across popular frameworks

## AI SDK UI Framework Compatibility

AI SDK UI supports the following frameworks: [React](https://react.dev/), [Svelte](https://svelte.dev/), [Vue.js](https://vuejs.org/), and [SolidJS](https://www.solidjs.com/). Here is a comparison of the supported functions across these frameworks:

| Function                                                   | React               | Svelte              | Vue.js              | SolidJS             |
| ---------------------------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| [useChat](/docs/reference/ai-sdk-ui/use-chat)              | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useChat](/docs/reference/ai-sdk-ui/use-chat) tool calling | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> | <Check size={18} /> |
| [useChat](/docs/reference/ai-sdk-ui/use-chat) attachments  | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useCompletion](/docs/reference/ai-sdk-ui/use-completion)  | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useObject](/docs/reference/ai-sdk-ui/use-object)          | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useAssistant](/docs/reference/ai-sdk-ui/use-assistant)    | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |

<Note>
  [Contributions](https://github.com/vercel/ai/blob/main/CONTRIBUTING.md) are
  welcome to implement missing features for non-React frameworks.
</Note>

## When to use AI SDK RSC

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

[React Server Components](https://nextjs.org/docs/app/building-your-application/rendering/server-components)
(RSCs) provide a new approach to building React applications that allow components
to render on the server, fetch data directly, and stream the results to the client,
reducing bundle size and improving performance. They also introduce a new way to
call server-side functions from anywhere in your application called [Server Actions](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations).

AI SDK RSC provides a number of utilities that allow you to stream values and UI directly from the server to the client. However, **it's important to be aware of current limitations**:

- **Cancellation**: currently, it is not possible to abort a stream using Server Actions. This will be improved in future releases of React and Next.js.
- **Increased Data Transfer**: using [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) can lead to quadratic data transfer (quadratic to the length of generated text). You can avoid this using [ `createStreamableValue` ](/docs/reference/ai-sdk-rsc/create-streamable-value) instead, and rendering the component client-side.
- **Re-mounting Issue During Streaming**: when using `createStreamableUI`, components re-mount on `.done()`, causing [flickering](https://github.com/vercel/ai/issues/2232).

Given these limitations, **we recommend using [AI SDK UI](/docs/ai-sdk-ui/overview) for production applications**.

---
title: Next.js App Router
description: Welcome to the AI SDK quickstart guide for Next.js App Router!
---

# Next.js App Router Quickstart

In this quick start tutorial, you'll build a simple AI-chatbot with a streaming user interface. Along the way, you'll learn key concepts and techniques that are fundamental to using the SDK in your own projects.

Check out [Prompt Engineering](/docs/advanced/prompt-engineering) and [HTTP Streaming](/docs/advanced/why-streaming) if you haven't heard of them.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Create Your Application

Start by creating a new Next.js application. This command will create a new directory named `my-ai-app` and set up a basic Next.js application inside it.

<div className="mb-4">
  <Note>
    Be sure to select yes when prompted to use the App Router. If you are
    looking for the Next.js Pages Router quickstart guide, you can find it
    [here](/docs/getting-started/nextjs-pages-router).
  </Note>
</div>

<Snippet text="pnpm create next-app@latest my-ai-app" />

Navigate to the newly created directory:

<Snippet text="cd my-ai-app" />

### Install dependencies

Install `ai` and `@ai-sdk/openai`, the AI package and AI SDK's [ OpenAI provider ](/providers/ai-sdk-providers/openai) respectively.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>
<div className="my-4">
  <Tabs items={['pnpm', 'npm', 'yarn']}>
    <Tab>
      <Snippet text="pnpm add ai @ai-sdk/openai zod" dark />
    </Tab>
    <Tab>
      <Snippet text="npm install ai @ai-sdk/openai zod" dark />
    </Tab>
    <Tab>
      <Snippet text="yarn add ai @ai-sdk/openai zod" dark />
    </Tab>
  </Tabs>
</div>

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.1 or higher.
</Note>

### Configure OpenAI API key

Create a `.env.local` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env.local" />

Edit the `.env.local` file:

```env filename=".env.local"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key.

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Create a Route Handler

Create a route handler, `app/api/chat/route.ts` and add the following code:

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

Let's take a look at what is happening in this code:

1. Define an asynchronous `POST` request handler and extract `messages` from the body of the request. The `messages` variable contains a history of the conversation between you and the chatbot and provides the chatbot with the necessary context to make the next generation.
2. Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider (imported from `@ai-sdk/openai`) and `messages` (defined in step 1). You can pass additional [settings](/docs/ai-sdk-core/settings) to further customise the model's behaviour.
3. The `streamText` function returns a [`StreamTextResult`](/docs/reference/ai-sdk-core/stream-text#result-object). This result object contains the [ `toDataStreamResponse` ](/docs/reference/ai-sdk-core/stream-text#to-data-stream-response) function which converts the result to a streamed response object.
4. Finally, return the result to the client to stream the response.

This Route Handler creates a POST request endpoint at `/api/chat`.

## Wire up the UI

Now that you have a Route Handler that can query an LLM, it's time to setup your frontend. The AI SDK's [ UI ](/docs/ai-sdk-ui) package abstracts the complexity of a chat interface into one hook, [`useChat`](/docs/reference/ai-sdk-ui/use-chat).

Update your root page (`app/page.tsx`) with the following code to show a list of chat messages and provide a user message input:

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

<Note>
  Make sure you add the `"use client"` directive to the top of your file. This
  allows you to add interactivity with Javascript.
</Note>

This page utilizes the `useChat` hook, which will, by default, use the `POST` API route you created earlier (`/api/chat`). The hook provides functions and state for handling user input and form submission. The `useChat` hook provides multiple utility functions and state variables:

- `messages` - the current chat messages (an array of objects with `id`, `role`, and `content` properties).
- `input` - the current value of the user's input field.
- `handleInputChange` and `handleSubmit` - functions to handle user interactions (typing into the input field and submitting the form, respectively).

## Running Your Application

With that, you have built everything you need for your chatbot! To start your application, use the command:

<Snippet text="pnpm run dev" />

Head to your browser and open http://localhost:3000. You should see an input field. Test it out by entering a message and see the AI chatbot respond in real-time! The AI SDK makes it fast and easy to build AI chat interfaces with Next.js.

## Enhance Your Chatbot with Tools

While large language models (LLMs) have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). This is where [tools](/docs/ai-sdk-core/tools-and-tool-calling) come in.

Tools are actions that an LLM can invoke. The results of these actions can be reported back to the LLM to be considered in the next response.

For example, if a user asks about the current weather, without tools, the model would only be able to provide general information based on its training data. But with a weather tool, it can fetch and provide up-to-date, location-specific weather information.

Let's enhance your chatbot by adding a simple weather tool.

### Update Your Route Handler

Modify your `app/api/chat/route.ts` file to include the new weather tool:

```tsx filename="app/api/chat/route.ts" highlight="2,13-27"
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

In this updated code:

1. You import the `tool` function from the `ai` package and `z` from `zod` for schema validation.
2. You define a `tools` object with a `weather` tool. This tool:

   - Has a description that helps the model understand when to use it.
   - Defines parameters using a Zod schema, specifying that it requires a `location` string to execute this tool. The model will attempt to extract this parameter from the context of the conversation. If it can't, it will ask the user for the missing information.
   - Defines an `execute` function that simulates getting weather data (in this case, it returns a random temperature). This is an asynchronous function running on the server so you can fetch real data from an external API.

   Now your chatbot can "fetch" weather information for any location the user asks about. When the model determines it needs to use the weather tool, it will generate a tool call with the necessary parameters. The `execute` function will then be automatically run, and you can access the results via `toolInvocations` that is available on the message object.

Try asking something like "What's the weather in New York?" and see how the model uses the new tool.

Notice the blank response in the UI? This is because instead of generating a text response, the model generated a tool call. You can access the tool call and subsequent tool result in the `toolInvocations` key of the message object.

### Update the UI

To display the tool invocations in your UI, update your `app/page.tsx` file:

```tsx filename="app/page.tsx" highlight="12-16"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.toolInvocations ? (
            <pre>{JSON.stringify(m.toolInvocations, null, 2)}</pre>
          ) : (
            <p>{m.content}</p>
          )}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

With this change, you check each message for any tool calls (`toolInvocations`). These tool calls will be displayed as stringified JSON. Otherwise, you show the message content as before.

Now, when you ask about the weather, you'll see the tool invocation and its result displayed in your chat interface.

## Enabling Multi-Step Tool Calls

You may have noticed that while the tool results are visible in the chat interface, the model isn't using this information to answer your original query. This is because once the model generates a tool call, it has technically completed its generation.

To solve this, you can enable multi-step tool calls using the `maxSteps` option in your `useChat` hook. This feature will automatically send tool results back to the model to trigger an additional generation. In this case, you want the model to answer your question using the results from the weather tool.

### Update Your Client-Side Code

Modify your `app/page.tsx` file to include the `maxSteps` option:

```tsx filename="app/page.tsx" highlight="7"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    maxSteps: 5,
  });

  // ... rest of your component code
}
```

Head back to the browser and ask about the weather in a location. You should now see the model using the weather tool results to answer your question.

By setting `maxSteps` to 5, you're allowing the model to use up to 5 "steps" for any given generation. This enables more complex interactions and allows the model to gather and process information over several steps if needed. You can see this in action by adding another tool to convert the temperature from Fahrenheit to Celsius.

### Update Your Route Handler

Update your `app/api/chat/route.ts` file to add a new tool to convert the temperature from Fahrenheit to Celsius:

```tsx filename="app/api/chat/route.ts" highlight="27-40"
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
      convertFarenheitToCelsius: tool({
        description: 'Convert a temperature in farenheit to celsius',
        parameters: z.object({
          temperature: z
            .number()
            .describe('The temperature in farenheit to convert'),
        }),
        execute: async ({ temperature }) => {
          const celsius = Math.round((temperature - 32) * (5 / 9));
          return {
            celsius,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

Now, when you ask "What's the weather in New York in celsius?", you should see a more complete interaction:

1. The model will call the weather tool for New York.
2. You'll see the tool result displayed.
3. It will then call the temperature conversion tool to convert the temperature from Fahrenheit to Celsius.
4. The model will then use that information to provide a natural language response about the weather in New York.

This multi-step approach allows the model to gather information and use it to provide more accurate and contextual responses, making your chatbot considerably more useful.

This simple example demonstrates how tools can expand your model's capabilities. You can create more complex tools to integrate with real APIs, databases, or any other external systems, allowing the model to access and process real-world data in real-time. Tools bridge the gap between the model's knowledge cutoff and current information.

## Where to Next?

You've built an AI chatbot using the AI SDK! From here, you have several paths to explore:

- To learn more about the AI SDK, read through the [documentation](/docs).
- If you're interested in diving deeper with guides, check out the [RAG (retrieval-augmented generation)](/docs/guides/rag-chatbot) and [multi-modal chatbot](/docs/guides/multi-modal-chatbot) guides.
- To jumpstart your first AI project, explore available [templates](https://vercel.com/templates?type=ai).

---
title: Next.js Pages Router
description: Welcome to the AI SDK quickstart guide for Next.js Pages Router!
---

# Next.js Pages Router Quickstart

The AI SDK is a powerful Typescript library designed to help developers build AI-powered applications.

In this quickstart tutorial, you'll build a simple AI-chatbot with a streaming user interface. Along the way, you'll learn key concepts and techniques that are fundamental to using the SDK in your own projects.

If you are unfamiliar with the concepts of [Prompt Engineering](/docs/advanced/prompt-engineering) and [HTTP Streaming](/docs/advanced/why-streaming), you can optionally read these documents first.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Setup Your Application

Start by creating a new Next.js application. This command will create a new directory named `my-ai-app` and set up a basic Next.js application inside it.

<Note>
  Be sure to select no when prompted to use the App Router. If you are looking
  for the Next.js App Router quickstart guide, you can find it
  [here](/docs/getting-started/nextjs-app-router).
</Note>

<Snippet text="pnpm create next-app@latest my-ai-app" />

Navigate to the newly created directory:

<Snippet text="cd my-ai-app" />

### Install dependencies

Install `ai` and `@ai-sdk/openai`, the AI SDK's OpenAI provider.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>
<div className="my-4">
  <Tabs items={['pnpm', 'npm', 'yarn']}>
    <Tab>
      <Snippet text="pnpm add ai @ai-sdk/openai zod" dark />
    </Tab>
    <Tab>
      <Snippet text="npm install ai @ai-sdk/openai zod" dark />
    </Tab>
    <Tab>
      <Snippet text="yarn add ai @ai-sdk/openai zod" dark />
    </Tab>
  </Tabs>
</div>

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.1 or higher.
</Note>

### Configure OpenAI API Key

Create a `.env.local` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env.local" />

Edit the `.env.local` file:

```env filename=".env.local"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key.

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Create a Route Handler

<Note>
  As long as you are on Next.js 13+, you can use Route Handlers (using the App
  Router) alongside the Pages Router. This is recommended to enable you to use
  the Web APIs interface/signature and to better support streaming.
</Note>

Create a Route Handler (`app/api/chat/route.ts`) and add the following code:

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

Let's take a look at what is happening in this code:

1. Define an asynchronous `POST` request handler and extract `messages` from the body of the request. The `messages` variable contains a history of the conversation between you and the chatbot and provides the chatbot with the necessary context to make the next generation.
2. Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider (imported from `@ai-sdk/openai`) and `messages` (defined in step 1). You can pass additional [settings](/docs/ai-sdk-core/settings) to further customise the model's behaviour.
3. The `streamText` function returns a [`StreamTextResult`](/docs/reference/ai-sdk-core/stream-text#result-object). This result object contains the [ `toDataStreamResponse` ](/docs/reference/ai-sdk-core/stream-text#to-data-stream-response) function which converts the result to a streamed response object.
4. Finally, return the result to the client to stream the response.

This Route Handler creates a POST request endpoint at `/api/chat`.

## Wire up the UI

Now that you have an API route that can query an LLM, it's time to setup your frontend. The AI SDK's [ UI ](/docs/ai-sdk-ui) package abstract the complexity of a chat interface into one hook, [`useChat`](/docs/reference/ai-sdk-ui/use-chat).

Update your root page (`pages/index.tsx`) with the following code to show a list of chat messages and provide a user message input:

```tsx filename="pages/index.tsx"
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

This page utilizes the `useChat` hook, which will, by default, use the `POST` API route you created earlier (`/api/chat`). The hook provides functions and state for handling user input and form submission. The `useChat` hook provides multiple utility functions and state variables:

- `messages` - the current chat messages (an array of objects with `id`, `role`, and `content` properties).
- `input` - the current value of the user's input field.
- `handleInputChange` and `handleSubmit` - functions to handle user interactions (typing into the input field and submitting the form, respectively).
- `isLoading` - boolean that indicates whether the API request is in progress.

## Running Your Application

With that, you have built everything you need for your chatbot! To start your application, use the command:

<Snippet text="pnpm run dev" />

Head to your browser and open http://localhost:3000. You should see an input field. Test it out by entering a message and see the AI chatbot respond in real-time! The AI SDK makes it fast and easy to build AI chat interfaces with Next.js.

## Enhance Your Chatbot with Tools

While large language models (LLMs) have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). This is where [tools](/docs/ai-sdk-core/tools-and-tool-calling) come in.

Tools are actions that an LLM can invoke. The results of these actions can be reported back to the LLM to be considered in the next response.

For example, if a user asks about the current weather, without tools, the model would only be able to provide general information based on its training data. But with a weather tool, it can fetch and provide up-to-date, location-specific weather information.

Let's enhance your chatbot by adding a simple weather tool.

### Update Your Route Handler

Modify your `app/api/chat/route.ts` file to include the new weather tool:

```tsx filename="app/api/chat/route.ts" highlight="2,13-27"
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

In this updated code:

1. You import the `tool` function from the `ai` package and `z` from `zod` for schema validation.
2. You define a `tools` object with a `weather` tool. This tool:

   - Has a description that helps the model understand when to use it.
   - Defines parameters using a Zod schema, specifying that it requires a `location` string to execute this tool. The model will attempt to extract this parameter from the context of the conversation. If it can't, it will ask the user for the missing information.
   - Defines an `execute` function that simulates getting weather data (in this case, it returns a random temperature). This is an asynchronous function running on the server so you can fetch real data from an external API.

   Now your chatbot can "fetch" weather information for any location the user asks about. When the model determines it needs to use the weather tool, it will generate a tool call with the necessary parameters. The `execute` function will then be automatically run, and you can access the results via `toolInvocations` that is available on the message object.

Try asking something like "What's the weather in New York?" and see how the model uses the new tool.

Notice the blank response in the UI? This is because instead of generating a text response, the model generated a tool call. You can access the tool call and subsequent tool result in the `toolInvocations` key of the message object.

### Update the UI

To display the tool invocations in your UI, update your `pages/index.tsx` file:

```tsx filename="pages/index.tsx" highlight="11-15"
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.toolInvocations ? (
            <pre>{JSON.stringify(m.toolInvocations, null, 2)}</pre>
          ) : (
            <p>{m.content}</p>
          )}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

With this change, you check each message for any tool calls (`toolInvocations`). These tool calls will be displayed as stringified JSON. Otherwise, you show the message content as before.

Now, when you ask about the weather, you'll see the tool invocation and its result displayed in your chat interface.

## Enabling Multi-Step Tool Calls

You may have noticed that while the tool results are visible in the chat interface, the model isn't using this information to answer your original query. This is because once the model generates a tool call, it has technically completed its generation.

To solve this, you can enable multi-step tool calls using the `maxSteps` option in your `useChat` hook. This feature will automatically send tool results back to the model to trigger an additional generation. In this case, you want the model to answer your question using the results from the weather tool.

### Update Your Client-Side Code

Modify your `pages/index.tsx` file to include the `maxSteps` option:

```tsx filename="pages/index.tsx" highlight="6"
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    maxSteps: 5,
  });

  // ... rest of your component code
}
```

Head back to the browser and ask about the weather in a location. You should now see the model using the weather tool results to answer your question.

By setting `maxSteps` to 5, you're allowing the model to use up to 5 "steps" for any given generation. This enables more complex interactions and allows the model to gather and process information over several steps if needed. You can see this in action by adding another tool to convert the temperature from Fahrenheit to Celsius.

### Update Your Route Handler

Update your `app/api/chat/route.ts` file to add a new tool to convert the temperature from Fahrenheit to Celsius:

```tsx filename="app/api/chat/route.ts" highlight="27-40"
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
      convertFarenheitToCelsius: tool({
        description: 'Convert a temperature in farenheit to celsius',
        parameters: z.object({
          temperature: z
            .number()
            .describe('The temperature in farenheit to convert'),
        }),
        execute: async ({ temperature }) => {
          const celsius = Math.round((temperature - 32) * (5 / 9));
          return {
            celsius,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

Now, when you ask "What's the weather in New York in celsius?", you should see a more complete interaction:

1. The model will call the weather tool for New York.
2. You'll see the tool result displayed.
3. It will then call the temperature conversion tool to convert the temperature from Fahrenheit to Celsius.
4. The model will then use that information to provide a natural language response about the weather in New York.

This multi-step approach allows the model to gather information and use it to provide more accurate and contextual responses, making your chatbot considerably more useful.

This simple example demonstrates how tools can expand your model's capabilities. You can create more complex tools to integrate with real APIs, databases, or any other external systems, allowing the model to access and process real-world data in real-time. Tools bridge the gap between the model's knowledge cutoff and current information.

## Where to Next?

You've built an AI chatbot using the AI SDK! From here, you have several paths to explore:

- To learn more about the AI SDK, read through the [documentation](/docs).
- If you're interested in diving deeper with guides, check out the [RAG (retrieval-augmented generation)](/docs/guides/rag-chatbot) and [multi-modal chatbot](/docs/guides/multi-modal-chatbot) guides.
- To jumpstart your first AI project, explore available [templates](https://vercel.com/templates?type=ai).

---
title: Svelte
description: Welcome to the AI SDK quickstart guide for Svelte!
---

# Svelte Quickstart

The AI SDK is a powerful Typescript library designed to help developers build AI-powered applications.

In this quickstart tutorial, you'll build a simple AI-chatbot with a streaming user interface. Along the way, you'll learn key concepts and techniques that are fundamental to using the SDK in your own projects.

If you are unfamiliar with the concepts of [Prompt Engineering](/docs/advanced/prompt-engineering) and [HTTP Streaming](/docs/advanced/why-streaming), you can optionally read these documents first.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Setup Your Application

Start by creating a new SvelteKit application. This command will create a new directory named `my-ai-app` and set up a basic SvelteKit application inside it.

<Snippet text="pnpm create svelte@latest my-ai-app" />

Navigate to the newly created directory:

<Snippet text="cd my-ai-app" />

### Install Dependencies

Install `ai` and `@ai-sdk/openai`, the AI SDK's OpenAI provider.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>
<div className="my-4">
  <Tabs items={['pnpm', 'npm', 'yarn']}>
    <Tab>
      <Snippet text="pnpm add ai @ai-sdk/openai @ai-sdk/svelte zod" dark />
    </Tab>
    <Tab>
      <Snippet text="npm install ai @ai-sdk/openai @ai-sdk/svelte zod" dark />
    </Tab>
    <Tab>
      <Snippet text="yarn add ai @ai-sdk/openai @ai-sdk/svelte zod" dark />
    </Tab>
  </Tabs>
</div>

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.1 or higher.
</Note>

### Configure OpenAI API Key

Create a `.env.local` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env.local" />

Edit the `.env.local` file:

```env filename=".env.local"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key.

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Create an API route

Create a SvelteKit Endpoint, `src/routes/api/chat/+server.ts` and add the following code:

```tsx filename="src/routes/api/chat/+server.ts"
import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';
import type { RequestHandler } from './$types';

import { env } from '$env/dynamic/private';

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY ?? '',
});

export const POST = (async ({ request }) => {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}) satisfies RequestHandler;
```

<Note>
  You may see an error with the `./$types` import. This will be resolved as soon
  as you run the dev server.
</Note>

Let's take a look at what is happening in this code:

1. Create an OpenAI provider instance with the `createOpenAI` function from the `@ai-sdk/openai` package.
2. Define a `POST` request handler and extract `messages` from the body of the request. The `messages` variable contains a history of the conversation with you and the chatbot and will provide the chatbot with the necessary context to make the next generation.
3. Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider (defined in step 1) and `messages` (defined in step 2). You can pass additional [settings](/docs/ai-sdk-core/settings) to further customise the model's behaviour.
4. The `streamText` function returns a [`StreamTextResult`](/docs/reference/ai-sdk-core/stream-text#result-object). This result object contains the [ `toDataStreamResponse` ](/docs/reference/ai-sdk-core/stream-text#to-data-stream-response) function which converts the result to a streamed response object.
5. Return the result to the client to stream the response.

## Wire up the UI

Now that you have an API route that can query an LLM, it's time to setup your frontend. The AI SDK's [ UI ](/docs/ai-sdk-ui) package abstract the complexity of a chat interface into one hook, [`useChat`](/docs/reference/ai-sdk-ui/use-chat).

Update your root page (`src/routes/+page.svelte`) with the following code to show a list of chat messages and provide a user message input:

```svelte filename="src/routes/+page.svelte"
<script>
  import { useChat } from '@ai-sdk/svelte';

  const { input, handleSubmit, messages } = useChat();
</script>

<main>
  <ul>
    {#each $messages as message}
      <li>{message.role}: {message.content}</li>
    {/each}
  </ul>
  <form on:submit={handleSubmit}>
    <input bind:value={$input} />
    <button type="submit">Send</button>
  </form>
</main>
```

This page utilizes the `useChat` hook, which will, by default, use the `POST` route handler you created earlier. The hook provides functions and state for handling user input and form submission. The `useChat` hook provides multiple utility functions and state variables:

- `messages` - the current chat messages (an array of objects with `id`, `role`, and `content` properties).
- `input` - the current value of the user's input field.
- `handleSubmit` - function to handle form submission.

## Running Your Application

With that, you have built everything you need for your chatbot! To start your application, use the command:

<Snippet text="pnpm run dev" />

Head to your browser and open http://localhost:5173. You should see an input field. Test it out by entering a message and see the AI chatbot respond in real-time! The AI SDK makes it fast and easy to build AI chat interfaces with Svelte.

## Enhance Your Chatbot with Tools

While large language models (LLMs) have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). This is where [tools](/docs/ai-sdk-core/tools-and-tool-calling) come in.

Tools are actions that an LLM can invoke. The results of these actions can be reported back to the LLM to be considered in the next response.

For example, if a user asks about the current weather, without tools, the model would only be able to provide general information based on its training data. But with a weather tool, it can fetch and provide up-to-date, location-specific weather information.

Let's enhance your chatbot by adding a simple weather tool.

### Update Your API Route

Modify your `src/routes/api/chat/+server.ts` file to include the new weather tool:

```tsx filename="src/routes/api/chat/+server.ts" highlight="2,4,18-32"
import { createOpenAI } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import type { RequestHandler } from './$types';
import { z } from 'zod';

import { env } from '$env/dynamic/private';

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY ?? '',
});

export const POST = (async ({ request }) => {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}) satisfies RequestHandler;
```

In this updated code:

1. You import the `tool` function from the `ai` package and `z` from `zod` for schema validation.
2. You define a `tools` object with a `weather` tool. This tool:

   - Has a description that helps the model understand when to use it.
   - Defines parameters using a Zod schema, specifying that it requires a `location` string to execute this tool. The model will attempt to extract this parameter from the context of the conversation. If it can't, it will ask the user for the missing information.
   - Defines an `execute` function that simulates getting weather data (in this case, it returns a random temperature). This is an asynchronous function running on the server so you can fetch real data from an external API.

Now your chatbot can "fetch" weather information for any location the user asks about. When the model determines it needs to use the weather tool, it will generate a tool call with the necessary parameters. The `execute` function will then be automatically run, and you can access the results via `toolInvocations` that is available on the message object.

Try asking something like "What's the weather in New York?" and see how the model uses the new tool.

Notice the blank response in the UI? This is because instead of generating a text response, the model generated a tool call. You can access the tool call and subsequent tool result in the `toolInvocations` key of the message object.

### Update the UI

To display the tool invocations in your UI, update your `src/routes/+page.svelte` file:

```svelte filename="src/routes/+page.svelte"
<script>
  import { useChat } from '@ai-sdk/svelte';

  const { input, handleSubmit, messages } = useChat();
</script>

<main>
  <ul>
    {#each $messages as message}
      <li>
        {message.role}:
        {#if message.toolInvocations}
          <pre>{JSON.stringify(message.toolInvocations, null, 2)}</pre>
        {:else}
          {message.content}
        {/if}
      </li>
    {/each}
  </ul>
  <form on:submit={handleSubmit}>
    <input bind:value={$input} />
    <button type="submit">Send</button>
  </form>
</main>
```

With this change, you check each message for any tool calls (`toolInvocations`). These tool calls will be displayed as stringified JSON. Otherwise, you show the message content as before.

Now, when you ask about the weather, you'll see the tool invocation and its result displayed in your chat interface.

## Enabling Multi-Step Tool Calls

You may have noticed that while the tool results are visible in the chat interface, the model isn't using this information to answer your original query. This is because once the model generates a tool call, it has technically completed its generation.

To solve this, you can enable multi-step tool calls using the `maxSteps` option in your `useChat` hook. This feature will automatically send tool results back to the model to trigger an additional generation. In this case, you want the model to answer your question using the results from the weather tool.

### Update Your UI

Modify your `src/routes/+page.svelte` file to include the `maxSteps` option:

```svelte filename="src/routes/+page.svelte" highlight="4"
<script>
  import { useChat } from '@ai-sdk/svelte';

  const { messages, input, handleSubmit } = useChat({ maxSteps: 5 });
</script>

<!-- ... rest of your component code -->
```

Head back to the browser and ask about the weather in a location. You should now see the model using the weather tool results to answer your question.

By setting `maxSteps` to 5, you're allowing the model to use up to 5 "steps" for any given generation. This enables more complex interactions and allows the model to gather and process information over several steps if needed. You can see this in action by adding another tool to convert the temperature from Fahrenheit to Celsius.

### Update Your API Route

Update your `src/routes/api/chat/+server.ts` file to add a new tool to convert the temperature from Fahrenheit to Celsius:

```tsx filename="src/routes/api/chat/+server.ts" highlight="32-45"
import { createOpenAI } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import type { RequestHandler } from './$types';
import { z } from 'zod';

import { env } from '$env/dynamic/private';

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY ?? '',
});

export const POST = (async ({ request }) => {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      weather: tool({
        description: 'Get the weather in a location (farenheit)',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => {
          const temperature = Math.round(Math.random() * (90 - 32) + 32);
          return {
            location,
            temperature,
          };
        },
      }),
      convertFarenheitToCelsius: tool({
        description: 'Convert a temperature in farenheit to celsius',
        parameters: z.object({
          temperature: z
            .number()
            .describe('The temperature in farenheit to convert'),
        }),
        execute: async ({ temperature }) => {
          const celsius = Math.round((temperature - 32) * (5 / 9));
          return {
            celsius,
          };
        },
      }),
    },
  });

  return result.toDataStreamResponse();
}) satisfies RequestHandler;
```

Now, when you ask "What's the weather in New York in celsius?", you should see a more complete interaction:

1. The model will call the weather tool for New York.
2. You'll see the tool result displayed.
3. It will then call the temperature conversion tool to convert the temperature from Fahrenheit to Celsius.
4. The model will then use that information to provide a natural language response about the weather in New York.

This multi-step approach allows the model to gather information and use it to provide more accurate and contextual responses, making your chatbot considerably more useful.

This simple example demonstrates how tools can expand your model's capabilities. You can create more complex tools to integrate with real APIs, databases, or any other external systems, allowing the model to access and process real-world data in real-time. Tools bridge the gap between the model's knowledge cutoff and current information.

## Where to Next?

You've built an AI chatbot using the AI SDK! From here, you have several paths to explore:

- To learn more about the AI SDK, read through the [documentation](/docs).
- If you're interested in diving deeper with guides, check out the [RAG (retrieval-augmented generation)](/docs/guides/rag-chatbot) and [multi-modal chatbot](/docs/guides/multi-modal-chatbot) guides.
- To jumpstart your first AI project, explore available [templates](https://vercel.com/templates?type=ai).

---
title: Nuxt
description: Welcome to the AI SDK quickstart guide for Nuxt!
---

# Nuxt Quickstart

The AI SDK is a powerful Typescript library designed to help developers build AI-powered applications.

In this quickstart tutorial, you'll build a simple AI-chatbot with a streaming user interface. Along the way, you'll learn key concepts and techniques that are fundamental to using the SDK in your own projects.

If you are unfamiliar with the concepts of [Prompt Engineering](/docs/advanced/prompt-engineering) and [HTTP Streaming](/docs/advanced/why-streaming), you can optionally read these documents first.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Setup Your Application

Start by creating a new Nuxt application. This command will create a new directory named `my-ai-app` and set up a basic Nuxt application inside it.

<Snippet text="pnpm dlx nuxi@latest init my-ai-app" />

Navigate to the newly created directory:

<Snippet text="cd my-ai-app" />

### Install dependencies

Install `ai` and `@ai-sdk/openai`, the AI SDK's OpenAI provider.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>
<div className="my-4">
  <Tabs items={['pnpm', 'npm', 'yarn']}>
    <Tab>
      <Snippet text="pnpm add ai @ai-sdk/openai @ai-sdk/vue zod" dark />
    </Tab>
    <Tab>
      <Snippet text="npm install ai @ai-sdk/openai @ai-sdk/vue zod" dark />
    </Tab>
    <Tab>
      <Snippet text="yarn add ai @ai-sdk/openai @ai-sdk/vue zod" dark />
    </Tab>
  </Tabs>
</div>

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.1 or higher.
</Note>

### Configure OpenAI API key

Create a `.env` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env" />

Edit the `.env` file:

```env filename=".env"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key and configure the environment variable in `nuxt.config.ts`:

```ts filename="nuxt.config.ts"
export default defineNuxtConfig({
  // rest of your nuxt config
  runtimeConfig: {
    openaiApiKey: process.env.OPENAI_API_KEY,
  },
});
```

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Create an API route

Create an API route, `server/api/chat.ts` and add the following code:

```typescript filename="server/api/chat.ts"
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';

export default defineLazyEventHandler(async () => {
  const apiKey = useRuntimeConfig().openaiApiKey;
  if (!apiKey) throw new Error('Missing OpenAI API key');
  const openai = createOpenAI({
    apiKey: apiKey,
  });

  return defineEventHandler(async (event: any) => {
    const { messages } = await readBody(event);

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
    });

    return result.toDataStreamResponse();
  });
});
```

Let's take a look at what is happening in this code:

1. Create an OpenAI provider instance with the `createOpenAI` function from the `@ai-sdk/openai` package.
2. Define an Event Handler and extract `messages` from the body of the request. The `messages` variable contains a history of the conversation with you and the chatbot and will provide the chatbot with the necessary context to make the next generation.
3. Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider (defined in step 1) and `messages` (defined in step 2). You can pass additional [settings](/docs/ai-sdk-core/settings) to further customise the model's behaviour.
4. The `streamText` function returns a [`StreamTextResult`](/docs/reference/ai-sdk-core/stream-text#result). This result object contains the [ `toDataStreamResponse` ](/docs/reference/ai-sdk-core/stream-text#to-data-stream-response) function which converts the result to a streamed response object.
5. Return the result to the client to stream the response.

## Wire up the UI

Now that you have an API route that can query an LLM, it's time to setup your frontend. The AI SDK's [ UI ](/docs/ai-sdk-ui/overview) package abstract the complexity of a chat interface into one hook, [`useChat`](/docs/reference/ai-sdk-ui/use-chat).

Update your root page (`pages/index.vue`) with the following code to show a list of chat messages and provide a user message input:

```typescript filename="pages/index.vue"
<script setup lang="ts">
import { useChat } from '@ai-sdk/vue';

const { messages, input, handleSubmit } = useChat();
</script>

<template>
  <div class="flex flex-col w-full max-w-md py-24 mx-auto stretch">
    <div v-for="m in messages" :key="m.id" class="whitespace-pre-wrap">
      {{ m.role === 'user' ? 'User: ' : 'AI: ' }}
      {{ m.content }}
    </div>

    <form @submit="handleSubmit">
      <input
        class="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
        v-model="input"
        placeholder="Say something..."
      />
    </form>
  </div>
</template>
```

<Note>
  If your project has `app.vue` instead of `pages/index.vue`, delete the
  `app.vue` file and create a new `pages/index.vue` file with the code above.
</Note>

This page utilizes the `useChat` hook, which will, by default, use the API route you created earlier (`/api/chat`). The hook provides functions and state for handling user input and form submission. The `useChat` hook provides multiple utility functions and state variables:

- `messages` - the current chat messages (an array of objects with `id`, `role`, and `content` properties).
- `input` - the current value of the user's input field.
- `handleSubmit` - function to handle form submission.

## Running Your Application

With that, you have built everything you need for your chatbot! To start your application, use the command:

<Snippet text="pnpm run dev" />

Head to your browser and open http://localhost:3000. You should see an input field. Test it out by entering a message and see the AI chatbot respond in real-time! The AI SDK makes it fast and easy to build AI chat interfaces with Nuxt.

## Enhance Your Chatbot with Tools

While large language models (LLMs) have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). This is where [tools](/docs/ai-sdk-core/tools-and-tool-calling) come in.

Tools are actions that an LLM can invoke. The results of these actions can be reported back to the LLM to be considered in the next response.

For example, if a user asks about the current weather, without tools, the model would only be able to provide general information based on its training data. But with a weather tool, it can fetch and provide up-to-date, location-specific weather information.

Let's enhance your chatbot by adding a simple weather tool.

### Update Your API Route

Modify your `server/api/chat.ts` file to include the new weather tool:

```typescript filename="server/api/chat.ts" highlight="1,18-34"
import { streamText, tool } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { z } from 'zod';

export default defineLazyEventHandler(async () => {
  const apiKey = useRuntimeConfig().openaiApiKey;
  if (!apiKey) throw new Error('Missing OpenAI API key');
  const openai = createOpenAI({
    apiKey: apiKey,
  });

  return defineEventHandler(async (event: any) => {
    const { messages } = await readBody(event);

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (farenheit)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => {
            const temperature = Math.round(Math.random() * (90 - 32) + 32);
            return {
              location,
              temperature,
            };
          },
        }),
      },
    });

    return result.toDataStreamResponse();
  });
});
```

In this updated code:

1. You import the `tool` function from the `ai` package and `z` from `zod` for schema validation.
2. You define a `tools` object with a `weather` tool. This tool:

   - Has a description that helps the model understand when to use it.
   - Defines parameters using a Zod schema, specifying that it requires a `location` string to execute this tool. The model will attempt to extract this parameter from the context of the conversation. If it can't, it will ask the user for the missing information.
   - Defines an `execute` function that simulates getting weather data (in this case, it returns a random temperature). This is an asynchronous function running on the server so you can fetch real data from an external API.

Now your chatbot can "fetch" weather information for any location the user asks about. When the model determines it needs to use the weather tool, it will generate a tool call with the necessary parameters. The `execute` function will then be automatically run, and you can access the results via `toolInvocations` that is available on the message object.

Try asking something like "What's the weather in New York?" and see how the model uses the new tool.

Notice the blank response in the UI? This is because instead of generating a text response, the model generated a tool call. You can access the tool call and subsequent tool result in the `toolInvocations` key of the message object.

### Update the UI

To display the tool invocations in your UI, update your `pages/index.vue` file:

```typescript filename="pages/index.vue" highlight="11-15"
<script setup lang="ts">
import { useChat } from '@ai-sdk/vue';

const { messages, input, handleSubmit } = useChat();
</script>

<template>
  <div class="flex flex-col w-full max-w-md py-24 mx-auto stretch">
    <div v-for="m in messages" :key="m.id" class="whitespace-pre-wrap">
      {{ m.role === 'user' ? 'User: ' : 'AI: ' }}
      <template v-if="m.toolInvocations">
        <pre>{{ JSON.stringify(m.toolInvocations, null, 2) }}</pre>
      </template>
      <template v-else>
        <p>{{ m.content }}</p>
      </template>
    </div>

    <form @submit="handleSubmit">
      <input
        class="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
        v-model="input"
        placeholder="Say something..."
      />
    </form>
  </div>
</template>
```

With this change, you check each message for any tool calls (`toolInvocations`). These tool calls will be displayed as stringified JSON. Otherwise, you show the message content as before.

Now, when you ask about the weather, you'll see the tool invocation and its result displayed in your chat interface.

## Enabling Multi-Step Tool Calls

You may have noticed that while the tool results are visible in the chat interface, the model isn't using this information to answer your original query. This is because once the model generates a tool call, it has technically completed its generation.

To solve this, you can enable multi-step tool calls using the `maxSteps` option in your `useChat` hook. This feature will automatically send tool results back to the model to trigger an additional generation. In this case, you want the model to answer your question using the results from the weather tool.

### Update Your Client-Side Code

Modify your `pages/index.vue` file to include the `maxSteps` option:

```typescript filename="pages/index.vue" highlight="4"
<script setup lang="ts">
import { useChat } from '@ai-sdk/vue';

const { messages, input, handleSubmit } = useChat({ maxSteps: 5 });
</script>

<!-- ... rest of your component code -->
```

Head back to the browser and ask about the weather in a location. You should now see the model using the weather tool results to answer your question.

By setting `maxSteps` to 5, you're allowing the model to use up to 5 "steps" for any given generation. This enables more complex interactions and allows the model to gather and process information over several steps if needed. You can see this in action by adding another tool to convert the temperature from Fahrenheit to Celsius.

### Update Your API Route

Update your `server/api/chat.ts` file to add a new tool to convert the temperature from Fahrenheit to Celsius:

```typescript filename="server/api/chat.ts" highlight="34-47"
import { streamText, tool } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { z } from 'zod';

export default defineLazyEventHandler(async () => {
  const apiKey = useRuntimeConfig().openaiApiKey;
  if (!apiKey) throw new Error('Missing OpenAI API key');
  const openai = createOpenAI({
    apiKey: apiKey,
  });

  return defineEventHandler(async (event: any) => {
    const { messages } = await readBody(event);

    const result = streamText({
      model: openai('gpt-4-turbo-preview'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (farenheit)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => {
            const temperature = Math.round(Math.random() * (90 - 32) + 32);
            return {
              location,
              temperature,
            };
          },
        }),
        convertFarenheitToCelsius: tool({
          description: 'Convert a temperature in farenheit to celsius',
          parameters: z.object({
            temperature: z
              .number()
              .describe('The temperature in farenheit to convert'),
          }),
          execute: async ({ temperature }) => {
            const celsius = Math.round((temperature - 32) * (5 / 9));
            return {
              celsius,
            };
          },
        }),
      },
    });

    return result.toDataStreamResponse();
  });
});
```

Now, when you ask "What's the weather in New York in celsius?", you should see a more complete interaction:

1. The model will call the weather tool for New York.
2. You'll see the tool result displayed.
3. It will then call the temperature conversion tool to convert the temperature from Fahrenheit to Celsius.
4. The model will then use that information to provide a natural language response about the weather in New York.

This multi-step approach allows the model to gather information and use it to provide more accurate and contextual responses, making your chatbot considerably more useful.

This simple example demonstrates how tools can expand your model's capabilities. You can create more complex tools to integrate with real APIs, databases, or any other external systems, allowing the model to access and process real-world data in real-time. Tools bridge the gap between the model's knowledge cutoff and current information.

## Where to Next?

You've built an AI chatbot using the AI SDK! From here, you have several paths to explore:

- To learn more about the AI SDK, read through the [documentation](/docs).
- If you're interested in diving deeper with guides, check out the [RAG (retrieval-augmented generation)](/docs/guides/rag-chatbot) and [multi-modal chatbot](/docs/guides/multi-modal-chatbot) guides.
- To jumpstart your first AI project, explore available [templates](https://vercel.com/templates?type=ai).

---
title: Node.js
description: Welcome to the AI SDK quickstart guide for Node.js!
---

# Node.js Quickstart

In this quickstart tutorial, you'll build a simple AI chatbot with a streaming user interface. Along the way, you'll learn key concepts and techniques that are fundamental to using the SDK in your own projects.

If you are unfamiliar with the concepts of [Prompt Engineering](/docs/advanced/prompt-engineering) and [HTTP Streaming](/docs/advanced/why-streaming), you can optionally read these documents first.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Setup Your Application

Start by creating a new directory using the `mkdir` command. Change into your new directory and then run the `pnpm init` command. This will create a `package.json` in your new directory.

```bash
mkdir my-ai-app
cd my-ai-app
pnpm init
```

### Install Dependencies

Install `ai` and `@ai-sdk/openai`, the AI SDK's OpenAI provider, along with other necessary dependencies.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>

```bash
pnpm add ai @ai-sdk/openai zod dotenv
pnpm add -D @types/node tsx typescript
```

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.1 or higher.
</Note>

The `ai` and `@ai-sdk/openai` packages contain the AI SDK and the [ AI SDK OpenAI provider](/providers/ai-sdk-providers/openai), respectively. You will use `zod` to define type-safe schemas that you will pass to the large language model (LLM). You will use `dotenv` to access environment variables (your OpenAI key) within your application. There are also three development dependencies, installed with the `-D` flag, that are necessary to run your Typescript code.

### Configure OpenAI API key

Create a `.env` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env" />

Edit the `.env` file:

```env filename=".env"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key.

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Create Your Application

Create an `index.ts` file in the root of your project and add the following code:

```ts filename="index.ts"
import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText } from 'ai';
import dotenv from 'dotenv';
import * as readline from 'node:readline/promises';

dotenv.config();

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
    });

    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push({ role: 'assistant', content: fullResponse });
  }
}

main().catch(console.error);
```

Let's take a look at what is happening in this code:

1. Set up a readline interface for taking input from the terminal, enabling interactive sessions directly from the command line.
2. Initialize an array called `messages` to store the history of your conversation. This history allows the model to maintain context in ongoing dialogues.
3. In the `main` function:

- Prompt for and capture user input, storing it in `userInput`.
- Add user input to the `messages` array as a user message.
- Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider and `messages`.
- Iterate over the text stream returned by the `streamText` function (`result.textStream`) and print the contents of the stream to the terminal.
- Add the assistant's response to the `messages` array.

## Running Your Application

With that, you have built everything you need for your chatbot! To start your application, use the command:

<Snippet text="pnpm tsx index.ts" />

You should see a prompt in your terminal. Test it out by entering a message and see the AI chatbot respond in real-time! The AI SDK makes it fast and easy to build AI chat interfaces with Node.js.

## Enhance Your Chatbot with Tools

While large language models (LLMs) have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). This is where [tools](/docs/ai-sdk-core/tools-and-tool-calling) come in.

Tools are actions that an LLM can invoke. The results of these actions can be reported back to the LLM to be considered in the next response.

For example, if a user asks about the current weather, without tools, the model would only be able to provide general information based on its training data. But with a weather tool, it can fetch and provide up-to-date, location-specific weather information.

Let's enhance your chatbot by adding a simple weather tool.

### Update Your Application

Modify your `index.ts` file to include the new weather tool:

```ts filename="index.ts" highlight="2,25-36"
import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';
import * as readline from 'node:readline/promises';

dotenv.config();

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (in Celsius)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: Math.round((Math.random() * 30 + 5) * 10) / 10, // Random temp between 5°C and 35°C
          }),
        }),
      },
    });

    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push({ role: 'assistant', content: fullResponse });
  }
}

main().catch(console.error);
```

In this updated code:

1. You import the `tool` function from the `ai` package.
2. You define a `tools` object with a `weather` tool. This tool:
   - Has a description that helps the model understand when to use it.
   - Defines parameters using a Zod schema, specifying that it requires a `location` string to execute this tool.
   - Defines an `execute` function that simulates getting weather data (in this case, it returns a random temperature). This is an asynchronous function, so you could fetch real data from an external API.

Now your chatbot can "fetch" weather information for any location the user asks about. When the model determines it needs to use the weather tool, it will generate a tool call with the necessary parameters. The `execute` function will then be automatically run, and the results will be used by the model to generate its response.

Try asking something like "What's the weather in New York?" and see how the model uses the new tool.

Notice the blank "assistant" response? This is because instead of generating a text response, the model generated a tool call. You can access the tool call and subsequent tool result in the `toolCall` and `toolResult` keys of the result object.

```typescript highlight="47-48"
import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';
import * as readline from 'node:readline/promises';

dotenv.config();

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (in Celsius)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: Math.round((Math.random() * 30 + 5) * 10) / 10, // Random temp between 5°C and 35°C
          }),
        }),
      },
    });

    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    console.log(await result.toolCalls);
    console.log(await result.toolResults);
    messages.push({ role: 'assistant', content: fullResponse });
  }
}

main().catch(console.error);
```

Now, when you ask about the weather, you'll see the tool call and its result displayed in your chat interface.

## Enabling Multi-Step Tool Calls

You may have noticed that while the tool results are visible in the chat interface, the model isn't using this information to answer your original query. This is because once the model generates a tool call, it has technically completed its generation.

To solve this, you can enable multi-step tool calls using `maxSteps`. This feature will automatically send tool results back to the model to trigger an additional generation. In this case, you want the model to answer your question using the results from the weather tool.

### Update Your Application

Modify your `index.ts` file to include the `maxSteps` option:

```ts filename="index.ts" highlight="37-40"
import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';
import * as readline from 'node:readline/promises';

dotenv.config();

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (in Celsius)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: Math.round((Math.random() * 30 + 5) * 10) / 10, // Random temp between 5°C and 35°C
          }),
        }),
      },
      maxSteps: 5,
      onStepFinish: step => {
        console.log(JSON.stringify(step, null, 2));
      },
    });

    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push({ role: 'assistant', content: fullResponse });
  }
}

main().catch(console.error);
```

In this updated code:

1. You set `maxSteps` to 5, allowing the model to use up to 5 "steps" for any given generation.
2. You add an `onStepFinish` callback to log each step of the interaction, helping you understand the model's tool usage. This means we can also delete the `toolCall` and `toolResult` `console.log` statements from the previous example.

Now, when you ask about the weather in a location, you should see the model using the weather tool results to answer your question.

By setting `maxSteps` to 5, you're allowing the model to use up to 5 "steps" for any given generation. This enables more complex interactions and allows the model to gather and process information over several steps if needed. You can see this in action by adding another tool to convert the temperature from Celsius to Fahrenheit.

### Adding a second tool

Update your `index.ts` file to add a new tool to convert the temperature from Celsius to Fahrenheit:

```ts filename="index.ts" highlight="36-45"
import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';
import * as readline from 'node:readline/promises';

dotenv.config();

const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');

    messages.push({ role: 'user', content: userInput });

    const result = streamText({
      model: openai('gpt-4-turbo'),
      messages,
      tools: {
        weather: tool({
          description: 'Get the weather in a location (in Celsius)',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: Math.round((Math.random() * 30 + 5) * 10) / 10, // Random temp between 5°C and 35°C
          }),
        }),
        convertCelsiusToFahrenheit: tool({
          description: 'Convert a temperature from Celsius to Fahrenheit',
          parameters: z.object({
            celsius: z
              .number()
              .describe('The temperature in Celsius to convert'),
          }),
          execute: async ({ celsius }) => {
            const fahrenheit = (celsius * 9) / 5 + 32;
            return { fahrenheit: Math.round(fahrenheit * 100) / 100 };
          },
        }),
      },
      maxSteps: 5,
      onStepFinish: step => {
        console.log(JSON.stringify(step, null, 2));
      },
    });

    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');

    messages.push({ role: 'assistant', content: fullResponse });
  }
}

main().catch(console.error);
```

Now, when you ask "What's the weather in New York in Celsius?", you should see a more complete interaction:

1. The model will call the weather tool for New York.
2. You'll see the tool result logged.
3. It will then call the temperature conversion tool to convert the temperature from Celsius to Fahrenheit.
4. The model will then use that information to provide a natural language response about the weather in New York.

This multi-step approach allows the model to gather information and use it to provide more accurate and contextual responses, making your chatbot considerably more useful.

This example shows how tools can expand the model's capabilities. You can create more complex tools to integrate with real APIs, databases, or any other external systems, allowing the model to access and process real-world data in real-time. Tools bridge the gap between the model's knowledge cutoff and current information.

## Where to Next?

You've built an AI chatbot using the AI SDK! From here, you have several paths to explore:

- To learn more about the AI SDK, read through the [documentation](/docs).
- If you're interested in diving deeper with guides, check out the [RAG (retrieval-augmented generation)](/docs/guides/rag-chatbot) and [multi-modal chatbot](/docs/guides/multi-modal-chatbot) guides.
- To jumpstart your first AI project, explore available [templates](https://vercel.com/templates?type=ai).

---
title: Getting Started
description: Welcome to the AI SDK documentation!
---

# Getting Started

The following guides are intended to provide you with an introduction to some of the core features provided by the AI SDK.

<QuickstartFrameworkCards />

## Backend Framework Examples

You can also use [AI SDK Core](/docs/ai-sdk-core/overview) and [AI SDK UI](/docs/ai-sdk-ui/overview) with the following backend frameworks:

<IndexCards
  cards={[
    {
      title: 'Node.js HTTP Server',
      description: 'Send AI responses from a Node.js HTTP server.',
      href: '/examples/api-servers/node-js-http-server',
    },
    {
      title: 'Express',
      description: 'Send AI responses from an Express server.',
      href: '/examples/api-servers/express',
    },
    {
      title: 'Hono',
      description: 'Send AI responses from a Hono server.',
      href: '/examples/api-servers/hono',
    },
    {
      title: 'Fastify',
      description: 'Send AI responses from a Fastify server.',
      href: '/examples/api-servers/fastify',
    },
    {
      title: 'Nest.js',
      description: 'Send AI responses from a Nest.js server.',
      href: '/examples/api-servers/nest',
    },
  ]}
/>

---
title: RAG Chatbot
description: Learn how to build a RAG Chatbot with the AI SDK and Next.js
---

# RAG Chatbot Guide

In this guide, you will learn how to build a retrieval-augmented generation (RAG) chatbot application.

<video
  src="/images/rag-guide-demo.mp4"
  autoplay
  height={540}
  width={910}
  controls
  playsinline
/>

Before we dive in, let's look at what RAG is, and why we would want to use it.

### What is RAG?

RAG stands for retrieval augmented generation. In simple terms, RAG is the process of providing a Large Language Model (LLM) with specific information relevant to the prompt.

### Why is RAG important?

While LLMs are powerful, the information they can reason on is restricted to the data they were trained on. This problem becomes apparent when asking an LLM for information outside of their training data, like proprietary data or common knowledge that has occurred after the model’s training cutoff. RAG solves this problem by fetching information relevant to the prompt and then passing that to the model as context.

To illustrate with a basic example, imagine asking the model for your favorite food:

```txt
**input**
What is my favorite food?

**generation**
I don't have access to personal information about individuals, including their
favorite foods.
```

Not surprisingly, the model doesn’t know. But imagine, alongside your prompt, the model received some extra context:

```txt
**input**
Respond to the user's prompt using only the provided context.
user prompt: 'What is my favorite food?'
context: user loves chicken nuggets

**generation**
Your favorite food is chicken nuggets!
```

Just like that, you have augmented the model’s generation by providing relevant information to the query. Assuming the model has the appropriate information, it is now highly likely to return an accurate response to the users query. But how does it retrieve the relevant information? The answer relies on a concept called embedding.

<Note>
  You could fetch any context for your RAG application (eg. Google search).
  Embeddings and Vector Databases are just a specific retrieval approach to
  achieve semantic search.
</Note>

### Embedding

[Embeddings](/docs/ai-sdk-core/embeddings) are a way to represent words, phrases, or images as vectors in a high-dimensional space. In this space, similar words are close to each other, and the distance between words can be used to measure their similarity.

In practice, this means that if you embedded the words `cat` and `dog`, you would expect them to be plotted close to each other in vector space. The process of calculating the similarity between two vectors is called ‘cosine similarity’ where a value of 1 would indicate high similarity and a value of -1 would indicate high opposition.

<Note>
  Don’t worry if this seems complicated. a high level understanding is all you
  need to get started! For a more in-depth introduction to embeddings, check out
  [this guide](https://jalammar.github.io/illustrated-word2vec/).
</Note>

As mentioned above, embeddings are a way to represent the semantic meaning of **words and phrases**. The implication here is that the larger the input to your embedding, the lower quality the embedding will be. So how would you approach embedding content longer than a simple phrase?

### Chunking

Chunking refers to the process of breaking down a particular source material into smaller pieces. There are many different approaches to chunking and it’s worth experimenting as the most effective approach can differ by use case. A simple and common approach to chunking (and what you will be using in this guide) is separating written content by sentences.

Once your source material is appropriately chunked, you can embed each one and then store the embedding and the chunk together in a database. Embeddings can be stored in any database that supports vectors. For this tutorial, you will be using [Postgres](https://www.postgresql.org/) alongside the [pgvector](https://github.com/pgvector/pgvector) plugin.

<MDXImage
  srcLight="/images/rag-guide-1.png"
  srcDark="/images/rag-guide-1-dark.png"
  width={800}
  height={800}
/>

### All Together Now

Combining all of this together, RAG is the process of enabling the model to respond with information outside of it’s training data by embedding a users query, retrieving the relevant source material (chunks) with the highest semantic similarity, and then passing them alongside the initial query as context. Going back to the example where you ask the model for your favorite food, the prompt preparation process would look like this.

<MDXImage
  srcLight="/images/rag-guide-2.png"
  srcDark="/images/rag-guide-2-dark.png"
  width={800}
  height={800}
/>

By passing the appropriate context and refining the model’s objective, you are able to fully leverage its strengths as a reasoning machine.

Onto the project!

## Project Setup

In this project, you will build a chatbot that will only respond with information that it has within its knowledge base. The chatbot will be able to both store and retrieve information. This project has many interesting use cases from customer support through to building your own second brain!

This project will use the following stack:

- [Next.js](https://nextjs.org) 14 (App Router)
- [ AI SDK ](https://sdk.vercel.ai/docs)
- [OpenAI](https://openai.com)
- [ Drizzle ORM ](https://orm.drizzle.team)
- [ Postgres ](https://www.postgresql.org/) with [ pgvector ](https://github.com/pgvector/pgvector)
- [ shadcn-ui ](https://ui.shadcn.com) and [ TailwindCSS ](https://tailwindcss.com) for styling

### Clone Repo

To reduce the scope of this guide, you will be starting with a [repository](https://github.com/vercel/ai-sdk-rag-starter) that already has a few things set up for you:

- Drizzle ORM (`lib/db`) including an initial migration and a script to migrate (`db:migrate`)
- a basic schema for the `resources` table (this will be for source material)
- a Server Action for creating a `resource`

To get started, clone the starter repository with the following command:

<Snippet
  text={[
    'git clone https://github.com/vercel/ai-sdk-rag-starter',
    'cd ai-sdk-rag-starter',
  ]}
/>

First things first, run the following command to install the project’s dependencies:

<Snippet text="pnpm install" />

### Create Database

You will need a Postgres database to complete this tutorial. If you don’t have Postgres setup on your local machine you can:

- Create a free Postgres database with [Vercel Postgres](https://vercel.com/docs/storage/vercel-postgres); or
- Follow [this guide](https://www.prisma.io/dataguide/postgresql/setting-up-a-local-postgresql-database) to set it up locally

### Migrate Database

Once you have a Postgres database, you need to add the connection string as an environment secret.

Make a copy of the `.env.example` file and rename it to `.env`.

<Snippet text="cp .env.example .env" />

Open the new `.env` file. You should see an item called `DATABASE_URL`. Copy in your database connection string after the equals sign.

With that set up, you can now run your first database migration. Run the following command:

<Snippet text="pnpm db:migrate" />

This will first add the `pgvector` extension to your database. Then it will create a new table for your `resources` schema that is defined in `lib/db/schema/resources.ts`. This schema has four columns: `id`, `content`, `createdAt`, and `updatedAt`.

<Note>
  If you experience an error with the migration, open your migration file
  (`lib/db/migrations/0000_yielding_bloodaxe.sql`), cut (copy and remove) the
  first line, and run it directly on your postgres instance. You should now be
  able to run the updated migration. [More
  info](https://github.com/vercel/ai-sdk-rag-starter/issues/1).
</Note>

### OpenAI API Key

For this guide, you will need an OpenAI API key. To generate an API key, go to [platform.openai.com](http://platform.openai.com/).

Once you have your API key, paste it into your `.env` file (`OPENAI_API_KEY`).

## Build

Let’s build a quick task list of what needs to be done:

1. Create a table in your database to store embeddings
2. Add logic to chunk and create embeddings when creating resources
3. Create a chatbot
4. Give the chatbot tools to query / create resources for it’s knowledge base

### Create Embeddings Table

Currently, your application has one table (`resources`) which has a column (`content`) for storing content. Remember, each `resource` (source material) will have to be chunked, embedded, and then stored. Let’s create a table called `embeddings` to store these chunks.

Create a new file (`lib/db/schema/embeddings.ts`) and add the following code:

```tsx filename="lib/db/schema/embeddings.ts"
import { generateId } from 'ai';
import { index, pgTable, text, varchar, vector } from 'drizzle-orm/pg-core';
import { resources } from './resources';

export const embeddings = pgTable(
  'embeddings',
  {
    id: varchar('id', { length: 191 })
      .primaryKey()
      .$defaultFn(() => generateId()),
    resourceId: varchar('resource_id', { length: 191 }).references(
      () => resources.id,
      { onDelete: 'cascade' },
    ),
    content: text('content').notNull(),
    embedding: vector('embedding', { dimensions: 1536 }).notNull(),
  },
  table => ({
    embeddingIndex: index('embeddingIndex').using(
      'hnsw',
      table.embedding.op('vector_cosine_ops'),
    ),
  }),
);
```

This table has four columns:

- `id` - unique identifier
- `resourceId` - a foreign key relation to the full source material
- `content` - the plain text chunk
- `embedding` - the vector representation of the plain text chunk

To perform similarity search, you also need to include an index ([HNSW](https://github.com/pgvector/pgvector?tab=readme-ov-file#hnsw) or [IVFFlat](https://github.com/pgvector/pgvector?tab=readme-ov-file#ivfflat)) on this column for better performance.

To push this change to the database, run the following command:

<Snippet text="pnpm db:push" />

### Add Embedding Logic

Now that you have a table to store embeddings, it’s time to write the logic to create the embeddings.

Create a file with the following command:

<Snippet text="mkdir lib/ai && touch lib/ai/embedding.ts" />

### Generate Chunks

Remember, to create an embedding, you will start with a piece of source material (unknown length), break it down into smaller chunks, embed each chunk, and then save the chunk to the database. Let’s start by creating a function to break the source material into small chunks.

```tsx filename="lib/ai/embedding.ts"
const generateChunks = (input: string): string[] => {
  return input
    .trim()
    .split('.')
    .filter(i => i !== '');
};
```

This function will take an input string and split it by periods, filtering out any empty items. This will return an array of strings. It is worth experimenting with different chunking techniques in your projects as the best technique will vary.

### Install AI SDK

You will use the AI SDK to create embeddings. This will require two more dependencies, which you can install by running the following command:

<Snippet text="pnpm add ai @ai-sdk/openai" />

This will install the [AI SDK](https://sdk.vercel.ai/docs) and the [OpenAI provider](/providers/ai-sdk-providers/openai).

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>

### Generate Embeddings

Let’s add a function to generate embeddings. Copy the following code into your `lib/ai/embedding.ts` file.

```tsx filename="lib/ai/embedding.ts" highlight="1-2,4,13-22"
import { embedMany } from 'ai';
import { openai } from '@ai-sdk/openai';

const embeddingModel = openai.embedding('text-embedding-ada-002');

const generateChunks = (input: string): string[] => {
  return input
    .trim()
    .split('.')
    .filter(i => i !== '');
};

export const generateEmbeddings = async (
  value: string,
): Promise<Array<{ embedding: number[]; content: string }>> => {
  const chunks = generateChunks(value);
  const { embeddings } = await embedMany({
    model: embeddingModel,
    values: chunks,
  });
  return embeddings.map((e, i) => ({ content: chunks[i], embedding: e }));
};
```

In this code, you first define the model you want to use for the embeddings. In this example, you are using OpenAI’s `text-embedding-ada-002` embedding model.

Next, you create an asynchronous function called `generateEmbeddings`. This function will take in the source material (`value`) as an input and return a promise of an array of objects, each containing an embedding and content. Within the function, you first generate chunks for the input. Then, you pass those chunks to the [`embedMany`](/docs/reference/ai-sdk-core/embed-many) function imported from the AI SDK which will return embeddings of the chunks you passed in. Finally, you map over and return the embeddings in a format that is ready to save in the database.

### Update Server Action

Open the file at `lib/actions/resources.ts`. This file has one function, `createResource`, which, as the name implies, allows you to create a resource.

```tsx filename="lib/actions/resources.ts"
'use server';

import {
  NewResourceParams,
  insertResourceSchema,
  resources,
} from '@/lib/db/schema/resources';
import { db } from '../db';

export const createResource = async (input: NewResourceParams) => {
  try {
    const { content } = insertResourceSchema.parse(input);

    const [resource] = await db
      .insert(resources)
      .values({ content })
      .returning();

    return 'Resource successfully created.';
  } catch (e) {
    if (e instanceof Error)
      return e.message.length > 0 ? e.message : 'Error, please try again.';
  }
};
```

This function is a [Server Action](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#with-client-components), as denoted by the `“use server”;` directive at the top of the file. This means that it can be called anywhere in your Next.js application. This function will take an input, run it through a [Zod](https://zod.dev) schema to ensure it adheres to the correct schema, and then creates a new resource in the database. This is the ideal location to generate and store embeddings of the newly created resources.

Update the file with the following code:

```tsx filename="lib/actions/resources.ts" highlight="9-10,21-27,29"
'use server';

import {
  NewResourceParams,
  insertResourceSchema,
  resources,
} from '@/lib/db/schema/resources';
import { db } from '../db';
import { generateEmbeddings } from '../ai/embedding';
import { embeddings as embeddingsTable } from '../db/schema/embeddings';

export const createResource = async (input: NewResourceParams) => {
  try {
    const { content } = insertResourceSchema.parse(input);

    const [resource] = await db
      .insert(resources)
      .values({ content })
      .returning();

    const embeddings = await generateEmbeddings(content);
    await db.insert(embeddingsTable).values(
      embeddings.map(embedding => ({
        resourceId: resource.id,
        ...embedding,
      })),
    );

    return 'Resource successfully created and embedded.';
  } catch (error) {
    return error instanceof Error && error.message.length > 0
      ? error.message
      : 'Error, please try again.';
  }
};
```

First, you call the `generateEmbeddings` function created in the previous step, passing in the source material (`content`). Once you have your embeddings (`e`) of the source material, you can save them to the database, passing the `resourceId` alongside each embedding.

### Create Root Page

Great! Let's build the frontend. The AI SDK’s [`useChat`](/docs/reference/ai-sdk-ui/use-chat) hook allows you to easily create a conversational user interface for your chatbot application.

Replace your root page (`app/page.tsx`) with the following code.

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      <div className="space-y-4">
        {messages.map(m => (
          <div key={m.id} className="whitespace-pre-wrap">
            <div>
              <div className="font-bold">{m.role}</div>
              <p>{m.content}</p>
            </div>
          </div>
        ))}
      </div>

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

The `useChat` hook enables the streaming of chat messages from your AI provider (you will be using OpenAI), manages the state for chat input, and updates the UI automatically as new messages are received.

Run the following command to start the Next.js dev server:

<Snippet text="pnpm run dev" />

Head to [http://localhost:3000](http://localhost:3000/). You should see an empty screen with an input bar floating at the bottom. Try to send a message. The message shows up in the UI for a fraction of a second and then disappears. This is because you haven’t set up the corresponding API route to call the model! By default, `useChat` will send a POST request to the `/api/chat` endpoint with the `messages` as the request body.

<Note>You can customize the endpoint in the useChat configuration object</Note>

### Create API Route

In Next.js, you can create custom request handlers for a given route using [Route Handlers](https://nextjs.org/docs/app/building-your-application/routing/route-handlers). Route Handlers are defined in a `route.ts` file and can export HTTP methods like `GET`, `POST`, `PUT`, `PATCH` etc.

Create a file at `app/api/chat/route.ts` by running the following command:

<Snippet text="mkdir -p app/api/chat && touch app/api/chat/route.ts" />

Open the file and add the following code:

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

In this code, you declare and export an asynchronous function called POST. You retrieve the `messages` from the request body and then pass them to the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function imported from the AI SDK, alongside the model you would like to use. Finally, you return the model’s response in `AIStreamResponse` format.

Head back to the browser and try to send a message again. You should see a response from the model streamed directly in!

### Refining your prompt

While you now have a working chatbot, it isn't doing anything special.

Let’s add system instructions to refine and restrict the model’s behavior. In this case, you want the model to only use information it has retrieved to generate responses. Update your route handler with the following code:

```tsx filename="app/api/chat/route.ts" highlight="12-14"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: `You are a helpful assistant. Check your knowledge base before answering any questions.
    Only respond to questions using information from tool calls.
    if no relevant information is found in the tool calls, respond, "Sorry, I don't know."`,
    messages,
  });

  return result.toDataStreamResponse();
}
```

Head back to the browser and try to ask the model what your favorite food is. The model should now respond exactly as you instructed above (“Sorry, I don’t know”) given it doesn’t have any relevant information.

In its current form, your chatbot is now, well, useless. How do you give the model the ability to add and query information?

### Using Tools

A [tool](/docs/foundations/tools) is a function that can be called by the model to perform a specific task. You can think of a tool like a program you give to the model that it can run as and when it deems necessary.

Let’s see how you can create a tool to give the model the ability to create, embed and save a resource to your chatbots’ knowledge base.

### Add Resource Tool

Update your route handler with the following code:

```tsx filename="app/api/chat/route.ts" highlight="18-29"
import { createResource } from '@/lib/actions/resources';
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: `You are a helpful assistant. Check your knowledge base before answering any questions.
    Only respond to questions using information from tool calls.
    if no relevant information is found in the tool calls, respond, "Sorry, I don't know."`,
    messages,
    tools: {
      addResource: tool({
        description: `add a resource to your knowledge base.
          If the user provides a random piece of knowledge unprompted, use this tool without asking for confirmation.`,
        parameters: z.object({
          content: z
            .string()
            .describe('the content or resource to add to the knowledge base'),
        }),
        execute: async ({ content }) => createResource({ content }),
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

In this code, you define a tool called `addResource`. This tool has three elements:

- **description**: description of the tool that will influence when the tool is picked.
- **parameters**: [Zod schema](https://sdk.vercel.ai/docs/foundations/tools#schema-specification-and-validation-with-zod) that defines the parameters necessary for the tool to run.
- **execute**: An asynchronous function that is called with the arguments from the tool call.

In simple terms, on each generation, the model will decide whether it should call the tool. If it deems it should call the tool, it will extract the parameters from the input and then append a new `message` to the `messages` array of type `tool-call`. The AI SDK will then run the `execute` function with the parameters provided by the `tool-call` message.

Head back to the browser and tell the model your favorite food. You should see an empty response in the UI. Did anything happen? Let’s see. Run the following command in a new terminal window.

<Snippet text="pnpm db:studio" />

This will start Drizzle Studio where we can view the rows in our database. You should see a new row in both the `embeddings` and `resources` table with your favorite food!

Let’s make a few changes in the UI to communicate to the user when a tool has been called. Head back to your root page (`app/page.tsx`) and add the following code:

```tsx filename="app/page.tsx" highlight="14-22"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      <div className="space-y-4">
        {messages.map(m => (
          <div key={m.id} className="whitespace-pre-wrap">
            <div>
              <div className="font-bold">{m.role}</div>
              <p>
                {m.content.length > 0 ? (
                  m.content
                ) : (
                  <span className="italic font-light">
                    {'calling tool: ' + m?.toolInvocations?.[0].toolName}
                  </span>
                )}
              </p>
            </div>
          </div>
        ))}
      </div>

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

With this change, you now conditionally render the tool that has been called directly in the UI. Save the file and head back to browser. Tell the model your favorite movie. You should see which tool is called in place of the model’s typical text response.

### Improving UX with Multi-Step Calls

It would be nice if the model could summarize the action too. However, technically, once the model calls a tool, it has completed its generation as it ‘generated’ a tool call. How could you achieve this desired behaviour?

The AI SDK has a feature called [`maxSteps`](/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls) which will automatically send tool call results back to the model!

Open your root page (`app/page.tsx`) and add the following key to the `useChat` configuration object:

```tsx filename="app/page.tsx" highlight="3-5"
// ... Rest of your code

const { messages, input, handleInputChange, handleSubmit } = useChat({
  maxSteps: 3,
});

// ... Rest of your code
```

Head back to the browser and tell the model your favorite pizza topping (note: pineapple is not an option). You should see a follow-up response from the model confirming the action.

### Retrieve Resource Tool

The model can now add and embed arbitrary information to your knowledge base. However, it still isn’t able to query it. Let’s create a new tool to allow the model to answer questions by finding relevant information in your knowledge base.

To find similar content, you will need to embed the users query, search the database for semantic similarities, then pass those items to the model as context alongside the query. To achieve this, let’s update your embedding logic file (`lib/ai/embedding.ts`):

```tsx filename="lib/ai/embedding.ts" highlight="1,3-5,27-34,36-49"
import { embed, embedMany } from 'ai';
import { openai } from '@ai-sdk/openai';
import { db } from '../db';
import { cosineDistance, desc, gt, sql } from 'drizzle-orm';
import { embeddings } from '../db/schema/embeddings';

const embeddingModel = openai.embedding('text-embedding-ada-002');

const generateChunks = (input: string): string[] => {
  return input
    .trim()
    .split('.')
    .filter(i => i !== '');
};

export const generateEmbeddings = async (
  value: string,
): Promise<Array<{ embedding: number[]; content: string }>> => {
  const chunks = generateChunks(value);
  const { embeddings } = await embedMany({
    model: embeddingModel,
    values: chunks,
  });
  return embeddings.map((e, i) => ({ content: chunks[i], embedding: e }));
};

export const generateEmbedding = async (value: string): Promise<number[]> => {
  const input = value.replaceAll('\\n', ' ');
  const { embedding } = await embed({
    model: embeddingModel,
    value: input,
  });
  return embedding;
};

export const findRelevantContent = async (userQuery: string) => {
  const userQueryEmbedded = await generateEmbedding(userQuery);
  const similarity = sql<number>`1 - (${cosineDistance(
    embeddings.embedding,
    userQueryEmbedded,
  )})`;
  const similarGuides = await db
    .select({ name: embeddings.content, similarity })
    .from(embeddings)
    .where(gt(similarity, 0.5))
    .orderBy(t => desc(t.similarity))
    .limit(4);
  return similarGuides;
};
```

In this code, you add two functions:

- `generateEmbedding`: generate a single embedding from an input string
- `findRelevantContent`: embeds the user’s query, searches the database for similar items, then returns relevant items

With that done, it’s onto the final step: creating the tool.

Go back to your route handler (`api/chat/route.ts`) and add a new tool called `getInformation`:

```ts filename="api/chat/route.ts" highlight="5,30-36"
import { createResource } from '@/lib/actions/resources';
import { openai } from '@ai-sdk/openai';
import { streamText, tool } from 'ai';
import { z } from 'zod';
import { findRelevantContent } from '@/lib/ai/embedding';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages,
    system: `You are a helpful assistant. Check your knowledge base before answering any questions.
    Only respond to questions using information from tool calls.
    if no relevant information is found in the tool calls, respond, "Sorry, I don't know."`,
    tools: {
      addResource: tool({
        description: `add a resource to your knowledge base.
          If the user provides a random piece of knowledge unprompted, use this tool without asking for confirmation.`,
        parameters: z.object({
          content: z
            .string()
            .describe('the content or resource to add to the knowledge base'),
        }),
        execute: async ({ content }) => createResource({ content }),
      }),
      getInformation: tool({
        description: `get information from your knowledge base to answer questions.`,
        parameters: z.object({
          question: z.string().describe('the users question'),
        }),
        execute: async ({ question }) => findRelevantContent(question),
      }),
    },
  });

  return result.toDataStreamResponse();
}
```

Head back to the browser, refresh the page, and ask for your favorite food. You should see the model call the `getInformation` tool, and then use the relevant information to formulate a response!

## Conclusion

Congratulations, you have successfully built an AI chatbot that can dynamically add and retrieve information to and from a knowledge base. Throughout this guide, you learned how to create and store embeddings, set up server actions to manage resources, and use tools to extend the capabilities of your chatbot.

---
title: Multi-Modal Chatbot
description: Learn how to build a multi-modal chatbot with the AI SDK!
---

# Multi-Modal Chatbot

In this guide, you will build a multi-modal AI-chatbot with a streaming user interface.

Multi-modal refers to the ability of the chatbot to understand and generate responses in multiple formats, such as text, images, and videos. In this example, we will focus on sending images and generating text-based responses.

## Prerequisites

To follow this quickstart, you'll need:

- Node.js 18+ and pnpm installed on your local development machine.
- An OpenAI API key.

If you haven't obtained your OpenAI API key, you can do so by [signing up](https://platform.openai.com/signup/) on the OpenAI website.

## Create Your Application

Start by creating a new Next.js application. This command will create a new directory named `multi-modal-chatbot` and set up a basic Next.js application inside it.

<div className="mb-4">
  <Note>
    Be sure to select yes when prompted to use the App Router. If you are
    looking for the Next.js Pages Router quickstart guide, you can find it
    [here](/docs/getting-started/nextjs-pages-router).
  </Note>
</div>

<Snippet text="pnpm create next-app@latest multi-modal-chatbot" />

Navigate to the newly created directory:

<Snippet text="cd multi-modal-chatbot" />

### Install dependencies

Install `ai` and `@ai-sdk/openai`, the Vercel AI package and the AI SDK's [ OpenAI provider ](/providers/ai-sdk-providers/openai) respectively.

<Note>
  The AI SDK is designed to be a unified interface to interact with any large
  language model. This means that you can change model and providers with just
  one line of code! Learn more about [available providers](/providers) and
  [building custom providers](/providers/community-providers/custom-providers)
  in the [providers](/providers) section.
</Note>
<div className="my-4">
  <Tabs items={['pnpm', 'npm', 'yarn']}>
    <Tab>
      <Snippet text="pnpm add ai @ai-sdk/openai" dark />
    </Tab>
    <Tab>
      <Snippet text="npm install ai @ai-sdk/openai" dark />
    </Tab>
    <Tab>
      <Snippet text="yarn add ai @ai-sdk/openai" dark />
    </Tab>
  </Tabs>
</div>

<Note type="secondary" fill>
  Make sure you are using `ai` version 3.2.27 or higher.
</Note>

### Configure OpenAI API key

Create a `.env.local` file in your project root and add your OpenAI API Key. This key is used to authenticate your application with the OpenAI service.

<Snippet text="touch .env.local" />

Edit the `.env.local` file:

```env filename=".env.local"
OPENAI_API_KEY=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual OpenAI API key.

<Note className="mb-4">
  The AI SDK's OpenAI Provider will default to using the `OPENAI_API_KEY`
  environment variable.
</Note>

## Implementation Plan

To build a multi-modal chatbot, you will need to:

- Create a Route Handler to handle incoming chat messages and generate responses.
- Wire up the UI to display chat messages, provide a user input, and handle submitting new messages.
- Add the ability to upload images and attach them alongside the chat messages.

## Create a Route Handler

Create a route handler, `app/api/chat/route.ts` and add the following code:

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

Let's take a look at what is happening in this code:

1. Define an asynchronous `POST` request handler and extract `messages` from the body of the request. The `messages` variable contains a history of the conversation between you and the chatbot and provides the chatbot with the necessary context to make the next generation.
2. Call [`streamText`](/docs/reference/ai-sdk-core/stream-text), which is imported from the `ai` package. This function accepts a configuration object that contains a `model` provider (imported from `@ai-sdk/openai`) and `messages` (defined in step 1). You can pass additional [settings](/docs/ai-sdk-core/settings) to further customise the model's behaviour.
3. The `streamText` function returns a [`StreamTextResult`](/docs/reference/ai-sdk-core/stream-text#result-object). This result object contains the [ `toDataStreamResponse` ](/docs/reference/ai-sdk-core/stream-text#to-ai-stream-response) function which converts the result to a streamed response object.
4. Finally, return the result to the client to stream the response.

This Route Handler creates a POST request endpoint at `/api/chat`.

## Wire up the UI

Now that you have a Route Handler that can query a large language model (LLM), it's time to setup your frontend. [ AI SDK UI ](/docs/ai-sdk-ui) abstracts the complexity of a chat interface into one hook, [`useChat`](/docs/reference/ai-sdk-ui/use-chat).

Update your root page (`app/page.tsx`) with the following code to show a list of chat messages and provide a user message input:

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
        </div>
      ))}

      <form
        onSubmit={handleSubmit}
        className="fixed bottom-0 w-full max-w-md mb-8 border border-gray-300 rounded shadow-xl"
      >
        <input
          className="w-full p-2"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

<Note>
  Make sure you add the `"use client"` directive to the top of your file. This
  allows you to add interactivity with Javascript.
</Note>

This page utilizes the `useChat` hook, which will, by default, use the `POST` API route you created earlier (`/api/chat`). The hook provides functions and state for handling user input and form submission. The `useChat` hook provides multiple utility functions and state variables:

- `messages` - the current chat messages (an array of objects with `id`, `role`, and `content` properties).
- `input` - the current value of the user's input field.
- `handleInputChange` and `handleSubmit` - functions to handle user interactions (typing into the input field and submitting the form, respectively).
- `isLoading` - boolean that indicates whether the API request is in progress.

## Add Image Upload

To make your chatbot multi-modal, let's add the ability to upload and send images to the model. There are two ways to send attachments alongside a message with the `useChat` hook: by [ providing a `FileList` object ](/docs/ai-sdk-ui/chatbot#filelist) or a [ list of URLs ](/docs/ai-sdk-ui/chatbot#urls) to the `handleSubmit` function. In this guide, you will be using the `FileList` approach as it does not require any additional setup.

Update your root page (`app/page.tsx`) with the following code:

```tsx filename="app/page.tsx" highlight="4-5,10-11,19-33,39-49,51-61"
'use client';

import { useChat } from 'ai/react';
import { useRef, useState } from 'react';
import Image from 'next/image';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  const [files, setFiles] = useState<FileList | undefined>(undefined);
  const fileInputRef = useRef<HTMLInputElement>(null);

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
          <div>
            {m?.experimental_attachments
              ?.filter(attachment =>
                attachment?.contentType?.startsWith('image/'),
              )
              .map((attachment, index) => (
                <Image
                  key={`${m.id}-${index}`}
                  src={attachment.url}
                  width={500}
                  height={500}
                  alt={attachment.name ?? `attachment-${index}`}
                />
              ))}
          </div>
        </div>
      ))}

      <form
        className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl space-y-2"
        onSubmit={event => {
          handleSubmit(event, {
            experimental_attachments: files,
          });

          setFiles(undefined);

          if (fileInputRef.current) {
            fileInputRef.current.value = '';
          }
        }}
      >
        <input
          type="file"
          className=""
          onChange={event => {
            if (event.target.files) {
              setFiles(event.target.files);
            }
          }}
          multiple
          ref={fileInputRef}
        />
        <input
          className="w-full p-2"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

In this code, you:

1. Create state to hold the files and create a ref to the file input field.
2. Display the "uploaded" files in the UI.
3. Update the `onSubmit` function, to call the `handleSubmit` function manually, passing the the files as an option using the `experimental_attachments` key.
4. Add a file input field to the form, including an `onChange` handler to handle updating the files state.

## Running Your Application

With that, you have built everything you need for your multi-modal chatbot! To start your application, use the command:

<Snippet text="pnpm run dev" />

Head to your browser and open http://localhost:3000. You should see an input field and a button to upload an image.

Upload a file and ask the model to describe what it sees. Watch as the model's response is streamed back to you!

## Where to Next?

You've built a multi-modal AI chatbot using the AI SDK! Experiment and extend the functionality of this application further by exploring [tool calling](/docs/ai-sdk-core/tools-and-tool-calling) or introducing more granular control over [AI and UI states](/docs/ai-sdk-rsc/generative-ui-state).

If you are looking to leverage the broader capabilities of LLMs, Vercel [AI SDK Core](/docs/ai-sdk-core) provides a comprehensive set of lower-level tools and APIs that will help you unlock a wider range of AI functionalities beyond the chatbot paradigm.

---
title: Get started with Llama 3.1
description: Get started with Llama 3.1 using the AI SDK.
---

# Get started with Llama 3.1

With the [release of Llama 3.1](https://ai.meta.com/blog/meta-llama-3-1/), there has never been a better time to start building AI applications.

The [AI SDK](/) is a powerful TypeScript toolkit for building AI application with large language models (LLMs) like Llama 3.1 alongside popular frameworks like React, Next.js, Vue, Svelte, Node.js, and more

## Llama 3.1

The release of Meta's Llama 3.1 is an important moment in AI development. As the first state-of-the-art open weight AI model, Llama 3.1 is helping accelerate developers building AI apps. Available in 8B, 70B, and 405B sizes, these instruction-tuned models work well for tasks like dialogue generation, translation, reasoning, and code generation.

## Benchmarks

Llama 3.1 surpasses most available open-source chat models on common industry benchmarks and even outperforms some closed-source models, offering superior performance in language nuances, contextual understanding, and complex multi-step tasks. The models' refined post-training processes significantly improve response alignment, reduce false refusal rates, and enhance answer diversity, making Llama 3.1 a powerful and accessible tool for building generative AI applications.

![Llama 3.1 Benchmarks](/images/llama-3_1-benchmarks.png)
Source: [Meta AI - Llama 3.1 Model Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md)

## Choosing Model Size

Llama 3.1 includes a new 405B parameter model, becoming the largest open-source model available today. This model is designed to handle the most complex and demanding tasks.

When choosing between the different sizes of Llama 3.1 models (405B, 70B, 8B), consider the trade-off between performance and computational requirements. The 405B model offers the highest accuracy and capability for complex tasks but requires significant computational resources. The 70B model provides a good balance of performance and efficiency for most applications, while the 8B model is suitable for simpler tasks or resource-constrained environments where speed and lower computational overhead are priorities.

## Getting Started with the AI SDK

The AI SDK is the TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more. Integrating LLMs into applications is complicated and heavily dependent on the specific model provider you use.

The AI SDK abstracts away the differences between model providers, eliminates boilerplate code for building chatbots, and allows you to go beyond text output to generate rich, interactive components.

At the center of the AI SDK is [AI SDK Core](/docs/ai-sdk-core/overview), which provides a unified API to call any LLM. The code snippet below is all you need to call Llama 3.1 (using [Groq](https://groq.com)) with the AI SDK:

```tsx
import { generateText } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const { text } = await generateText({
  model: groq('llama-3.1-405b-reasoning'),
  prompt: 'What is love?',
});
```

<Note>
  Llama 3.1 is available to use with many AI SDK providers including
  [Groq](/providers/ai-sdk-providers/groq), [Amazon
  Bedrock](/providers/ai-sdk-providers/amazon-bedrock),
  [Perplexity](/providers/ai-sdk-providers/perplexity),
  [Baseten](/providers/openai-compatible-providers/baseten)
  [Fireworks](/providers/ai-sdk-providers/fireworks), and more.
</Note>

AI SDK Core abstracts away the differences between model providers, allowing you to focus on building great applications. Prefer to use [Amazon Bedrock](/providers/ai-sdk-providers/amazon-bedrock)? The unified interface also means that you can easily switch between models by changing just two lines of code.

```tsx highlight="2,5"
import { generateText } from 'ai';
import { bedrock } from '@ai-sdk/amazon-bedrock';

const { text } = await generateText({
  model: bedrock('meta.llama3-1-405b-instruct-v1'),
  prompt: 'What is love?',
});
```

### Streaming the Response

To stream the model's response as it's being generated, update your code snippet to use the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function.

```tsx
import { streamText } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const { textStream } = streamText({
  model: groq('llama-3.1-70b-versatile'),
  prompt: 'What is love?',
});
```

### Generating Structured Data

While text generation can be useful, you might want to generate structured JSON data. For example, you might want to extract information from text, classify data, or generate synthetic data. AI SDK Core provides two functions ([`generateObject`](/docs/reference/ai-sdk-core/generate-object) and [`streamObject`](/docs/reference/ai-sdk-core/stream-object)) to generate structured data, allowing you to constrain model outputs to a specific schema.

```tsx
import { generateObject } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';
import { z } from 'zod';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const { object } = await generateObject({
  model: groq('llama-3.1-70b-versatile'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});
```

This code snippet will generate a type-safe recipe that conforms to the specified zod schema.

### Tools

While LLMs have incredible generation capabilities, they struggle with discrete tasks (e.g. mathematics) and interacting with the outside world (e.g. getting the weather). The solution: tools, which are like programs that you provide to the model, which it can choose to call as necessary.

### Using Tools with the AI SDK

The AI SDK supports tool usage across several of its functions, including [`generateText`](/docs/reference/ai-sdk-core/generate-text) and [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui). By passing one or more tools to the `tools` parameter, you can extend the capabilities of LLMs, allowing them to perform discrete tasks and interact with external systems.

Here's an example of how you can use a tool with the AI SDK and Llama 3.1:

```tsx
import { generateText, tool } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';
import { getWeather } from './weatherTool';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const { text } = await generateText({
  model: groq('llama-3.1-70b-versatile'),
  prompt: 'What is the weather like today?',
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
});
```

In this example, the `getWeather` tool allows the model to fetch real-time weather data, enhancing its ability to provide accurate and up-to-date information.

### Agents

Agents take your AI applications a step further by allowing models to execute multiple steps (i.e. tools) in a non-deterministic way, making decisions based on context and user input.

Agents use LLMs to choose the next step in a problem-solving process. They can reason at each step and make decisions based on the evolving context.

### Implementing Agents with the AI SDK

The AI SDK supports agent implementation through the `maxSteps` parameter. This allows the model to make multiple decisions and tool calls in a single interaction.

Here's an example of an agent that solves math problems:

```tsx
import { generateText, tool } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';
import * as mathjs from 'mathjs';
import { z } from 'zod';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const problem =
  'Calculate the profit for a day if revenue is $5000 and expenses are $3500.';

const { text: answer } = await generateText({
  model: groq('llama-3.1-70b-versatile'),
  system:
    'You are solving math problems. Reason step by step. Use the calculator when necessary.',
  prompt: problem,
  tools: {
    calculate: tool({
      description: 'A tool for evaluating mathematical expressions.',
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => mathjs.evaluate(expression),
    }),
  },
  maxSteps: 5,
});
```

In this example, the agent can use the calculator tool multiple times if needed, reasoning through the problem step by step.

### Building Interactive Interfaces

AI SDK Core can be paired with [AI SDK UI](/docs/ai-sdk-ui/overview), another powerful component of the AI SDK, to streamline the process of building chat, completion, and assistant interfaces with popular frameworks like Next.js, Nuxt, SvelteKit, and SolidStart.

AI SDK UI provides robust abstractions that simplify the complex tasks of managing chat streams and UI updates on the frontend, enabling you to develop dynamic AI-driven interfaces more efficiently.

With four main hooks — [`useChat`](/docs/reference/ai-sdk-ui/use-chat), [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion), [`useObject`](/docs/reference/ai-sdk-ui/use-object), and [`useAssistant`](/docs/reference/ai-sdk-ui/use-assistant) — you can incorporate real-time chat capabilities, text completions, streamed JSON, and interactive assistant features into your app.

Let's explore building a chatbot with [Next.js](https://nextjs.org), the AI SDK, and Llama 3.1 (via [Groq](https://groq.com/)):

```tsx filename="app/api/chat/route.ts"
import { streamText } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: groq('llama-3.1-70b-versatile'),
    system: 'You are a helpful assistant.',
    messages,
  });

  return result.toDataStreamResponse();
}
```

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input name="prompt" value={input} onChange={handleInputChange} />
        <button type="submit">Submit</button>
      </form>
    </>
  );
}
```

The useChat hook on your root page (`app/page.tsx`) will make a request to your AI provider endpoint (`app/api/chat/route.ts`) whenever the user submits a message. The messages are then streamed back in real-time and displayed in the chat UI.

This enables a seamless chat experience where the user can see the AI response as soon as it is available, without having to wait for the entire response to be received.

### Going Beyond Text

The AI SDK's React Server Components (RSC) API enables you to create rich, interactive interfaces that go beyond simple text generation. With the [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function, you can dynamically stream React components from the server to the client.

Let's dive into how you can leverage tools with [AI SDK RSC](/docs/ai-sdk-rsc/overview) to build a generative user interface with Next.js (App Router).

First, create a Server Action.

```tsx filename="app/actions.tsx"
'use server';

import { streamUI } from 'ai/rsc';
import { createOpenAI as createGroq } from '@ai-sdk/openai';
import { z } from 'zod';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

export async function streamComponent() {
  const result = await streamUI({
    model: groq('llama-3.1-70b-versatile'),
    prompt: 'Get the weather for San Francisco',
    text: ({ content }) => <div>{content}</div>,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        parameters: z.object({ location: z.string() }),
        generate: async function* ({ location }) {
          yield <div>loading...</div>;
          const weather = '25c'; // await getWeather(location);
          return (
            <div>
              the weather in {location} is {weather}.
            </div>
          );
        },
      },
    },
  });
  return result.value;
}
```

In this example, if the model decides to use the `getWeather` tool, it will first yield a `div` while fetching the weather data, then return a weather component with the fetched data (note: static data in this example). This allows for a more dynamic and responsive UI that can adapt based on the AI's decisions and external data.

On the frontend, you can call this Server Action like any other asynchronous function in your application. In this case, the function returns a regular React component.

```tsx filename="app/page.tsx"
'use client';

import { useState } from 'react';
import { streamComponent } from './actions';

export default function Page() {
  const [component, setComponent] = useState<React.ReactNode>();

  return (
    <div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setComponent(await streamComponent());
        }}
      >
        <button>Stream Component</button>
      </form>
      <div>{component}</div>
    </div>
  );
}
```

To see AI SDK RSC in action, check out our open-source [Next.js Gemini Chatbot](https://gemini.vercel.ai/).

## Migrate from OpenAI

One of the key advantages of the AI SDK is its unified API, which makes it incredibly easy to switch between different AI models and providers. This flexibility is particularly useful when you want to migrate from one model to another, such as moving from OpenAI's GPT models to Meta's Llama models hosted on Groq.

Here's how simple the migration process can be:

**OpenAI Example:**

```tsx
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'What is love?',
});
```

**Llama on Groq Example:**

```tsx
import { generateText } from 'ai';
import { createOpenAI as createGroq } from '@ai-sdk/openai';

const groq = createGroq({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: process.env.GROQ_API_KEY,
});

const { text } = await generateText({
  model: groq('llama-3.1-70b-versatile'),
  prompt: 'What is love?',
});
```

Thanks to the unified API, the core structure of the code remains the same. The main differences are:

1. Creating a Groq client
2. Changing the model name from `openai("gpt-4-turbo")` to `groq("llama-3.1-70b-versatile")`.

With just these few changes, you've migrated from using OpenAI's GPT-4-Turbo to Meta's Llama 3.1 hosted on Groq. The `generateText` function and its usage remain identical, showcasing the power of the AI SDK's unified API.

This feature allows you to easily experiment with different models, compare their performance, and choose the best one for your specific use case without having to rewrite large portions of your codebase.

## Prompt Engineering and Fine-tuning

While the Llama 3.1 family of models are powerful out-of-the-box, their performance can be enhanced through effective prompt engineering and fine-tuning techniques.

### Prompt Engineering

Prompt engineering is the practice of crafting input prompts to elicit desired outputs from language models. It involves structuring and phrasing prompts in ways that guide the model towards producing more accurate, relevant, and coherent responses.

For more information on prompt engineering techniques (specific to Llama models), check out these resources:

- [Official Llama 3.1 Prompt Guide](https://llama.meta.com/docs/how-to-guides/prompting)
- [Prompt Engineering with Llama 3](https://github.com/amitsangani/Llama/blob/main/Llama_3_Prompt_Engineering.ipynb)
- [How to prompt Llama 3](https://huggingface.co/blog/llama3#how-to-prompt-llama-3)

### Fine-tuning

Fine-tuning involves further training a pre-trained model on a specific dataset or task to customize its performance for particular use cases. This process allows you to adapt Llama 3.1 to your specific domain or application, potentially improving its accuracy and relevance for your needs.

To learn more about fine-tuning Llama models, check out these resources:

- [Official Fine-tuning Llama Guide](https://llama.meta.com/docs/how-to-guides/fine-tuning)
- [Fine-tuning and Inference with Llama 3](https://docs.inferless.com/how-to-guides/how-to-finetune--and-inference-llama3)
- [Fine-tuning Models with Fireworks AI](https://docs.fireworks.ai/fine-tuning/fine-tuning-models)
- [Fine-tuning Llama with Modal](https://modal.com/docs/examples/llm-finetuning)

## Conclusion

The AI SDK offers a powerful and flexible way to integrate cutting-edge AI models like Llama 3.1 into your applications. With AI SDK Core, you can seamlessly switch between different AI models and providers by changing just two lines of code. This flexibility allows for quick experimentation and adaptation, reducing the time required to change models from days to minutes.

The AI SDK ensures that your application remains clean and modular, accelerating development and future-proofing against the rapidly evolving landscape.

Ready to get started? Here's how you can dive in:

1. Explore the documentation at [sdk.vercel.ai/docs](/docs) to understand the full capabilities of the AI SDK.
2. Check out practical examples at [sdk.vercel.ai/examples](/examples) to see the SDK in action and get inspired for your own projects.
3. Dive deeper with advanced guides on topics like Retrieval-Augmented Generation (RAG) and multi-modal chat at [sdk.vercel.ai/docs/guides](/docs/guides).
4. Check out ready-to-deploy AI templates at [vercel.com/templates?type=ai](https://vercel.com/templates?type=ai).

---
title: Get started with OpenAI o1
description: Get started with OpenAI o1 using the AI SDK.
---

# Get started with OpenAI o1

With the [release of OpenAI's o1 series models](https://openai.com/index/introducing-openai-o1-preview/), there has never been a better time to start building AI applications, particularly those that require complex reasoning capabilities.

The [AI SDK](/) is a powerful TypeScript toolkit for building AI applications with large language models (LLMs) like OpenAI o1 alongside popular frameworks like React, Next.js, Vue, Svelte, Node.js, and more.

<Note>
  OpenAI o1 models are currently [in beta with limited
  features](https://platform.openai.com/docs/guides/reasoning/beta-limitations).
  Access is restricted to developers in tier 4 and tier 5, with low rate limits
  (20 RPM). OpenAI is working on adding more features, increasing rate limits,
  and expanding access to more developers in the coming weeks.
</Note>

## OpenAI o1

OpenAI released a series of AI models designed to spend more time thinking before responding. They can reason through complex tasks and solve harder problems than previous models in science, coding, and math. These models, named the o1 series, are trained with reinforcement learning and can "think before they answer". As a result, they are able to produce a long internal chain of thought before responding to a prompt.

There are two reasoning models available in the API:

1. [**o1-preview**](https://platform.openai.com/docs/models/o1): An early preview of the o1 model, designed to reason about hard problems using broad general knowledge about the world.
2. [**o1-mini**](https://platform.openai.com/docs/models/ohttps://platform.openai.com/docs/models/o1): A faster and cheaper version of o1, particularly adept at coding, math, and science tasks where extensive general knowledge isn't required.

### Benchmarks

OpenAI o1 models excel in scientific reasoning, with impressive performance across various domains:

- Ranking in the 89th percentile on competitive programming questions (Codeforces)
- Placing among the top 500 students in the US in a qualifier for the USA Math Olympiad (AIME)
- Exceeding human PhD-level accuracy on a benchmark of physics, biology, and chemistry problems (GPQA)

[Source](https://openai.com/index/learning-to-reason-with-llms/)

### Prompt Engineering for o1 Models

The o1 models perform best with straightforward prompts. Some prompt engineering techniques, like few-shot prompting or instructing the model to "think step by step," may not enhance performance and can sometimes hinder it. Here are some best practices:

1. Keep prompts simple and direct: The models excel at understanding and responding to brief, clear instructions without the need for extensive guidance.
2. Avoid chain-of-thought prompts: Since these models perform reasoning internally, prompting them to "think step by step" or "explain your reasoning" is unnecessary.
3. Use delimiters for clarity: Use delimiters like triple quotation marks, XML tags, or section titles to clearly indicate distinct parts of the input, helping the model interpret different sections appropriately.
4. Limit additional context in retrieval-augmented generation (RAG): When providing additional context or documents, include only the most relevant information to prevent the model from overcomplicating its response.

## Getting Started with the AI SDK

The AI SDK is the TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more. Integrating LLMs into applications is complicated and heavily dependent on the specific model provider you use.

The AI SDK abstracts away the differences between model providers, eliminates boilerplate code for building chatbots, and allows you to go beyond text output to generate rich, interactive components.

At the center of the AI SDK is [AI SDK Core](/docs/ai-sdk-core/overview), which provides a unified API to call any LLM. The code snippet below is all you need to call OpenAI o1-mini with the AI SDK:

```tsx
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('o1-mini'),
  prompt: 'Explain the concept of quantum entanglement.',
});
```

<Note>
  To use the o1 series of models, you must either be using @ai-sdk/openai
  version 0.0.59 or greater, or set `temperature: 1`.
</Note>

AI SDK Core abstracts away the differences between model providers, allowing you to focus on building great applications. The unified interface also means that you can easily switch between models by changing just one line of code.

```tsx highlight="5"
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('o1-preview'),
  prompt: 'Explain the concept of quantum entanglement.',
});
```

<Note>
  During the beta phase, access to most chat completions parameters is not
  supported for o1 models. Features like function calling and image inputs are
  currently unavailable, and streaming is simulated.
</Note>

### Building Interactive Interfaces

AI SDK Core can be paired with [AI SDK UI](/docs/ai-sdk-ui/overview), another powerful component of the AI SDK, to streamline the process of building chat, completion, and assistant interfaces with popular frameworks like Next.js, Nuxt, SvelteKit, and SolidStart.

AI SDK UI provides robust abstractions that simplify the complex tasks of managing chat streams and UI updates on the frontend, enabling you to develop dynamic AI-driven interfaces more efficiently.

With four main hooks — [`useChat`](/docs/reference/ai-sdk-ui/use-chat), [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion), [`useObject`](/docs/reference/ai-sdk-ui/use-object), and [`useAssistant`](/docs/reference/ai-sdk-ui/use-assistant) — you can incorporate real-time chat capabilities, text completions, streamed JSON, and interactive assistant features into your app.

Let's explore building a chatbot with [Next.js](https://nextjs.org), the AI SDK, and OpenAI o1:

```tsx filename="app/api/chat/route.ts"
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Allow responses up to 5 minutes
export const maxDuration = 300;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const { text } = await generateText({
    model: openai('o1-preview'),
    messages,
  });

  return new Response(text);
}
```

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit, error } = useChat({
    streamProtocol: 'text',
  });

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input name="prompt" value={input} onChange={handleInputChange} />
        <button type="submit">Submit</button>
      </form>
    </>
  );
}
```

The useChat hook on your root page (`app/page.tsx`) will make a request to your AI provider endpoint (`app/api/chat/route.ts`) whenever the user submits a message. The messages are then displayed in the chat UI.

<Note>
  Due to the current limitations of o1 models during the beta phase, real-time
  streaming is not supported. The response will be sent once the model completes
  its reasoning and generates the full output.
</Note>

## Get Started

Ready to get started? Here's how you can dive in:

1. Explore the documentation at [sdk.vercel.ai/docs](/docs) to understand the full capabilities of the AI SDK.
1. Check out our support for the o1 series of reasoning models in the [OpenAI Provider](/providers/ai-sdk-providers/openai#reasoning-models).
1. Check out practical examples at [sdk.vercel.ai/examples](/examples) to see the SDK in action and get inspired for your own projects.
1. Dive deeper with advanced guides on topics like Retrieval-Augmented Generation (RAG) and multi-modal chat at [sdk.vercel.ai/docs/guides](/docs/guides).
1. Check out ready-to-deploy AI templates at [vercel.com/templates?type=ai](https://vercel.com/templates?type=ai).

Remember that OpenAI o1 models are currently in beta with limited features and access. Stay tuned for updates as OpenAI expands access and adds more features to these powerful reasoning models.

---
title: Get started with Computer Use
description: Get started with Claude's Computer Use capabilities with the AI SDK
---

# Get started with Computer Use

With the [release of Computer Use in Claude 3.5 Sonnet](https://www.anthropic.com/news/3-5-models-and-computer-use), you can now direct AI models to interact with computers like humans do - moving cursors, clicking buttons, and typing text. This capability enables automation of complex tasks while leveraging Claude's advanced reasoning abilities.

The AI SDK is a powerful TypeScript toolkit for building AI applications with large language models (LLMs) like Anthropic's Claude alongside popular frameworks like React, Next.js, Vue, Svelte, Node.js, and more. In this guide, you will learn how to integrate Computer Use into your AI SDK applications.

<Note>
  Computer Use is currently in beta with some [ limitations
  ](https://docs.anthropic.com/en/docs/build-with-claude/computer-use#understand-computer-use-limitations).
  The feature may be error-prone at times. Anthropic recommends starting with
  low-risk tasks and implementing appropriate safety measures.
</Note>

## Computer Use

Anthropic recently released a new version of the Claude 3.5 Sonnet model which is capable of 'Computer Use'. This allows the model to interact with computer interfaces through basic actions like:

- Moving the cursor
- Clicking buttons
- Typing text
- Taking screenshots
- Reading screen content

## How It Works

Computer Use enables the model to read and interact with on-screen content through a series of coordinated steps. Here's how the process works:

1. **Start with a prompt and tools**

   Add Anthropic-defined Computer Use tools to your request and provide a task (prompt) for the model. For example: "save an image to your downloads folder."

2. **Select the right tool**

   The model evaluates which computer tools can help accomplish the task. It then sends a formatted `tool_call` to use the appropriate tool.

3. **Execute the action and return results**

   The AI SDK processes Claude's request by running the selected tool. The results can then be sent back to Claude through a `tool_result` message.

4. **Complete the task through iterations**

   Claude analyzes each result to determine if more actions are needed. It continues requesting tool use and processing results until it completes your task or requires additional input.

### Available Tools

There are three main tools available in the Computer Use API:

1. **Computer Tool**: Enables basic computer control like mouse movement, clicking, and keyboard input
2. **Text Editor Tool**: Provides functionality for viewing and editing text files
3. **Bash Tool**: Allows execution of bash commands

### Implementation Considerations

Computer Use tools in the AI SDK are predefined interfaces that require your own implementation of the execution layer. While the SDK provides the type definitions and structure for these tools, you need to:

1. Set up a controlled environment for Computer Use execution
2. Implement core functionality like mouse control and keyboard input
3. Handle screenshot capture and processing
4. Set up rules and limits for how Claude can interact with your system

The recommended approach is to start with [ Anthropic's reference implementation ](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo), which provides:

- A containerized environment configured for safe Computer Use
- Ready-to-use (Python) implementations of Computer Use tools
- An agent loop for API interaction and tool execution
- A web interface for monitoring and control

This reference implementation serves as a foundation to understand the requirements before building your own custom solution.

## Getting Started with the AI SDK

<Note>
  If you have never used the AI SDK before, start by following the [Getting
  Started guide](/docs/getting-started).
</Note>

First, ensure you have the AI SDK and [Anthropic AI SDK provider](/providers/ai-sdk-providers/anthropic) installed:

<Snippet text="pnpm add ai @ai-sdk/anthropic" />

You can add Computer Use to your AI SDK applications using provider-defined tools. These tools accept various input parameters (like display height and width in the case of the computer tool) and then require that you define an execute function.

Here's how you could set up the Computer Tool with the AI SDK:

```ts
import { anthropic } from '@ai-sdk/anthropic';
import { getScreenshot, executeComputerAction } from '@/utils/computer-use';

const computerTool = anthropic.tools.computer_20241022({
  displayWidthPx: 1920,
  displayHeightPx: 1080,
  execute: async ({ action, coordinate, text }) => {
    switch (action) {
      case 'screenshot': {
        return {
          type: 'image',
          data: getScreenshot(),
        };
      }
      default: {
        return executeComputerAction(action, coordinate, text);
      }
    }
  },
  experimental_toToolResultContent(result) {
    return typeof result === 'string'
      ? [{ type: 'text', text: result }]
      : [{ type: 'image', data: result.data, mimeType: 'image/png' }];
  },
});
```

The `computerTool` handles two main actions: taking screenshots via `getScreenshot()` and executing computer actions like mouse movements and clicks through `executeComputerAction()`. Remember, you have to implement this execution logic (eg. the `getScreenshot` and `executeComputerAction` functions) to handle the actual computer interactions. The `execute` function should handle all low-level interactions with the operating system.

Finally, to send tool results back to the model, use the [`experimental_toToolResultContent()`](/docs/foundations/prompts#multi-modal-tool-results) function to convert text and image responses into a format the model can process. The AI SDK includes experimental support for these multi-modal tool results when using Anthropic's models.

<Note>
  Computer Use requires appropriate safety measures like using virtual machines,
  limiting access to sensitive data, and implementing human oversight for
  critical actions.
</Note>

### Using Computer Tools with Text Generation

Once your tool is defined, you can use it with both the [`generateText`](/docs/reference/ai-sdk-core/generate-text) and [`streamText`](/docs/reference/ai-sdk-core/stream-text) functions.

For one-shot text generation, use `generateText`:

```ts
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  prompt: 'Move the cursor to the center of the screen and take a screenshot',
  tools: { computer: computerTool },
});

console.log(response.text);
```

For streaming responses, use `streamText` to receive updates in real-time:

```ts
const result = streamText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  prompt: 'Open the browser and navigate to vercel.com',
  tools: { computer: computerTool },
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}
```

### Configure Multi-Step (Agentic) Generations

To allow the model to perform multiple steps without user intervention, specify a `maxSteps` value. This will automatically send any tool results back to the model to trigger a subsequent generation:

```ts highlight="5"
const stream = streamText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  prompt: 'Open the browser and navigate to vercel.com',
  tools: { computer: computerTool },
  maxSteps: 10, // experiment with this value based on your use case
});
```

### Combine Multiple Tools

You can combine multiple tools in a single request to enable more complex workflows. The AI SDK supports all three of Claude's Computer Use tools:

```ts
const computerTool = anthropic.tools.computer_20241022({
  ...
});

const bashTool = anthropic.tools.bash_20241022({
  execute: async ({ command, restart }) => execSync(command).toString()
});

const textEditorTool = anthropic.tools.textEditor_20241022({
  execute: async ({
    command,
    path,
    file_text,
    insert_line,
    new_str,
    old_str,
    view_range
  }) => {
    // Handle file operations based on command
    switch(command) {
      return executeTextEditorFunction({
        command,
        path,
        fileText: file_text,
        insertLine: insert_line,
        newStr: new_str,
        oldStr: old_str,
        viewRange: view_range
      });
    }
  }
});


const response = await generateText({
  model: anthropic("claude-3-5-sonnet-20241022"),
  prompt: "Create a new file called example.txt, write 'Hello World' to it, and run 'cat example.txt' in the terminal",
  tools: {
    computer: computerTool,
    textEditor: textEditorTool,
    bash: bashTool
  },
});
```

<Note>
  Always implement appropriate [security measures](#security-measures) and
  obtain user consent before enabling Computer Use in production applications.
</Note>

### Best Practices for Computer Use

To get the best results when using Computer Use:

1. Specify simple, well-defined tasks with explicit instructions for each step
2. Prompt Claude to verify outcomes through screenshots
3. Use keyboard shortcuts when UI elements are difficult to manipulate
4. Include example screenshots for repeatable tasks
5. Provide explicit tips in system prompts for known tasks

## Security Measures

Remember, Computer Use is a beta feature. Please be aware that it poses unique risks that are distinct from standard API features or chat interfaces. These risks are heightened when using Computer Use to interact with the internet. To minimize risks, consider taking precautions such as:

1. Use a dedicated virtual machine or container with minimal privileges to prevent direct system attacks or accidents.
2. Avoid giving the model access to sensitive data, such as account login information, to prevent information theft.
3. Limit internet access to an allowlist of domains to reduce exposure to malicious content.
4. Ask a human to confirm decisions that may result in meaningful real-world consequences as well as any tasks requiring affirmative consent, such as accepting cookies, executing financial transactions, or agreeing to terms of service.

---
title: Natural Language Postgres
description: Learn how to build a Next.js app that lets you talk to a PostgreSQL database in natural language.
---

# Natural Language Postgres Guide

In this guide, you will learn how to build an app that uses AI to interact with a PostgreSQL database using natural language.

The application will:

- Generate SQL queries from a natural language input
- Explain query components in plain English
- Create a chart to visualise query results

You can find a completed version of this project at [natural-language-postgres.vercel.app](https://natural-language-postgres.vercel.app).

## Project setup

This project uses the following stack:

- [Next.js](https://nextjs.org) (App Router)
- [AI SDK](https://sdk.vercel.ai/docs)
- [OpenAI](https://openai.com)
- [Zod](https://zod.dev)
- [Postgres](https://www.postgresql.org/) with [ Vercel Postgres ](https://vercel.com/postgres)
- [shadcn-ui](https://ui.shadcn.com) and [TailwindCSS](https://tailwindcss.com) for styling
- [Recharts](https://recharts.org) for data visualization

### Clone repo

To focus on the AI-powered functionality rather than project setup and configuration we've prepared a starter repository which includes a database schema and a few components.

Clone the starter repository and check out the `starter` branch:

<Snippet
  text={[
    'git clone https://github.com/vercel-labs/natural-language-postgres',
    'cd natural-language-postgres',
    'git checkout starter',
  ]}
/>

### Project setup and data

Let's set up the project and seed the database with the dataset:

1. Install dependencies:

<Snippet text={['pnpm install']} />

2. Copy the example environment variables file:

<Snippet text={['cp .env.example .env']} />

3. Add your environment variables to `.env`:

```bash filename=".env"
OPENAI_API_KEY="your_api_key_here"
POSTGRES_URL="..."
POSTGRES_PRISMA_URL="..."
POSTGRES_URL_NO_SSL="..."
POSTGRES_URL_NON_POOLING="..."
POSTGRES_USER="..."
POSTGRES_HOST="..."
POSTGRES_PASSWORD="..."
POSTGRES_DATABASE="..."
```

<Note>
  This project uses Vercel Postgres. You can learn more about how to set up at
  the [Vercel Postgres documentation](https://vercel.com/postgres).
</Note>

4. This project uses CB Insights' Unicorn Companies dataset. You can download the dataset by following these instructions:
   - Navigate to [CB Insights Unicorn Companies](https://www.cbinsights.com/research-unicorn-companies)
   - Enter in your email. You will receive a link to download the dataset.
   - Save it as `unicorns.csv` in your project root

### About the dataset

The Unicorn List dataset contains the following information about unicorn startups (companies with a valuation above $1bn):

- Company name
- Valuation
- Date joined (unicorn status)
- Country
- City
- Industry
- Select investors

This dataset contains over 1000 rows of data over 7 columns, giving us plenty of structured data to analyze. This makes it perfect for exploring various SQL queries that can reveal interesting insights about the unicorn startup ecosystem.

5. Now that you have the dataset downloaded and added to your project, you can initialize the database with the following command:

<Snippet text={['pnpm run seed']} />

Note: this step can take a little while. You should see a message indicating the Unicorns table has been created and then that the database has been seeded successfully.

<Note>
  Remember, the dataset should be named `unicorns.csv` and located in root of
  your project.
</Note>

6. Start the development server:

<Snippet text={['pnpm run dev']} />

Your application should now be running at [http://localhost:3000](http://localhost:3000).

## Project structure

The starter repository already includes everything that you will need, including:

- Database seed script (`lib/seed.ts`)
- Basic components built with shadcn/ui (`components/`)
- Function to run SQL queries (`app/actions.ts`)
- Type definitions for the database schema (`lib/types.ts`)

### Existing components

The application contains a single page in `app/page.tsx` that serves as the main interface.

At the top, you'll find a header (`header.tsx`) displaying the application title and description. Below that is an input field and search button (`search.tsx`) where you can enter natural language queries.

Initially, the page shows a collection of suggested example queries (`suggested-queries.tsx`) that you can click to quickly try out the functionality.

When you submit a query:

- The suggested queries section disappears and a loading state appears
- Once complete, a card appears with "TODO - IMPLEMENT ABOVE" (`query-viewer.tsx`) which will eventually show your generated SQL
- Below that is an empty results area with "No results found" (`results.tsx`)

After you implement the core functionality:

- The results section will display data in a table format
- A toggle button will allow switching between table and chart views
- The chart view will visualize your query results

Let's implement the AI-powered functionality to bring it all together.

## Building the application

As a reminder, this application will have three main features:

1. Generate SQL queries from natural language
2. Create a chart from the query results
3. Explain SQL queries in plain English

For each of these features, you'll use the AI SDK via [ Server Actions ](https://react.dev/reference/rsc/server-actions) to interact with OpenAI's GPT-4o and GPT-4o-mini models. Server Actions are a powerful React Server Component feature that allows you to call server-side functions directly from your frontend code.

Let's start with generating a SQL query from natural language.

## Generate SQL queries

### Providing context

For the model to generate accurate SQL queries, it needs context about your database schema, tables, and relationships. You will communicate this information through a prompt that should include:

1. Schema information
2. Example data formats
3. Available SQL operations
4. Best practices for query structure
5. Nuanced advice for specific fields

Let's write a prompt that includes all of this information:

```txt
You are a SQL (postgres) and data visualization expert. Your job is to help the user write a SQL query to retrieve the data they need. The table schema is as follows:

unicorns (
  id SERIAL PRIMARY KEY,
  company VARCHAR(255) NOT NULL UNIQUE,
  valuation DECIMAL(10, 2) NOT NULL,
  date_joined DATE,
  country VARCHAR(255) NOT NULL,
  city VARCHAR(255) NOT NULL,
  industry VARCHAR(255) NOT NULL,
  select_investors TEXT NOT NULL
);

Only retrieval queries are allowed.

For things like industry, company names and other string fields, use the ILIKE operator and convert both the search term and the field to lowercase using LOWER() function. For example: LOWER(industry) ILIKE LOWER('%search_term%').

Note: select_investors is a comma-separated list of investors. Trim whitespace to ensure you're grouping properly. Note, some fields may be null or have only one value.
When answering questions about a specific field, ensure you are selecting the identifying column (ie. what is Vercel's valuation would select company and valuation').

The industries available are:
- healthcare & life sciences
- consumer & retail
- financial services
- enterprise tech
- insurance
- media & entertainment
- industrials
- health

If the user asks for a category that is not in the list, infer based on the list above.

Note: valuation is in billions of dollars so 10b would be 10.0.
Note: if the user asks for a rate, return it as a decimal. For example, 0.1 would be 10%.

If the user asks for 'over time' data, return by year.

When searching for UK or USA, write out United Kingdom or United States respectively.

EVERY QUERY SHOULD RETURN QUANTITATIVE DATA THAT CAN BE PLOTTED ON A CHART! There should always be at least two columns. If the user asks for a single column, return the column and the count of the column. If the user asks for a rate, return the rate as a decimal. For example, 0.1 would be 10%.
```

There are several important elements of this prompt:

- Schema description helps the model understand exactly what data fields to work with
- Includes rules for handling queries based on common SQL patterns - for example, always using ILIKE for case-insensitive string matching
- Explains how to handle edge cases in the dataset, like dealing with the comma-separated investors field and ensuring whitespace is properly handled
- Instead of having the model guess at industry categories, it provides the exact list that exists in the data, helping avoid mismatches
- The prompt helps standardize data transformations - like knowing to interpret "10b" as "10.0" billion dollars, or that rates should be decimal values
- Clear rules ensure the query output will be chart-friendly by always including at least two columns of data that can be plotted

This prompt structure provides a strong foundation for query generation, but you should experiment and iterate based on your specific needs and the model you're using.

### Create a Server Action

With the prompt done, let's create a Server Action.

Open `app/actions.ts`. You should see one action already defined (`runGeneratedSQLQuery`).

Add a new action. This action should be asynchronous and take in one parameter - the natural language query.

```ts filename="app/actions.ts"
/* ...rest of the file... */

export const generateQuery = async (input: string) => {};
```

In this action, you'll use the `generateObject` function from the AI SDK which allows you to constrain the model's output to a pre-defined schema. This process, sometimes called structured output, ensures the model returns only the SQL query without any additional prefixes, explanations, or formatting that would require manual parsing.

```ts filename="app/actions.ts"
/* ...other imports... */
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

/* ...rest of the file... */

export const generateQuery = async (input: string) => {
  'use server';
  try {
    const result = await generateObject({
      model: openai('gpt-4o'),
      system: `You are a SQL (postgres) ...`, // SYSTEM PROMPT AS ABOVE - OMITTED FOR BREVITY
      prompt: `Generate the query necessary to retrieve the data the user wants: ${input}`,
      schema: z.object({
        query: z.string(),
      }),
    });
    return result.object.query;
  } catch (e) {
    console.error(e);
    throw new Error('Failed to generate query');
  }
};
```

Note, you are constraining the output to a single string field called `query` using `zod`, a TypeScript schema validation library. This will ensure the model only returns the SQL query itself. The resulting generated query will then be returned.

### Update the frontend

With the Server Action in place, you can now update the frontend to call this action when the user submits a natural language query. In the root page (`app/page.tsx`), you should see a `handleSubmit` function that is called when the user submits a query.

Import the `generateQuery` function and call it with the user's input.

```typescript filename="app/page.tsx" highlight="21"
/* ...other imports... */
import { runGeneratedSQLQuery, generateQuery } from './actions';

/* ...rest of the file... */

const handleSubmit = async (suggestion?: string) => {
  clearExistingData();

  const question = suggestion ?? inputValue;
  if (inputValue.length === 0 && !suggestion) return;

  if (question.trim()) {
    setSubmitted(true);
  }

  setLoading(true);
  setLoadingStep(1);
  setActiveQuery('');

  try {
    const query = await generateQuery(question);

    if (query === undefined) {
      toast.error('An error occurred. Please try again.');
      setLoading(false);
      return;
    }

    setActiveQuery(query);
    setLoadingStep(2);

    const companies = await runGeneratedSQLQuery(query);
    const columns = companies.length > 0 ? Object.keys(companies[0]) : [];
    setResults(companies);
    setColumns(columns);

    setLoading(false);
  } catch (e) {
    toast.error('An error occurred. Please try again.');
    setLoading(false);
  }
};

/* ...rest of the file... */
```

Now, when the user submits a natural language query (ie. "how many unicorns are from San Francisco?"), that question will be sent to your newly created Server Action. The Server Action will call the model, passing in your system prompt and the users query, and return the generated SQL query in a structured format. This query is then passed to the `runGeneratedSQLQuery` action to run the query against your database. The results are then saved in local state and displayed to the user.

Save the file, make sure the dev server is running, and then head to `localhost:3000` in your browser. Try submitting a natural language query and see the generated SQL query and results. You should see a SQL query generated and displayed under the input field. You should also see the results of the query displayed in a table below the input field.

Try clicking the SQL query to see the full query if it's too long to display in the input field. You should see a button on the right side of the input field with a question mark icon. Clicking this button currently does nothing, but you'll add the "explain query" functionality to it in the next step.

## Explain SQL Queries

Next, let's add the ability to explain SQL queries in plain English. This feature helps users understand how the generated SQL query works by breaking it down into logical sections.
As with the SQL query generation, you'll need a prompt to guide the model when explaining queries.

Let's craft a prompt for the explain query functionality:

```txt
You are a SQL (postgres) expert. Your job is to explain to the user write a SQL query you wrote to retrieve the data they asked for. The table schema is as follows:
unicorns (
  id SERIAL PRIMARY KEY,
  company VARCHAR(255) NOT NULL UNIQUE,
  valuation DECIMAL(10, 2) NOT NULL,
  date_joined DATE,
  country VARCHAR(255) NOT NULL,
  city VARCHAR(255) NOT NULL,
  industry VARCHAR(255) NOT NULL,
  select_investors TEXT NOT NULL
);

When you explain you must take a section of the query, and then explain it. Each "section" should be unique. So in a query like: "SELECT * FROM unicorns limit 20", the sections could be "SELECT *", "FROM UNICORNS", "LIMIT 20".
If a section doesnt have any explanation, include it, but leave the explanation empty.
```

Like the prompt for generating SQL queries, you provide the model with the schema of the database. Additionally, you provide an example of what each section of the query might look like. This helps the model understand the structure of the query and how to break it down into logical sections.

### Create a Server Action

Add a new Server Action to generate explanations for SQL queries.

This action takes two parameters - the original natural language input and the generated SQL query.

```ts filename="app/actions.ts"
/* ...rest of the file... */

export const explainQuery = async (input: string, sqlQuery: string) => {
  'use server';
  try {
    const result = await generateObject({
      model: openai('gpt-4o'),
      system: `You are a SQL (postgres) expert. ...`, // SYSTEM PROMPT AS ABOVE - OMITTED FOR BREVITY
      prompt: `Explain the SQL query you generated to retrieve the data the user wanted. Assume the user is not an expert in SQL. Break down the query into steps. Be concise.

      User Query:
      ${input}

      Generated SQL Query:
      ${sqlQuery}`,
    });
    return result.object;
  } catch (e) {
    console.error(e);
    throw new Error('Failed to generate query');
  }
};
```

This action uses the `generateObject` function again. However, you haven't defined the schema yet. Let's define it in another file so it can also be used as a type in your components.

Update your `lib/types.ts` file to include the schema for the explanations:

```ts filename="lib/types.ts"
import { z } from 'zod';

/* ...rest of the file... */

export const explanationSchema = z.object({
  section: z.string(),
  explanation: z.string(),
});

export type QueryExplanation = z.infer<typeof explanationSchema>;
```

This schema defines the structure of the explanation that the model will generate. Each explanation will have a `section` and an `explanation`. The `section` is the part of the query being explained, and the `explanation` is the plain English explanation of that section. Go back to your `actions.ts` file and import and use the `explanationSchema`:

```ts filename="app/actions.ts" highlight="2,19,20"
// other imports
import { explanationSchema } from '@/lib/types';

/* ...rest of the file... */

export const explainQuery = async (input: string, sqlQuery: string) => {
  'use server';
  try {
    const result = await generateObject({
      model: openai('gpt-4o'),
      system: `You are a SQL (postgres) expert. ...`, // SYSTEM PROMPT AS ABOVE - OMITTED FOR BREVITY
      prompt: `Explain the SQL query you generated to retrieve the data the user wanted. Assume the user is not an expert in SQL. Break down the query into steps. Be concise.

      User Query:
      ${input}

      Generated SQL Query:
      ${sqlQuery}`,
      schema: explanationSchema,
      output: 'array',
    });
    return result.object;
  } catch (e) {
    console.error(e);
    throw new Error('Failed to generate query');
  }
};
```

<Note>
  You can use `output: "array"` to indicate to the model that you expect an
  array of objects matching the schema to be returned.
</Note>

### Update query viewer

Next, update the `query-viewer.tsx` component to display these explanations. The `handleExplainQuery` function is called every time the user clicks the question icon button on the right side of the query. Let's update this function to use the new `explainQuery` action:

```ts filename="components/query-viewer.tsx" highlight="2,10,11"
/* ...other imports... */
import { explainQuery } from '@/app/actions';

/* ...rest of the component... */

const handleExplainQuery = async () => {
  setQueryExpanded(true);
  setLoadingExplanation(true);

  const explanations = await explainQuery(inputValue, activeQuery);
  setQueryExplanations(explanations);

  setLoadingExplanation(false);
};

/* ...rest of the component... */
```

Now when users click the explanation button (the question mark icon), the component will:

1. Show a loading state
2. Send the active SQL query and the users natural language query to your Server Action
3. The model will generate an array of explanations
4. The explanations will be set in the component state and rendered in the UI

Submit a new query and then click the explanation button. Hover over different elements of the query. You should see the explanations for each section!

## Visualizing query results

Finally, let's render the query results visually in a chart. There are two approaches you could take:

1. Send both the query and data to the model and ask it to return the data in a visualization-ready format. While this provides complete control over the visualization, it requires the model to send back all of the data, which significantly increases latency and costs.

2. Send the query and data to the model and ask it to generate a chart configuration (fixed-size and not many tokens) that maps your data appropriately. This configuration specifies how to visualize the information while delivering the insights from your natural language query. Importnatly, this is done without requiring the model return the full dataset.

Since you don't know the SQL query or data shape beforehand, let's use the second approach to dynamically generate chart configurations based on the query results and user intent.

### Generate the chart configuration

For this feature, you'll create a Server Action that takes the query results and the user's original natural language query to determine the best visualization approach. Your application is already set up to use `shadcn` charts (which uses [`Recharts`](https://recharts.org/en-US/) under the hood) so the model will need to generate:

- Chart type (bar, line, area, or pie)
- Axis mappings
- Visual styling

Let's start by defining the schema for the chart configuration in `lib/types.ts`:

```ts filename="lib/types.ts"
/* ...rest of the file... */

export const configSchema = z
  .object({
    description: z
      .string()
      .describe(
        'Describe the chart. What is it showing? What is interesting about the way the data is displayed?',
      ),
    takeaway: z.string().describe('What is the main takeaway from the chart?'),
    type: z.enum(['bar', 'line', 'area', 'pie']).describe('Type of chart'),
    title: z.string(),
    xKey: z.string().describe('Key for x-axis or category'),
    yKeys: z
      .array(z.string())
      .describe(
        'Key(s) for y-axis values this is typically the quantitative column',
      ),
    multipleLines: z
      .boolean()
      .describe(
        'For line charts only: whether the chart is comparing groups of data.',
      )
      .optional(),
    measurementColumn: z
      .string()
      .describe(
        'For line charts only: key for quantitative y-axis column to measure against (eg. values, counts etc.)',
      )
      .optional(),
    lineCategories: z
      .array(z.string())
      .describe(
        'For line charts only: Categories used to compare different lines or data series. Each category represents a distinct line in the chart.',
      )
      .optional(),
    colors: z
      .record(
        z.string().describe('Any of the yKeys'),
        z.string().describe('Color value in CSS format (e.g., hex, rgb, hsl)'),
      )
      .describe('Mapping of data keys to color values for chart elements')
      .optional(),
    legend: z.boolean().describe('Whether to show legend'),
  })
  .describe('Chart configuration object');

export type Config = z.infer<typeof configSchema>;
```

<Note>
  Replace the existing `export type Config = any;` type with the new one.
</Note>

This schema makes extensive use of Zod's `.describe()` function to give the model extra context about each of the key's you are expecting in the chart configuration. This will help the model understand the purpose of each key and generate more accurate results.

Another important technique to note here is that you are defining `description` and `takeaway` fields. Not only are these useful for the user to quickly understand what the chart means and what they should take away from it, but they also force the model to generate a description of the data first, before it attempts to generate configuration attributes like axis and columns. This will help the model generate more accurate and relevant chart configurations.

### Create the Server Action

Create a new action in `app/actions.ts`:

```ts
/* ...other imports... */
import { Config, configSchema, explanationsSchema, Result } from '@/lib/types';

/* ...rest of the file... */

export const generateChartConfig = async (
  results: Result[],
  userQuery: string,
) => {
  'use server';

  try {
    const { object: config } = await generateObject({
      model: openai('gpt-4o'),
      system: 'You are a data visualization expert.',
      prompt: `Given the following data from a SQL query result, generate the chart config that best visualises the data and answers the users query.
      For multiple groups use multi-lines.

      Here is an example complete config:
      export const chartConfig = {
        type: "pie",
        xKey: "month",
        yKeys: ["sales", "profit", "expenses"],
        colors: {
          sales: "#4CAF50",    // Green for sales
          profit: "#2196F3",   // Blue for profit
          expenses: "#F44336"  // Red for expenses
        },
        legend: true
      }

      User Query:
      ${userQuery}

      Data:
      ${JSON.stringify(results, null, 2)}`,
      schema: configSchema,
    });

    // Override with shadcn theme colors
    const colors: Record<string, string> = {};
    config.yKeys.forEach((key, index) => {
      colors[key] = `hsl(var(--chart-${index + 1}))`;
    });

    const updatedConfig = { ...config, colors };
    return { config: updatedConfig };
  } catch (e) {
    console.error(e);
    throw new Error('Failed to generate chart suggestion');
  }
};
```

### Update the chart component

With the action in place, you'll want to trigger it automatically after receiving query results. This ensures the visualization appears almost immediately after data loads.

Update the `handleSubmit` function in your root page (`app/page.tsx`) to generate and set the chart configuration after running the query:

```typescript filename="app/page.tsx" highlight="38,39"
/* ...other imports... */
import { getCompanies, generateQuery, generateChartConfig } from './actions';

/* ...rest of the file... */
const handleSubmit = async (suggestion?: string) => {
  clearExistingData();

  const question = suggestion ?? inputValue;
  if (inputValue.length === 0 && !suggestion) return;

  if (question.trim()) {
    setSubmitted(true);
  }

  setLoading(true);
  setLoadingStep(1);
  setActiveQuery('');

  try {
    const query = await generateQuery(question);

    if (query === undefined) {
      toast.error('An error occurred. Please try again.');
      setLoading(false);
      return;
    }

    setActiveQuery(query);
    setLoadingStep(2);

    const companies = await runGeneratedSQLQuery(query);
    const columns = companies.length > 0 ? Object.keys(companies[0]) : [];
    setResults(companies);
    setColumns(columns);

    setLoading(false);

    const { config } = await generateChartConfig(companies, question);
    setChartConfig(config);
  } catch (e) {
    toast.error('An error occurred. Please try again.');
    setLoading(false);
  }
};

/* ...rest of the file... */
```

Now when users submit queries, the application will:

1. Generate and run the SQL query
2. Display the table results
3. Generate a chart configuration for the results
4. Allow toggling between table and chart views

Head back to the browser and test the application with a few queries. You should see the chart visualization appear after the table results.

## Next steps

You've built an AI-powered SQL analysis tool that can convert natural language to SQL queries, visualize query results, and explain SQL queries in plain English.

You could, for example, extend the application to use your own data sources or add more advanced features like customizing the chart configuration schema to support more chart types and options. You could also add more complex SQL query generation capabilities.

---
title: Guides
description: Learn how to build AI applications with the AI SDK
---

# Guides

These use-case specific guides are intended to help you build real applications with the AI SDK.

<IndexCards
  cards={[
    {
      title: 'RAG Chatbot',
      description:
        'Learn how to build a retrieval-augmented generation chatbot with the AI SDK.',
      href: '/docs/guides/rag-chatbot',
    },
    {
      title: 'Multimodal Chatbot',
      description: 'Learn how to build a multimodal chatbot with the AI SDK.',
      href: '/docs/guides/multi-modal-chatbot',
    },
    {
      title: 'Get started with Llama 3.1',
      description: 'Get started with Llama 3.1 using the AI SDK.',
      href: '/docs/guides/llama-3_1',
    },
    {
      title: 'Get started with OpenAI o1',
      description: 'Get started with OpenAI o1 using the AI SDK.',
      href: '/docs/guides/o1',
    },
  ]}
/>

---
title: Overview
description: An overview of AI SDK Core.
---

# AI SDK Core

Large Language Models (LLMs) are advanced programs that can understand, create, and engage with human language on a large scale.
They are trained on vast amounts of written material to recognize patterns in language and predict what might come next in a given piece of text.

AI SDK Core **simplifies working with LLMs by offering a standardized way of integrating them into your app** - so you can focus on building great AI applications for your users, not waste time on technical details.

For example, here’s how you can generate text with various models using the AI SDK:

<PreviewSwitchProviders />

## AI SDK Core Functions

AI SDK Core has various functions designed for [text generation](./generating-text), [structured data generation](./generating-structured-data), and [tool usage](./tools-and-tool-calling).
These functions take a standardized approach to setting up [prompts](./prompts) and [settings](./settings), making it easier to work with different models.

- [`generateText`](/docs/ai-sdk-core/generating-text): Generates text and [tool calls](./tools-and-tool-calling).
  This function is ideal for non-interactive use cases such as automation tasks where you need to write text (e.g. drafting email or summarizing web pages) and for agents that use tools.
- [`streamText`](/docs/ai-sdk-core/generating-text): Stream text and tool calls.
  You can use the `streamText` function for interactive use cases such as [chat bots](/docs/ai-sdk-ui/chatbot) and [content streaming](/docs/ai-sdk-ui/completion).
- [`generateObject`](/docs/ai-sdk-core/generating-structured-data): Generates a typed, structured object that matches a [Zod](https://zod.dev/) schema.
  You can use this function to force the language model to return structured data, e.g. for information extraction, synthetic data generation, or classification tasks.
- [`streamObject`](/docs/ai-sdk-core/generating-structured-data): Stream a structured object that matches a Zod schema.
  You can use this function to [stream generated UIs](/docs/ai-sdk-ui/object-generation).

## API Reference

Please check out the [AI SDK Core API Reference](/docs/reference/ai-sdk-core) for more details on each function.

---
title: Generating Text
description: Learn how to generate text with the AI SDK.
---

# Generating and Streaming Text

Large language models (LLMs) can generate text in response to a prompt, which can contain instructions and information to process.
For example, you can ask a model to come up with a recipe, draft an email, or summarize a document.

The AI SDK Core provides two functions to generate text and stream it from LLMs:

- [`generateText`](#generatetext): Generates text for a given prompt and model.
- [`streamText`](#streamtext): Streams text from a given prompt and model.

Advanced LLM features such as [tool calling](./tools-and-tool-calling) and [structured data generation](./generating-structured-data) are built on top of text generation.

## `generateText`

You can generate text using the [`generateText`](/docs/reference/ai-sdk-core/generate-text) function. This function is ideal for non-interactive use cases where you need to write text (e.g. drafting email or summarizing web pages) and for agents that use tools.

```tsx
import { generateText } from 'ai';

const { text } = await generateText({
  model: yourModel,
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

You can use more [advanced prompts](./prompts) to generate text with more complex instructions and content:

```tsx
import { generateText } from 'ai';

const { text } = await generateText({
  model: yourModel,
  system:
    'You are a professional writer. ' +
    'You write simple, clear, and concise content.',
  prompt: `Summarize the following article in 3-5 sentences: ${article}`,
});
```

The result object of `generateText` contains several promises that resolve when all required data is available:

- `result.text`: The generated text.
- `result.finishReason`: The reason the model finished generating text.
- `result.usage`: The usage of the model during text generation.

## `streamText`

Depending on your model and prompt, it can take a large language model (LLM) up to a minute to finish generating it's response. This delay can be unacceptable for interactive use cases such as chatbots or real-time applications, where users expect immediate responses.

AI SDK Core provides the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function which simplifies streaming text from LLMs:

```ts
import { streamText } from 'ai';

const result = streamText({
  model: yourModel,
  prompt: 'Invent a new holiday and describe its traditions.',
});

// example: use textStream as an async iterable
for await (const textPart of result.textStream) {
  console.log(textPart);
}
```

<Note>
  `result.textStream` is both a `ReadableStream` and an `AsyncIterable`.
</Note>

You can use `streamText` on it's own or in combination with [AI SDK
UI](/examples/next-pages/basics/streaming-text-generation) and [AI SDK
RSC](/examples/next-app/basics/streaming-text-generation).
The result object contains several helper functions to make the integration into [AI SDK UI](/docs/ai-sdk-ui) easier:

- `result.toDataStreamResponse()`: Creates a data stream HTTP response (with tool calls etc.) that can be used in a Next.js App Router API route.
- `result.pipeDataStreamToResponse()`: Writes data stream delta output to a Node.js response-like object.
- `result.toTextStreamResponse()`: Creates a simple text stream HTTP response.
- `result.pipeTextStreamToResponse()`: Writes text delta output to a Node.js response-like object.

<Note>
  `streamText` is using backpressure and only generates tokens as they are
  requested. You need to consume the stream in order for it to finish.
</Note>

It also provides several promises that resolve when the stream is finished:

- `result.text`: The generated text.
- `result.finishReason`: The reason the model finished generating text.
- `result.usage`: The usage of the model during text generation.

### `onChunk` callback

When using `streamText`, you can provide an `onChunk` callback that is triggered for each chunk of the stream.

It receives the following chunk types:

- `text-delta`
- `tool-call`
- `tool-result`
- `tool-call-streaming-start` (when `experimental_streamToolCalls` is enabled)
- `tool-call-delta` (when `experimental_streamToolCalls` is enabled)

```tsx highlight="6-11"
import { streamText } from 'ai';

const result = streamText({
  model: yourModel,
  prompt: 'Invent a new holiday and describe its traditions.',
  onChunk({ chunk }) {
    // implement your own logic here, e.g.:
    if (chunk.type === 'text-delta') {
      console.log(chunk.text);
    }
  },
});
```

### `onFinish` callback

When using `streamText`, you can provide an `onFinish` callback that is triggered when the stream is finished.
It contains the text, usage information, finish reason, and more:

```tsx highlight="6-8"
import { streamText } from 'ai';

const result = streamText({
  model: yourModel,
  prompt: 'Invent a new holiday and describe its traditions.',
  onFinish({ text, finishReason, usage }) {
    // your own logic, e.g. for saving the chat history or recording usage
  },
});
```

### `fullStream` property

You can read a stream with all events using the `fullStream` property.
This can be useful if you want to implement your own UI or handle the stream in a different way.
Here is an example of how to use the `fullStream` property:

```tsx
import { streamText } from 'ai';
import { z } from 'zod';

const result = streamText({
  model: yourModel,
  tools: {
    cityAttractions: {
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => ({
        attractions: ['attraction1', 'attraction2', 'attraction3'],
      }),
    },
  },
  prompt: 'What are some San Francisco tourist attractions?',
});

for await (const part of result.fullStream) {
  switch (part.type) {
    case 'text-delta': {
      // handle text delta here
      break;
    }
    case 'tool-call': {
      switch (part.toolName) {
        case 'cityAttractions': {
          // handle tool call here
          break;
        }
      }
      break;
    }
    case 'tool-result': {
      switch (part.toolName) {
        case 'cityAttractions': {
          // handle tool result here
          break;
        }
      }
      break;
    }
    case 'finish': {
      // handle finish here
      break;
    }
    case 'error': {
      // handle error here
      break;
    }
  }
}
```

## Generating Long Text

Most language models have an output limit that is much shorter than their context window.
This means that you cannot generate long text in one go,
but it is possible to add responses back to the input and continue generating
to create longer text.

`generateText` and `streamText` support such continuations for long text generation using the experimental `continueSteps` setting:

```tsx highlight="5-6,9-10"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const {
  text, // combined text
  usage, // combined usage of all steps
} = await generateText({
  model: openai('gpt-4o'), // 4096 output tokens
  maxSteps: 5, // enable multi-step calls
  experimental_continueSteps: true,
  prompt:
    'Write a book about Roman history, ' +
    'from the founding of the city of Rome ' +
    'to the fall of the Western Roman Empire. ' +
    'Each chapter MUST HAVE at least 1000 words.',
});
```

<Note>
  When `experimental_continueSteps` is enabled, only full words are streamed in
  `streamText`, and both `generateText` and `streamText` might drop the trailing
  tokens of some calls to prevent whitespace issues.
</Note>

<Note type="warning">
  Some models might not always stop correctly on their own and keep generating
  until `maxSteps` is reached. You can hint the model to stop by e.g. using a
  system message such as "Stop when sufficient information was provided."
</Note>

## Examples

You can see `generateText` and `streamText` in action using various frameworks in the following examples:

### `generateText`

<ExampleLinks
  examples={[
    {
      title: 'Learn to generate text in Node.js',
      link: '/examples/node/generating-text/generate-text',
    },
    {
      title:
        'Learn to generate text in Next.js with Route Handlers (AI SDK UI)',
      link: '/examples/next-pages/basics/generating-text',
    },
    {
      title:
        'Learn to generate text in Next.js with Server Actions (AI SDK RSC)',
      link: '/examples/next-app/basics/generating-text',
    },
  ]}
/>

### `streamText`

<ExampleLinks
  examples={[
    {
      title: 'Learn to stream text in Node.js',
      link: '/examples/node/generating-text/stream-text',
    },
    {
      title: 'Learn to stream text in Next.js with Route Handlers (AI SDK UI)',
      link: '/examples/next-pages/basics/streaming-text-generation',
    },
    {
      title: 'Learn to stream text in Next.js with Server Actions (AI SDK RSC)',
      link: '/examples/next-app/basics/streaming-text-generation',
    },
  ]}
/>

---
title: Generating Structured Data
description: Learn how to generate structured data with the AI SDK.
---

# Generating Structured Data

While text generation can be useful, your use case will likely call for generating structured data.
For example, you might want to extract information from text, classify data, or generate synthetic data.

Many language models are capable of generating structured data, often defined as using "JSON modes" or "tools".
However, you need to manually provide schemas and then validate the generated data as LLMs can produce incorrect or incomplete structured data.

The AI SDK standardises structured object generation across model providers
with the [`generateObject`](/docs/reference/ai-sdk-core/generate-object)
and [`streamObject`](/docs/reference/ai-sdk-core/stream-object) functions.
You can use both functions with different output strategies, e.g. `array`, `object`, or `no-schema`,
and with different generation modes, e.g. `auto`, `tool`, or `json`.
You can use [Zod schemas](./schemas-and-zod) or [JSON schemas](/docs/reference/ai-sdk-core/json-schema) to specify the shape of the data that you want,
and the AI model will generate data that conforms to that structure.

## Generate Object

The `generateObject` generates structured data from a prompt.
The schema is also used to validate the generated data, ensuring type safety and correctness.

```ts
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: yourModel,
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});
```

## Stream Object

Given the added complexity of returning structured data, model response time can be unacceptable for your interactive use case.
With the [`streamObject`](/docs/reference/ai-sdk-core/stream-object) function, you can stream the model's response as it is generated.

```ts
import { streamObject } from 'ai';

const { partialObjectStream } = streamObject({
  // ...
});

// use partialObjectStream as an async iterable
for await (const partialObject of partialObjectStream) {
  console.log(partialObject);
}
```

You can use `streamObject` to stream generated UIs in combination with React Server Components (see [Generative UI](../ai-sdk-rsc))) or the [`useObject`](/docs/reference/ai-sdk-ui/use-object) hook.

## Output Strategy

You can use both functions with different output strategies, e.g. `array`, `object`, or `no-schema`.

### Object

The default output strategy is `object`, which returns the generated data as an object.
You don't need to specify the output strategy if you want to use the default.

### Array

If you want to generate an array of objects, you can set the output strategy to `array`.
When you use the `array` output strategy, the schema specifies the shape of an array element.
With `streamObject`, you can also stream the generated array elements using `elementStream`.

```ts highlight="7,18"
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const { elementStream } = streamObject({
  model: openai('gpt-4-turbo'),
  output: 'array',
  schema: z.object({
    name: z.string(),
    class: z
      .string()
      .describe('Character class, e.g. warrior, mage, or thief.'),
    description: z.string(),
  }),
  prompt: 'Generate 3 hero descriptions for a fantasy role playing game.',
});

for await (const hero of elementStream) {
  console.log(hero);
}
```

### Enum

If you want to generate a specific enum value, e.g. for classification tasks,
you can set the output strategy to `enum`
and provide a list of possible values in the `enum` parameter.

<Note>Enum output is only available with `generateObject`.</Note>

```ts highlight="5-6"
import { generateObject } from 'ai';

const { object } = await generateObject({
  model: yourModel,
  output: 'enum',
  enum: ['action', 'comedy', 'drama', 'horror', 'sci-fi'],
  prompt:
    'Classify the genre of this movie plot: ' +
    '"A group of astronauts travel through a wormhole in search of a ' +
    'new habitable planet for humanity."',
});
```

### No Schema

In some cases, you might not want to use a schema,
for example when the data is a dynamic user request.
You can use the `output` setting to set the output format to `no-schema` in those cases
and omit the schema parameter.

```ts highlight="6"
import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  output: 'no-schema',
  prompt: 'Generate a lasagna recipe.',
});
```

## Generation Mode

While some models (like OpenAI) natively support object generation, others require alternative methods, like modified [tool calling](/docs/ai-sdk-core/tools-and-tool-calling). The `generateObject` function allows you to specify the method it will use to return structured data.

- `auto`: The provider will choose the best mode for the model. This recommended mode is used by default.
- `tool`: A tool with the JSON schema as parameters is provided and the provider is instructed to use it.
- `json`: The response format is set to JSON when supported by the provider, e.g. via json modes or grammar-guided generation. If grammar-guided generation is not supported, the JSON schema and instructions to generate JSON that conforms to the schema are injected into the system prompt.

<Note>
  Please note that not every provider supports all generation modes. Some
  providers do not support object generation at all.
</Note>

## Schema Name and Description

You can optionally specify a name and description for the schema. These are used by some providers for additional LLM guidance, e.g. via tool or schema name.

```ts highlight="6-7"
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: yourModel,
  schemaName: 'Recipe',
  schemaDescription: 'A recipe for a dish.',
  schema: z.object({
    name: z.string(),
    ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
    steps: z.array(z.string()),
  }),
  prompt: 'Generate a lasagna recipe.',
});
```

## Error Handling

When you use `generateObject`, errors are thrown when the model fails to generate proper JSON (`JSONParseError`)
or when the generated JSON does not match the schema (`TypeValidationError`).
Both error types contain additional information, e.g. the generated text or the invalid value.

You can use this to e.g. design a function that safely process the result object and also returns values in error cases:

```ts
import { openai } from '@ai-sdk/openai';
import { JSONParseError, TypeValidationError, generateObject } from 'ai';
import { z } from 'zod';

const recipeSchema = z.object({
  recipe: z.object({
    name: z.string(),
    ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
    steps: z.array(z.string()),
  }),
});

type Recipe = z.infer<typeof recipeSchema>;

async function generateRecipe(
  food: string,
): Promise<
  | { type: 'success'; recipe: Recipe }
  | { type: 'parse-error'; text: string }
  | { type: 'validation-error'; value: unknown }
  | { type: 'unknown-error'; error: unknown }
> {
  try {
    const result = await generateObject({
      model: openai('gpt-4-turbo'),
      schema: recipeSchema,
      prompt: `Generate a ${food} recipe.`,
    });

    return { type: 'success', recipe: result.object };
  } catch (error) {
    if (TypeValidationError.isTypeValidationError(error)) {
      return { type: 'validation-error', value: error.value };
    } else if (JSONParseError.isJSONParseError(error)) {
      return { type: 'parse-error', text: error.text };
    } else {
      return { type: 'unknown-error', error };
    }
  }
}
```

## More Examples

You can see `generateObject` and `streamObject` in action using various frameworks in the following examples:

### `generateObject`

<ExampleLinks
  examples={[
    {
      title: 'Learn to generate objects in Node.js',
      link: '/examples/node/generating-structured-data/generate-object',
    },
    {
      title:
        'Learn to generate objects in Next.js with Route Handlers (AI SDK UI)',
      link: '/examples/next-pages/basics/generating-object',
    },
    {
      title:
        'Learn to generate objects in Next.js with Server Actions (AI SDK RSC)',
      link: '/examples/next-app/basics/generating-object',
    },
  ]}
/>

### `streamObject`

<ExampleLinks
  examples={[
    {
      title: 'Learn to stream objects in Node.js',
      link: '/examples/node/streaming-structured-data/stream-object',
    },
    {
      title:
        'Learn to stream objects in Next.js with Route Handlers (AI SDK UI)',
      link: '/examples/next-pages/basics/streaming-object-generation',
    },
    {
      title:
        'Learn to stream objects in Next.js with Server Actions (AI SDK RSC)',
      link: '/examples/next-app/basics/streaming-object-generation',
    },
  ]}
/>

---
title: Tool Calling
description: Learn about tool calling with AI SDK Core.
---

# Tool Calling

As covered under Foundations, [tools](/docs/foundations/tools) are objects that can be called by the model to perform a specific task.
AI SDK Core tools contain three elements:

- **`description`**: An optional description of the tool that can influence when the tool is picked.
- **`parameters`**: A [Zod schema](/docs/foundations/tools#schemas) or a [JSON schema](/docs/reference/ai-sdk-core/json-schema) that defines the parameters. The schema is consumed by the LLM, and also used to validate the LLM tool calls.
- **`execute`**: An optional async function that is called with the arguments from the tool call. It produces a value of type `RESULT` (generic type). It is optional because you might want to forward tool calls to the client or to a queue instead of executing them in the same process.

<Note className="mb-2">
  You can use the [`tool`](/docs/reference/ai-sdk-core/tool) helper function to
  infer the types of the `execute` parameters.
</Note>

The `tools` parameter of `generateText` and `streamText` is an object that has the tool names as keys and the tools as values:

```ts highlight="6-17"
import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  prompt: 'What is the weather in San Francisco?',
});
```

<Note>
  When a model uses a tool, it is called a "tool call" and the output of the
  tool is called a "tool result".
</Note>

Tool calling is not restricted to only text generation.
You can also use it to render user interfaces (Generative UI).

## Multi-Step Calls

Large language models need to know the tool results before they can continue to generate text.
This requires sending the tool results back to the model.
You can enable this feature by setting the `maxSteps` setting to a number greater than 1.

When `maxSteps` is set to a number greater than 1, the language model will be called
in a loop when there are tool calls and for every tool call there is a tool result, until there
are no further tool calls or the maximum number of tool steps is reached.

### Example

In the following example, there are two steps:

1. **Step 1**
   1. The prompt `'What is the weather in San Francisco?'` is sent to the model.
   1. The model generates a tool call.
   1. The tool call is executed.
1. **Step 2**
   1. The tool result is sent to the model.
   1. The model generates a response considering the tool result.

```ts highlight="18"
import { z } from 'zod';
import { generateText, tool } from 'ai';

const { text, steps } = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  maxSteps: 5, // allow up to 5 steps
  prompt: 'What is the weather in San Francisco?',
});
```

<Note>You can use `streamText` in a similar way.</Note>

### Steps

To access intermediate tool calls and results, you can use the `steps` property in the result object
or the `streamText` `onFinish` callback.
It contains all the text, tool calls, tool results, and more from each step.

#### Example: Extract tool results from all steps

```ts highlight="3,9-10"
import { generateText } from 'ai';

const { steps } = await generateText({
  model: openai('gpt-4-turbo'),
  maxSteps: 10,
  // ...
});

// extract all tool calls from the steps:
const allToolCalls = steps.flatMap(step => step.toolCalls);
```

### `onStepFinish` callback

When using `generateText` or `streamText`, you can provide an `onStepFinish` callback that
is triggered when a step is finished,
i.e. all text deltas, tool calls, and tool results for the step are available.
When you have multiple steps, the callback is triggered for each step.

```tsx highlight="5-7"
import { generateText } from 'ai';

const result = await generateText({
  // ...
  onStepFinish({ text, toolCalls, toolResults, finishReason, usage }) {
    // your own logic, e.g. for saving the chat history or recording usage
  },
});
```

## Response Messages

Adding the generated assistant and tool messages to your conversation history is a common task,
especially if you are using multi-step tool calls.

Both `generateText` and `streamText` have a `responseMessages` property that you can use to
add the assistant and tool messages to your conversation history.
It is also available in the `onFinish` callback of `streamText`.

The `responseMessages` property contains an array of `CoreMessage` objects that you can add to your conversation history:

```ts
import { generateText } from 'ai';

const messages: CoreMessage[] = [
  // ...
];

const { responseMessages } = await generateText({
  // ...
  messages,
});

// add the response messages to your conversation history:
messages.push(...responseMessages); // streamText: ...(await responseMessages)
```

## Tool Choice

You can use the `toolChoice` setting to influence when a tool is selected.
It supports the following settings:

- `auto` (default): the model can choose whether and which tools to call.
- `required`: the model must call a tool. It can choose which tool to call.
- `none`: the model must not call tools
- `{ type: 'tool', toolName: string (typed) }`: the model must call the specified tool

```ts highlight="18"
import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  toolChoice: 'required', // force the model to call a tool
  prompt: 'What is the weather in San Francisco?',
});
```

## Abort Signals

The abort signals from `generateText` and `streamText` are forwarded to the tool execution.
You can access them in the second parameter of the `execute` function and e.g. abort long-running computations or forward them to fetch calls inside tools.

```ts highlight="6,11,14"
import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  abortSignal: myAbortSignal, // signal that will be forwarded to tools
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({ location: z.string() }),
      execute: async ({ location }, { abortSignal }) => {
        return fetch(
          `https://api.weatherapi.com/v1/current.json?q=${location}`,
          { signal: abortSignal }, // forward the abort signal to fetch
        );
      },
    }),
  },
  prompt: 'What is the weather in San Francisco?',
});
```

## Types

Modularizing your code often requires defining types to ensure type safety and reusability.
To enable this, the AI SDK provides several helper types for tools, tool calls, and tool results.

You can use them to strongly type your variables, function parameters, and return types
in parts of the code that are not directly related to `streamText` or `generateText`.

Each tool call is typed with `CoreToolCall<NAME extends string, ARGS>`, depending
on the tool that has been invoked.
Similarly, the tool results are typed with `CoreToolResult<NAME extends string, ARGS, RESULT>`.

The tools in `streamText` and `generateText` are defined as a `Record<string, CoreTool>`.
The type inference helpers `CoreToolCallUnion<TOOLS extends Record<string, CoreTool>>`
and `CoreToolResultUnion<TOOLS extends Record<string, CoreTool>>` can be used to
extract the tool call and tool result types from the tools.

```ts highlight="18-19,23-24"
import { openai } from '@ai-sdk/openai';
import { CoreToolCallUnion, CoreToolResultUnion, generateText, tool } from 'ai';
import { z } from 'zod';

const myToolSet = {
  firstTool: tool({
    description: 'Greets the user',
    parameters: z.object({ name: z.string() }),
    execute: async ({ name }) => `Hello, ${name}!`,
  }),
  secondTool: tool({
    description: 'Tells the user their age',
    parameters: z.object({ age: z.number() }),
    execute: async ({ age }) => `You are ${age} years old!`,
  }),
};

type MyToolCall = CoreToolCallUnion<typeof myToolSet>;
type MyToolResult = CoreToolResultUnion<typeof myToolSet>;

async function generateSomething(prompt: string): Promise<{
  text: string;
  toolCalls: Array<MyToolCall>; // typed tool calls
  toolResults: Array<MyToolResult>; // typed tool results
}> {
  return generateText({
    model: openai('gpt-4o'),
    tools: myToolSet,
    prompt,
  });
}
```

## Active Tools

<Note type="warning">
  The `activeTools` property is experimental and may change in the future.
</Note>

Language models can only handle a limited number of tools at a time, depending on the model.
To allow for static typing using a large number of tools and limiting the available tools to the model at the same time,
the AI SDK provides the `experimental_activeTools` property.

It is an array of tool names that are currently active.
By default, the value is `undefined` and all tools are active.

```ts highlight="7"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const { text } = await generateText({
  model: openai('gpt-4o'),
  tools: myToolSet,
  experimental_activeTools: ['firstTool'],
});
```

## Multi-modal Tool Results

<Note type="warning">
  Multi-modal tool results are experimental and only supported by Anthropic.
</Note>

In order to send multi-modal tool results, e.g. screenshots, back to the model,
they need to be converted into a specific format.

AI SDK Core tools have an optional `experimental_toToolResultContent` function
that converts the tool result into a content part.

Here is an example for converting a screenshot into a content part:

```ts highlight="22-27"
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  tools: {
    computer: anthropic.tools.computer_20241022({
      // ...
      async execute({ action, coordinate, text }) {
        switch (action) {
          case 'screenshot': {
            return {
              type: 'image',
              data: fs
                .readFileSync('./data/screenshot-editor.png')
                .toString('base64'),
            };
          }
          default: {
            return `executed ${action}`;
          }
        }
      },

      // map to tool result content for LLM consumption:
      experimental_toToolResultContent(result) {
        return typeof result === 'string'
          ? [{ type: 'text', text: result }]
          : [{ type: 'image', data: result.data, mimeType: 'image/png' }];
      },
    }),
  },
  // ...
});
```

## Examples

You can see tools in action using various frameworks in the following examples:

<ExampleLinks
  examples={[
    {
      title: 'Learn to use tools in Node.js',
      link: '/examples/node/tools/call-tool',
    },
    {
      title: 'Learn to use tools in Next.js with Route Handlers (AI SDK UI)',
      link: '/examples/next-pages/tools/call-tool',
    },
    {
      title: 'Learn to use tools in Next.js with Server Actions (AI SDK RSC)',
      link: '/examples/next-app/tools/call-tool',
    },
  ]}
/>

---
title: Agents
description: Learn about creating agents with AI SDK Core.
---

# Agents

AI agents let the language model execute a series of steps in a non-deterministic way.
The model can make tool calling decisions based on the context of the conversation, the user's input,
and previous tool calls and results.

One approach to implementing agents is to allow the LLM to choose the next step in a loop.
With `generateText`, you can combine [tools](/docs/ai-sdk-core/tools-and-tool-calling) with `maxSteps`.
This makes it possible to implement agents that reason at each step and make decisions based on the context.

### Example

This example demonstrates how to create an agent that solves math problems.
It has a calculator tool (using [math.js](https://mathjs.org/)) that it can call to evaluate mathematical expressions.

```ts file='main.ts'
import { openai } from '@ai-sdk/openai';
import { generateText, tool } from 'ai';
import * as mathjs from 'mathjs';
import { z } from 'zod';

const { text: answer } = await generateText({
  model: openai('gpt-4o-2024-08-06', { structuredOutputs: true }),
  tools: {
    calculate: tool({
      description:
        'A tool for evaluating mathematical expressions. ' +
        'Example expressions: ' +
        "'1.2 * (2 + 4.5)', '12.7 cm to inch', 'sin(45 deg) ^ 2'.",
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => mathjs.evaluate(expression),
    }),
  },
  maxSteps: 10,
  system:
    'You are solving math problems. ' +
    'Reason step by step. ' +
    'Use the calculator when necessary. ' +
    'When you give the final answer, ' +
    'provide an explanation for how you arrived at it.',
  prompt:
    'A taxi driver earns $9461 per 1-hour of work. ' +
    'If he works 12 hours a day and in 1 hour ' +
    'he uses 12 liters of petrol with a price  of $134 for 1 liter. ' +
    'How much money does he earn in one day?',
});

console.log(`ANSWER: ${answer}`);
```

## Structured Answers

You can use an **answer tool** and the `toolChoice: 'required'` setting to force
the LLM to answer with a structured output that matches the schema of the answer tool.
The answer tool has no `execute` function, so invoking it will terminate the agent.

### Example

```ts highlight="6,16-29,31,45"
import { openai } from '@ai-sdk/openai';
import { generateText, tool } from 'ai';
import 'dotenv/config';
import { z } from 'zod';

const { toolCalls } = await generateText({
  model: openai('gpt-4o-2024-08-06', { structuredOutputs: true }),
  tools: {
    calculate: tool({
      description:
        'A tool for evaluating mathematical expressions. Example expressions: ' +
        "'1.2 * (2 + 4.5)', '12.7 cm to inch', 'sin(45 deg) ^ 2'.",
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => mathjs.evaluate(expression),
    }),
    // answer tool: the LLM will provide a structured answer
    answer: tool({
      description: 'A tool for providing the final answer.',
      parameters: z.object({
        steps: z.array(
          z.object({
            calculation: z.string(),
            reasoning: z.string(),
          }),
        ),
        answer: z.string(),
      }),
      // no execute function - invoking it will terminate the agent
    }),
  },
  toolChoice: 'required',
  maxSteps: 10,
  system:
    'You are solving math problems. ' +
    'Reason step by step. ' +
    'Use the calculator when necessary. ' +
    'The calculator can only do simple additions, subtractions, multiplications, and divisions. ' +
    'When you give the final answer, provide an explanation for how you got it.',
  prompt:
    'A taxi driver earns $9461 per 1-hour work. ' +
    'If he works 12 hours a day and in 1 hour he uses 14-liters petrol with price $134 for 1-liter. ' +
    'How much money does he earn in one day?',
});

console.log(`FINAL TOOL CALLS: ${JSON.stringify(toolCalls, null, 2)}`);
```

## Accessing all steps

Calling `generateText` with `maxSteps` can result in several calls to the LLM (steps).
You can access information from all steps by using the `steps` property of the response.

```ts highlight="3,9-10"
import { generateText } from 'ai';

const { steps } = await generateText({
  model: openai('gpt-4-turbo'),
  maxSteps: 10,
  // ...
});

// extract all tool calls from the steps:
const allToolCalls = steps.flatMap(step => step.toolCalls);
```

## Getting notified on each completed step

You can use the `onStepFinish` callback to get notified on each completed step.
It is triggered when a step is finished,
i.e. all text deltas, tool calls, and tool results for the step are available.

```tsx highlight="6-8"
import { generateText } from 'ai';

const result = await generateText({
  model: yourModel,
  maxSteps: 10,
  onStepFinish({ text, toolCalls, toolResults, finishReason, usage }) {
    // your own logic, e.g. for saving the chat history or recording usage
  },
  // ...
});
```

---
title: Prompt Engineering
description: Learn how to develop prompts with AI SDK Core.
---

# Prompt Engineering

## Tips

### Prompts for Tools

When you create prompts that include tools, getting good results can be tricky as the number and complexity of your tools increases.

Here are a few tips to help you get the best results:

1. Use a model that is strong at tool calling, such as `gpt-4` or `gpt-4-turbo`. Weaker models will often struggle to call tools effectively and flawlessly.
1. Keep the number of tools low, e.g. to 5 or less.
1. Keep the complexity of the tool parameters low. Complex Zod schemas with many nested and optional elements, unions, etc. can be challenging for the model to work with.
1. Use semantically meaningful names for your tools, parameters, parameter properties, etc. The more information you pass to the model, the better it can understand what you want.
1. Add `.describe("...")` to your Zod schema properties to give the model hints about what a particular property is for.
1. When the output of a tool might be unclear to the model and there are dependencies between tools, use the `description` field of a tool to provide information about the output of the tool execution.
1. You can include example input/outputs of tool calls in your prompt to help the model understand how to use the tools. Keep in mind that the tools work with JSON objects, so the examples should use JSON.

In general, the goal should be to give the model all information it needs in a clear way.

### Tool & Structured Data Schemas

The mapping from Zod schemas to LLM inputs (typically JSON schema) is not always straightforward, since the mapping is not one-to-one.

#### Zod Dates

Zod expects JavaScript Date objects, but models return dates as strings.
You can specify and validate the date format using `z.string().datetime()` or `z.string().date()`,
and then use a Zod transformer to convert the string to a Date object.

```ts highlight="7-10"
const result = await generateObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        date: z
          .string()
          .date()
          .transform(value => new Date(value)),
      }),
    ),
  }),
  prompt: 'List 5 important events from the the year 2000.',
});
```

## Debugging

### Inspecting Warnings

Not all providers support all AI SDK features.
Providers either throw exceptions or return warnings when they do not support a feature.
To check if your prompt, tools, and settings are handled correctly by the provider, you can check the call warnings:

```ts
const result = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Hello, world!',
});

console.log(result.warnings);
```

### HTTP Request Bodies

You can inspect the raw HTTP request bodies for models that expose them, e.g. [OpenAI](/providers/ai-sdk-providers/openai).
This allows you to inspect the exact payload that is sent to the model provider in the provider-specific way.

Request bodies are available via the `request.body` property of the response:

```ts highlight="6"
const result = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Hello, world!',
});

console.log(result.request.body);
```

---
title: Settings
description: Learn how to configure the AI SDK.
---

# Settings

Large language models (LLMs) typically provide settings to augment their output.

All AI SDK functions support the following common settings in addition to the model, the [prompt](./prompts), and additional provider-specific settings:

```ts highlight="3-5"
const result = await generateText({
  model: yourModel,
  maxTokens: 512,
  temperature: 0.3,
  maxRetries: 5,
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

<Note>
  Some providers do not support all common settings. If you use a setting with a
  provider that does not support it, a warning will be generated. You can check
  the `warnings` property in the result object to see if any warnings were
  generated.
</Note>

### `maxTokens`

Maximum number of tokens to generate.

### `temperature`

Temperature setting.

The value is passed through to the provider. The range depends on the provider and model.
For most providers, `0` means almost deterministic results, and higher values mean more randomness.

It is recommended to set either `temperature` or `topP`, but not both.

### `topP`

Nucleus sampling.

The value is passed through to the provider. The range depends on the provider and model.
For most providers, nucleus sampling is a number between 0 and 1.
E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.

It is recommended to set either `temperature` or `topP`, but not both.

### `topK`

Only sample from the top K options for each subsequent token.

Used to remove "long tail" low probability responses.
Recommended for advanced use cases only. You usually only need to use `temperature`.

### `presencePenalty`

The presence penalty affects the likelihood of the model to repeat information that is already in the prompt.

The value is passed through to the provider. The range depends on the provider and model.
For most providers, `0` means no penalty.

### `frequencyPenalty`

The frequency penalty affects the likelihood of the model to repeatedly use the same words or phrases.

The value is passed through to the provider. The range depends on the provider and model.
For most providers, `0` means no penalty.

### `stopSequences`

The stop sequences to use for stopping the text generation.

If set, the model will stop generating text when one of the stop sequences is generated.
Providers may have limits on the number of stop sequences.

### `seed`

It is the seed (integer) to use for random sampling.
If set and supported by the model, calls will generate deterministic results.

### `maxRetries`

Maximum number of retries. Set to 0 to disable retries. Default: `2`.

### `abortSignal`

An optional abort signal that can be used to cancel the call.

The abort signal can e.g. be forwarded from a user interface to cancel the call,
or to define a timeout.

#### Example: Timeout

```ts
const result = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Invent a new holiday and describe its traditions.',
  abortSignal: AbortSignal.timeout(5000), // 5 seconds
});
```

### `headers`

Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.

You can use the request headers to provide additional information to the provider,
depending on what the provider supports. For example, some observability providers support
headers such as `Prompt-Id`.

```ts
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Invent a new holiday and describe its traditions.',
  headers: {
    'Prompt-Id': 'my-prompt-id',
  },
});
```

<Note>
  The `headers` setting is for request-specific headers. You can also set
  `headers` in the provider configuration. These headers will be sent with every
  request made by the provider.
</Note>

---
title: Embeddings
description: Learn how to embed values with the AI SDK.
---

# Embeddings

Embeddings are a way to represent words, phrases, or images as vectors in a high-dimensional space.
In this space, similar words are close to each other, and the distance between words can be used to measure their similarity.

## Embedding a Single Value

The AI SDK provides the [`embed`](/docs/reference/ai-sdk-core/embed) function to embed single values, which is useful for tasks such as finding similar words
or phrases or clustering text.
You can use it with embeddings models, e.g. `openai.embedding('text-embedding-3-large')` or `mistral.embedding('mistral-embed')`.

```tsx
import { embed } from 'ai';
import { openai } from '@ai-sdk/openai';

// 'embedding' is a single embedding object (number[])
const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
});
```

## Embedding Many Values

When loading data, e.g. when preparing a data store for retrieval-augmented generation (RAG),
it is often useful to embed many values at once (batch embedding).

The AI SDK provides the [`embedMany`](/docs/reference/ai-sdk-core/embed-many) function for this purpose.
Similar to `embed`, you can use it with embeddings models,
e.g. `openai.embedding('text-embedding-3-large')` or `mistral.embedding('mistral-embed')`.

```tsx
import { openai } from '@ai-sdk/openai';
import { embedMany } from 'ai';

// 'embeddings' is an array of embedding objects (number[][]).
// It is sorted in the same order as the input values.
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: [
    'sunny day at the beach',
    'rainy afternoon in the city',
    'snowy night in the mountains',
  ],
});
```

## Embedding Similarity

After embedding values, you can calculate the similarity between them using the [`cosineSimilarity`](/docs/reference/ai-sdk-core/cosine-similarity) function.
This is useful to e.g. find similar words or phrases in a dataset.
You can also rank and filter related items based on their similarity.

```ts highlight={"2,10"}
import { openai } from '@ai-sdk/openai';
import { cosineSimilarity, embedMany } from 'ai';

const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: ['sunny day at the beach', 'rainy afternoon in the city'],
});

console.log(
  `cosine similarity: ${cosineSimilarity(embeddings[0], embeddings[1])}`,
);
```

## Token Usage

Many providers charge based on the number of tokens used to generate embeddings.
Both `embed` and `embedMany` provide token usage information in the `usage` property of the result object:

```ts highlight={"4,9"}
import { openai } from '@ai-sdk/openai';
import { embed } from 'ai';

const { embedding, usage } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
});

console.log(usage); // { tokens: 10 }
```

## Settings

### Retries

Both `embed` and `embedMany` accept an optional `maxRetries` parameter of type `number`
that you can use to set the maximum number of retries for the embedding process.
It defaults to `2` retries (3 attempts in total). You can set it to `0` to disable retries.

```ts highlight={"7"}
import { openai } from '@ai-sdk/openai';
import { embed } from 'ai';

const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
  maxRetries: 0, // Disable retries
});
```

### Abort Signals and Timeouts

Both `embed` and `embedMany` accept an optional `abortSignal` parameter of
type [`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)
that you can use to abort the embedding process or set a timeout.

```ts highlight={"7"}
import { openai } from '@ai-sdk/openai';
import { embed } from 'ai';

const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
  abortSignal: AbortSignal.timeout(1000), // Abort after 1 second
});
```

### Custom Headers

Both `embed` and `embedMany` accept an optional `headers` parameter of type `Record<string, string>`
that you can use to add custom headers to the embedding request.

```ts highlight={"7"}
import { openai } from '@ai-sdk/openai';
import { embed } from 'ai';

const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
  headers: { 'X-Custom-Header': 'custom-value' },
});
```

## Embedding Providers & Models

Several providers offer embedding models:

| Provider                                                                                  | Model                           | Embedding Dimensions |
| ----------------------------------------------------------------------------------------- | ------------------------------- | -------------------- |
| [OpenAI](/providers/ai-sdk-providers/openai#embedding-models)                             | `text-embedding-3-large`        | 3072                 |
| [OpenAI](/providers/ai-sdk-providers/openai#embedding-models)                             | `text-embedding-3-small`        | 1536                 |
| [OpenAI](/providers/ai-sdk-providers/openai#embedding-models)                             | `text-embedding-ada-002`        | 1536                 |
| [Google Generative AI](/providers/ai-sdk-providers/google-generative-ai#embedding-models) | `text-embedding-004`            | 768                  |
| [Mistral](/providers/ai-sdk-providers/mistral#embedding-models)                           | `mistral-embed`                 | 1024                 |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-english-v3.0`            | 1024                 |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-multilingual-v3.0`       | 1024                 |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-english-light-v3.0`      | 384                  |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-multilingual-light-v3.0` | 384                  |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-english-v2.0`            | 4096                 |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-english-light-v2.0`      | 1024                 |
| [Cohere](/providers/ai-sdk-providers/cohere#embedding-models)                             | `embed-multilingual-v2.0`       | 768                  |
| [Amazon Bedrock](/providers/ai-sdk-providers/amazon-bedrock#embedding-models)             | `amazon.titan-embed-text-v1`    | 1024                 |
| [Amazon Bedrock](/providers/ai-sdk-providers/amazon-bedrock#embedding-models)             | `amazon.titan-embed-text-v2:0`  | 1024                 |

---
title: Provider Management
description: Learn how to work with multiple providers
---

# Provider Management

<Note type="warning">Provider management is an experimental feature.</Note>

When you work with multiple providers and models, it is often desirable to manage them in a central place
and access the models through simple string ids.

The AI SDK offers [custom providers](/docs/reference/ai-sdk-core/custom-provider) and
a [provider registry](/docs/reference/ai-sdk-core/provider-registry) for this purpose.
With custom providers, you can **pre-configure model settings**, **provide model name aliases**,
and **limit the available models** .
The provider registry lets you mix **multiple providers** and access them through simple string ids.

## Custom Providers

You can create a [custom provider](/docs/reference/ai-sdk-core/custom-provider) using `experimental_customProvider`.

### Example: custom model settings

You might want to override the default model settings for a provider or provide model name aliases
with pre-configured settings.

```ts
import { openai as originalOpenAI } from '@ai-sdk/openai';
import { experimental_customProvider as customProvider } from 'ai';

// custom provider with different model settings:
export const openai = customProvider({
  languageModels: {
    // replacement model with custom settings:
    'gpt-4o': originalOpenAI('gpt-4o', { structuredOutputs: true }),
    // alias model with custom settings:
    'gpt-4o-mini-structured': originalOpenAI('gpt-4o-mini', {
      structuredOutputs: true,
    }),
  },
  fallbackProvider: originalOpenAI,
});
```

### Example: model name alias

You can also provide model name aliases, so you can update the model version in one place in the future:

```ts
import { anthropic as originalAnthropic } from '@ai-sdk/anthropic';
import { experimental_customProvider as customProvider } from 'ai';

// custom provider with alias names:
export const anthropic = customProvider({
  languageModels: {
    opus: originalAnthropic('claude-3-opus-20240229'),
    sonnet: originalAnthropic('claude-3-5-sonnet-20240620'),
    haiku: originalAnthropic('claude-3-haiku-20240307'),
  },
  fallbackProvider: originalAnthropic,
});
```

### Example: limit available models

You can limit the available models in the system, even if you have multiple providers.

```ts
import { anthropic } from '@ai-sdk/anthropic';
import { openai } from '@ai-sdk/openai';
import { experimental_customProvider as customProvider } from 'ai';

export const myProvider = customProvider({
  languageModels: {
    'text-medium': anthropic('claude-3-5-sonnet-20240620'),
    'text-small': openai('gpt-4o-mini'),
    'structure-medium': openai('gpt-4o', { structuredOutputs: true }),
    'structure-fast': openai('gpt-4o-mini', { structuredOutputs: true }),
  },
  embeddingModels: {
    emdedding: openai.textEmbeddingModel('text-embedding-3-small'),
  },
  // no fallback provider
});
```

## Provider Registry

You can create a [provider registry](/docs/reference/ai-sdk-core/provider-registry) with multiple providers and models using `experimental_createProviderRegistry`.

### Example: Setup

```ts filename={"registry.ts"}
import { anthropic } from '@ai-sdk/anthropic';
import { createOpenAI } from '@ai-sdk/openai';
import { experimental_createProviderRegistry as createProviderRegistry } from 'ai';

export const registry = createProviderRegistry({
  // register provider with prefix and default setup:
  anthropic,

  // register provider with prefix and custom setup:
  openai: createOpenAI({
    apiKey: process.env.OPENAI_API_KEY,
  }),
});
```

### Example: Use language models

You can access language models by using the `languageModel` method on the registry.
The provider id will become the prefix of the model id: `providerId:modelId`.

```ts highlight={"5"}
import { generateText } from 'ai';
import { registry } from './registry';

const { text } = await generateText({
  model: registry.languageModel('openai:gpt-4-turbo'),
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

### Example: Use text embedding models

You can access text embedding models by using the `textEmbeddingModel` method on the registry.
The provider id will become the prefix of the model id: `providerId:modelId`.

```ts highlight={"5"}
import { embed } from 'ai';
import { registry } from './registry';

const { embedding } = await embed({
  model: registry.textEmbeddingModel('openai:text-embedding-3-small'),
  value: 'sunny day at the beach',
});
```

---
title: Language Model Middleware
description: Learn how to use middleware to enhance the behavior of language models
---

# Language Model Middleware

<Note type="warning">
  Language model middleware is an experimental feature.
</Note>

Language model middleware is a way to enhance the behavior of language models
by intercepting and modifying the calls to the language model.

It can be used to add features like guardrails, RAG, caching, and logging
in a language model agnostic way. Such middleware can be developed and
distributed independently from the language models that they are applied to.

## Using Language Model Middleware

You can use language model middleware with the `wrapLanguageModel` function.
It takes a language model and a language model middleware and returns a new
language model that incorporates the middleware.

```ts
import { experimental_wrapLanguageModel as wrapLanguageModel } from 'ai';

const wrappedLanguageModel = wrapLanguageModel({
  model: yourModel,
  middleware: yourLanguageModelMiddleware,
});
```

The wrapped language model can be used just like any other language model, e.g. in `streamText`:

```ts highlight="2"
const result = streamText({
  model: wrappedLanguageModel,
  prompt: 'What cities are in the United States?',
});
```

## Implementing Language Model Middleware

<Note>
  Implementing language model middleware is advanced functionality and requires
  a solid understanding of the [language model
  specification](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts).
</Note>

You can implement any of the following three function to modify the behavior of the language model:

1. `transformParams`: Transforms the parameters before they are passed to the language model, for both `doGenerate` and `doStream`.
2. `wrapGenerate`: Wraps the `doGenerate` method of the [language model](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts).
   You can modify the parameters, call the language model, and modify the result.
3. `wrapStream`: Wraps the `doStream` method of the [language model](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts).
   You can modify the parameters, call the language model, and modify the result.

Here are some examples of how to implement language model middleware:

## Examples

<Note>
  These examples are not meant to be used in production. They are just to show
  how you can use middleware to enhance the behavior of language models.
</Note>

### Logging

This example shows how to log the parameters and generated text of a language model call.

```ts
import type {
  Experimental_LanguageModelV1Middleware as LanguageModelV1Middleware,
  LanguageModelV1StreamPart,
} from 'ai';

export const yourLogMiddleware: LanguageModelV1Middleware = {
  wrapGenerate: async ({ doGenerate, params }) => {
    console.log('doGenerate called');
    console.log(`params: ${JSON.stringify(params, null, 2)}`);

    const result = await doGenerate();

    console.log('doGenerate finished');
    console.log(`generated text: ${result.text}`);

    return result;
  },

  wrapStream: async ({ doStream, params }) => {
    console.log('doStream called');
    console.log(`params: ${JSON.stringify(params, null, 2)}`);

    const { stream, ...rest } = await doStream();

    let generatedText = '';

    const transformStream = new TransformStream<
      LanguageModelV1StreamPart,
      LanguageModelV1StreamPart
    >({
      transform(chunk, controller) {
        if (chunk.type === 'text-delta') {
          generatedText += chunk.textDelta;
        }

        controller.enqueue(chunk);
      },

      flush() {
        console.log('doStream finished');
        console.log(`generated text: ${generatedText}`);
      },
    });

    return {
      stream: stream.pipeThrough(transformStream),
      ...rest,
    };
  },
};
```

### Caching

This example shows how to build a simple cache for the generated text of a language model call.

```ts
import type { Experimental_LanguageModelV1Middleware as LanguageModelV1Middleware } from 'ai';

const cache = new Map<string, any>();

export const yourCacheMiddleware: LanguageModelV1Middleware = {
  wrapGenerate: async ({ doGenerate, params }) => {
    const cacheKey = JSON.stringify(params);

    if (cache.has(cacheKey)) {
      return cache.get(cacheKey);
    }

    const result = await doGenerate();

    cache.set(cacheKey, result);

    return result;
  },

  // here you would implement the caching logic for streaming
};
```

### Retrieval Augmented Generation (RAG)

This example shows how to use RAG as middleware.

<Note>
  Helper functions like `getLastUserMessageText` and `findSources` are not part
  of the AI SDK. They are just used in this example to illustrate the concept of
  RAG.
</Note>

```ts
import type { Experimental_LanguageModelV1Middleware as LanguageModelV1Middleware } from 'ai';

export const yourRagMiddleware: LanguageModelV1Middleware = {
  transformParams: async ({ params }) => {
    const lastUserMessageText = getLastUserMessageText({
      prompt: params.prompt,
    });

    if (lastUserMessageText == null) {
      return params; // do not use RAG (send unmodified parameters)
    }

    const instruction =
      'Use the following information to answer the question:\n' +
      findSources({ text: lastUserMessageText })
        .map(chunk => JSON.stringify(chunk))
        .join('\n');

    return addToLastUserMessage({ params, text: instruction });
  },
};
```

### Guardrails

Guard rails are a way to ensure that the generated text of a language model call
is safe and appropriate. This example shows how to use guardrails as middleware.

```ts
import type { Experimental_LanguageModelV1Middleware as LanguageModelV1Middleware } from 'ai';

export const yourGuardrailMiddleware: LanguageModelV1Middleware = {
  wrapGenerate: async ({ doGenerate }) => {
    const { text, ...rest } = await doGenerate();

    // filtering approach, e.g. for PII or other sensitive information:
    const cleanedText = text?.replace(/badword/g, '<REDACTED>');

    return { text: cleanedText, ...rest };
  },

  // here you would implement the guardrail logic for streaming
  // Note: streaming guardrails are difficult to implement, because
  // you do not know the full content of the stream until it's finished.
};
```

---
title: Error Handling
description: Learn how to handle errors in the AI SDK Core
---

# Error Handling

## Handling regular errors

Regular errors are thrown and can be handled using the `try/catch` block.

```ts highlight="3,8-10"
import { generateText } from 'ai';

try {
  const { text } = await generateText({
    model: yourModel,
    prompt: 'Write a vegetarian lasagna recipe for 4 people.',
  });
} catch (error) {
  // handle error
}
```

## Handling streaming errors (simple streams)

When errors occur during streams that do not support error chunks,
the error is thrown as a regular error.
You can handle these errors using the `try/catch` block.

```ts highlight="3,12-14"
import { generateText } from 'ai';

try {
  const { textStream } = streamText({
    model: yourModel,
    prompt: 'Write a vegetarian lasagna recipe for 4 people.',
  });

  for await (const textPart of textStream) {
    process.stdout.write(textPart);
  }
} catch (error) {
  // handle error
}
```

## Handling streaming errors (streaming with `error` support)

Full streams support error parts.
You can handle those parts similar to other parts.
It is recommended to also add a try-catch block for errors that
happen outside of the streaming.

```ts highlight="13-17"
import { generateText } from 'ai';

try {
  const { fullStream } = streamText({
    model: yourModel,
    prompt: 'Write a vegetarian lasagna recipe for 4 people.',
  });

  for await (const part of fullStream) {
    switch (part.type) {
      // ... handle other part types

      case 'error': {
        const error = part.error;
        // handle error
        break;
      }
    }
  }
} catch (error) {
  // handle error
}
```

---
title: Testing
description: Learn how to use AI SDK Core mock providers for testing.
---

# Testing

Testing language models can be challenging, because they are non-deterministic
and calling them is slow and expensive.

To enable you to unit test your code that uses the AI SDK, the AI SDK Core
includes mock providers and test helpers. You can import the following helpers from `ai/test`:

- `MockEmbeddingModelV1`: A mock embedding model using the [embedding model v1 specification](https://github.com/vercel/ai/blob/main/packages/provider/src/embedding-model/v1/embedding-model-v1.ts).
- `MockLanguageModelV1`: A mock language model using the [language model v1 specification](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts).
- `mockId`: Provides an incrementing integer ID.
- `mockValues`: Iterates over an array of values with each call. Returns the last value when the array is exhausted.
- `simulateReadableStream`: Simulates a readable stream with delays.

With mock providers and test helpers, you can control the output of the AI SDK
and test your code in a repeatable and deterministic way without actually calling
a language model provider.

## Examples

You can use the test helpers with the AI Core functions in your unit tests:

### generateText

```ts
import { generateText } from 'ai';
import { MockLanguageModelV1 } from 'ai/test';

const result = await generateText({
  model: new MockLanguageModelV1({
    doGenerate: async () => ({
      rawCall: { rawPrompt: null, rawSettings: {} },
      finishReason: 'stop',
      usage: { promptTokens: 10, completionTokens: 20 },
      text: `Hello, world!`,
    }),
  }),
  prompt: 'Hello, test!',
});
```

### streamText

```ts
import { streamText } from 'ai';
import { simulateReadableStream, MockLanguageModelV1 } from 'ai/test';

const result = streamText({
  model: new MockLanguageModelV1({
    doStream: async () => ({
      stream: simulateReadableStream({
        chunks: [
          { type: 'text-delta', textDelta: 'Hello' },
          { type: 'text-delta', textDelta: ', ' },
          { type: 'text-delta', textDelta: `world!` },
          {
            type: 'finish',
            finishReason: 'stop',
            logprobs: undefined,
            usage: { completionTokens: 10, promptTokens: 3 },
          },
        ],
      }),
      rawCall: { rawPrompt: null, rawSettings: {} },
    }),
  }),
  prompt: 'Hello, test!',
});
```

### generateObject

```ts
import { generateObject } from 'ai';
import { MockLanguageModelV1 } from 'ai/test';
import { z } from 'zod';

const result = await generateObject({
  model: new MockLanguageModelV1({
    defaultObjectGenerationMode: 'json',
    doGenerate: async () => ({
      rawCall: { rawPrompt: null, rawSettings: {} },
      finishReason: 'stop',
      usage: { promptTokens: 10, completionTokens: 20 },
      text: `{"content":"Hello, world!"}`,
    }),
  }),
  schema: z.object({ content: z.string() }),
  prompt: 'Hello, test!',
});
```

### streamObject

```ts
import { streamObject } from 'ai';
import { simulateReadableStream, MockLanguageModelV1 } from 'ai/test';
import { z } from 'zod';

const result = streamObject({
  model: new MockLanguageModelV1({
    defaultObjectGenerationMode: 'json',
    doStream: async () => ({
      stream: simulateReadableStream({
        chunks: [
          { type: 'text-delta', textDelta: '{ ' },
          { type: 'text-delta', textDelta: '"content": ' },
          { type: 'text-delta', textDelta: `"Hello, ` },
          { type: 'text-delta', textDelta: `world` },
          { type: 'text-delta', textDelta: `!"` },
          { type: 'text-delta', textDelta: ' }' },
          {
            type: 'finish',
            finishReason: 'stop',
            logprobs: undefined,
            usage: { completionTokens: 10, promptTokens: 3 },
          },
        ],
      }),
      rawCall: { rawPrompt: null, rawSettings: {} },
    }),
  }),
  schema: z.object({ content: z.string() }),
  prompt: 'Hello, test!',
});
```

### Simulate Data Stream Protocol Responses

You can also simulate [Data Stream Protocol](/docs/ai-sdk-ui/stream-protocol#data-stream-protocol) responses for testing,
debugging, or demonstration purposes.

Here is a Next example:

```ts filename="route.ts"
import { simulateReadableStream } from 'ai/test';

export async function POST(req: Request) {
  return new Response(
    simulateReadableStream({
      initialDelayInMs: 1000, // Delay before the first chunk
      chunkDelayInMs: 300, // Delay between chunks
      chunks: [
        `0:"This"\n`,
        `0:" is an"\n`,
        `0:"example."\n`,
        `e:{"finishReason":"stop","usage":{"promptTokens":20,"completionTokens":50},"isContinued":false}\n`,
        `d:{"finishReason":"stop","usage":{"promptTokens":20,"completionTokens":50}}\n`,
      ],
    }).pipeThrough(new TextEncoderStream()),
    {
      status: 200,
      headers: {
        'X-Vercel-AI-Data-Stream': 'v1',
        'Content-Type': 'text/plain; charset=utf-8',
      },
    },
  );
}
```

---
title: Telemetry
description: Using OpenTelemetry with AI SDK Core
---

# Telemetry

<Note type="warning">
  AI SDK Telemetry is experimental and may change in the future.
</Note>

The AI SDK uses [OpenTelemetry](https://opentelemetry.io/) to collect telemetry data.
OpenTelemetry is an open-source observability framework designed to provide
standardized instrumentation for collecting telemetry data.

Check out the [AI SDK Observability Integrations](/providers/observability)
to see providers that offer monitoring and tracing for AI SDK applications.

## Enabling telemetry

For Next.js applications, please follow the [Next.js OpenTelemetry guide](https://nextjs.org/docs/app/building-your-application/optimizing/open-telemetry) to enable telemetry first.

You can then use the `experimental_telemetry` option to enable telemetry on specific function calls while the feature is experimental:

```ts highlight="4"
const result = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Write a short story about a cat.',
  experimental_telemetry: { isEnabled: true },
});
```

When telemetry is enabled, you can also control if you want to record the input values and the output values for the function.
By default, both are enabled. You can disable them by setting the `recordInputs` and `recordOutputs` options to `false`.

Disabling the recording of inputs and outputs can be useful for privacy, data transfer, and performance reasons.
You might for example want to disable recording inputs if they contain sensitive information.

## Telemetry Metadata

You can provide a `functionId` to identify the function that the telemetry data is for,
and `metadata` to include additional information in the telemetry data.

```ts highlight="6-10"
const result = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Write a short story about a cat.',
  experimental_telemetry: {
    isEnabled: true,
    functionId: 'my-awesome-function',
    metadata: {
      something: 'custom',
      someOtherThing: 'other-value',
    },
  },
});
```

## Custom Tracer

You may provide a `tracer` which must return an OpenTelemetry `Tracer`. This is useful in situations where
you want your traces to use a `TracerProvider` other than the one provided by the `@opentelemetry/api` singleton.

```ts highlight="7"
const tracerProvider = new NodeTracerProvider();
const result = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Write a short story about a cat.',
  experimental_telemetry: {
    isEnabled: true,
    tracer: tracerProvider.getTracer('ai'),
  },
});
```

## Collected Data

### generateText function

`generateText` records 3 types of spans:

- `ai.generateText` (span): the full length of the generateText call. It contains 1 or more `ai.generateText.doGenerate` spans.
  It contains the [basic LLM span information](#basic-llm-span-information) and the following attributes:

  - `operation.name`: `ai.generateText` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.generateText"`
  - `ai.prompt`: the prompt that was used when calling `generateText`
  - `ai.response.text`: the text that was generated
  - `ai.response.toolCalls`: the tool calls that were made as part of the generation (stringified JSON)
  - `ai.response.finishReason`: the reason why the generation finished
  - `ai.settings.maxSteps`: the maximum number of steps that were set

- `ai.generateText.doGenerate` (span): a provider doGenerate call. It can contain `ai.toolCall` spans.
  It contains the [call LLM span information](#call-llm-span-information) and the following attributes:

  - `operation.name`: `ai.generateText.doGenerate` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.generateText.doGenerate"`
  - `ai.prompt.format`: the format of the prompt
  - `ai.prompt.messages`: the messages that were passed into the provider
  - `ai.prompt.tools`: array of stringified tool definitions. The tools can be of type `function` or `provider-defined`.
    Function tools have a `name`, `description` (optional), and `parameters` (JSON schema).
    Provider-defined tools have a `name`, `id`, and `args` (Record).
  - `ai.prompt.toolChoice`: the stringified tool choice setting (JSON). It has a `type` property
    (`auto`, `none`, `required`, `tool`), and if the type is `tool`, a `toolName` property with the specific tool.
  - `ai.response.text`: the text that was generated
  - `ai.response.toolCalls`: the tool calls that were made as part of the generation (stringified JSON)
  - `ai.response.finishReason`: the reason why the generation finished

- `ai.toolCall` (span): a tool call that is made as part of the generateText call. See [Tool call spans](#tool-call-spans) for more details.

### streamText function

`streamText` records 3 types of spans and 2 types of events:

- `ai.streamText` (span): the full length of the streamText call. It contains a `ai.streamText.doStream` span.
  It contains the [basic LLM span information](#basic-llm-span-information) and the following attributes:

  - `operation.name`: `ai.streamText` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.streamText"`
  - `ai.prompt`: the prompt that was used when calling `streamText`
  - `ai.response.text`: the text that was generated
  - `ai.response.toolCalls`: the tool calls that were made as part of the generation (stringified JSON)
  - `ai.response.finishReason`: the reason why the generation finished
  - `ai.settings.maxSteps`: the maximum number of steps that were set

- `ai.streamText.doStream` (span): a provider doStream call.
  This span contains an `ai.stream.firstChunk` event and `ai.toolCall` spans.
  It contains the [call LLM span information](#call-llm-span-information) and the following attributes:

  - `operation.name`: `ai.streamText.doStream` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.streamText.doStream"`
  - `ai.prompt.format`: the format of the prompt
  - `ai.prompt.messages`: the messages that were passed into the provider
  - `ai.prompt.tools`: array of stringified tool definitions. The tools can be of type `function` or `provider-defined`.
    Function tools have a `name`, `description` (optional), and `parameters` (JSON schema).
    Provider-defined tools have a `name`, `id`, and `args` (Record).
  - `ai.prompt.toolChoice`: the stringified tool choice setting (JSON). It has a `type` property
    (`auto`, `none`, `required`, `tool`), and if the type is `tool`, a `toolName` property with the specific tool.
  - `ai.response.text`: the text that was generated
  - `ai.response.toolCalls`: the tool calls that were made as part of the generation (stringified JSON)
  - `ai.response.msToFirstChunk`: the time it took to receive the first chunk in milliseconds
  - `ai.response.msToFinish`: the time it took to receive the finish part of the LLM stream in milliseconds
  - `ai.response.avgCompletionTokensPerSecond`: the average number of completion tokens per second
  - `ai.response.finishReason`: the reason why the generation finished

- `ai.toolCall` (span): a tool call that is made as part of the generateText call. See [Tool call spans](#tool-call-spans) for more details.

- `ai.stream.firstChunk` (event): an event that is emitted when the first chunk of the stream is received.

  - `ai.response.msToFirstChunk`: the time it took to receive the first chunk

- `ai.stream.finish` (event): an event that is emitted when the finish part of the LLM stream is received.

It also records a `ai.stream.firstChunk` event when the first chunk of the stream is received.

### generateObject function

`generateObject` records 2 types of spans:

- `ai.generateObject` (span): the full length of the generateObject call. It contains 1 or more `ai.generateObject.doGenerate` spans.
  It contains the [basic LLM span information](#basic-llm-span-information) and the following attributes:

  - `operation.name`: `ai.generateObject` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.generateObject"`
  - `ai.prompt`: the prompt that was used when calling `generateObject`
  - `ai.schema`: Stringified JSON schema version of the schema that was passed into the `generateObject` function
  - `ai.schema.name`: the name of the schema that was passed into the `generateObject` function
  - `ai.schema.description`: the description of the schema that was passed into the `generateObject` function
  - `ai.response.object`: the object that was generated (stringified JSON)
  - `ai.settings.mode`: the object generation mode, e.g. `json`
  - `ai.settings.output`: the output type that was used, e.g. `object` or `no-schema`

- `ai.generateObject.doGenerate` (span): a provider doGenerate call.
  It contains the [call LLM span information](#call-llm-span-information) and the following attributes:

  - `operation.name`: `ai.generateObject.doGenerate` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.generateObject.doGenerate"`
  - `ai.prompt.format`: the format of the prompt
  - `ai.prompt.messages`: the messages that were passed into the provider
  - `ai.response.object`: the object that was generated (stringified JSON)
  - `ai.settings.mode`: the object generation mode
  - `ai.response.finishReason`: the reason why the generation finished

### streamObject function

`streamObject` records 2 types of spans and 1 type of event:

- `ai.streamObject` (span): the full length of the streamObject call. It contains 1 or more `ai.streamObject.doStream` spans.
  It contains the [basic LLM span information](#basic-llm-span-information) and the following attributes:

  - `operation.name`: `ai.streamObject` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.streamObject"`
  - `ai.prompt`: the prompt that was used when calling `streamObject`
  - `ai.schema`: Stringified JSON schema version of the schema that was passed into the `streamObject` function
  - `ai.schema.name`: the name of the schema that was passed into the `streamObject` function
  - `ai.schema.description`: the description of the schema that was passed into the `streamObject` function
  - `ai.response.object`: the object that was generated (stringified JSON)
  - `ai.settings.mode`: the object generation mode, e.g. `json`
  - `ai.settings.output`: the output type that was used, e.g. `object` or `no-schema`

- `ai.streamObject.doStream` (span): a provider doStream call.
  This span contains an `ai.stream.firstChunk` event.
  It contains the [call LLM span information](#call-llm-span-information) and the following attributes:

  - `operation.name`: `ai.streamObject.doStream` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.streamObject.doStream"`
  - `ai.prompt.format`: the format of the prompt
  - `ai.prompt.messages`: the messages that were passed into the provider
  - `ai.settings.mode`: the object generation mode
  - `ai.response.object`: the object that was generated (stringified JSON)
  - `ai.response.msToFirstChunk`: the time it took to receive the first chunk
  - `ai.response.finishReason`: the reason why the generation finished

- `ai.stream.firstChunk` (event): an event that is emitted when the first chunk of the stream is received.
  - `ai.response.msToFirstChunk`: the time it took to receive the first chunk

### embed function

`embed` records 2 types of spans:

- `ai.embed` (span): the full length of the embed call. It contains 1 `ai.embed.doEmbed` spans.
  It contains the [basic embedding span information](#basic-embedding-span-information) and the following attributes:

  - `operation.name`: `ai.embed` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.embed"`
  - `ai.value`: the value that was passed into the `embed` function
  - `ai.embedding`: a JSON-stringified embedding

- `ai.embed.doEmbed` (span): a provider doEmbed call.
  It contains the [basic embedding span information](#basic-embedding-span-information) and the following attributes:

  - `operation.name`: `ai.embed.doEmbed` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.embed.doEmbed"`
  - `ai.values`: the values that were passed into the provider (array)
  - `ai.embeddings`: an array of JSON-stringified embeddings

### embedMany function

`embedMany` records 2 types of spans:

- `ai.embedMany` (span): the full length of the embedMany call. It contains 1 or more `ai.embedMany.doEmbed` spans.
  It contains the [basic embedding span information](#basic-embedding-span-information) and the following attributes:

  - `operation.name`: `ai.embedMany` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.embedMany"`
  - `ai.values`: the values that were passed into the `embedMany` function
  - `ai.embeddings`: an array of JSON-stringified embedding

- `ai.embedMany.doEmbed` (span): a provider doEmbed call.
  It contains the [basic embedding span information](#basic-embedding-span-information) and the following attributes:

  - `operation.name`: `ai.embedMany.doEmbed` and the functionId that was set through `telemetry.functionId`
  - `ai.operationId`: `"ai.embedMany.doEmbed"`
  - `ai.values`: the values that were sent to the provider
  - `ai.embeddings`: an array of JSON-stringified embeddings for each value

## Span Details

### Basic LLM span information

Many spans that use LLMs (`ai.generateText`, `ai.generateText.doGenerate`, `ai.streamText`, `ai.streamText.doStream`,
`ai.generateObject`, `ai.generateObject.doGenerate`, `ai.streamObject`, `ai.streamObject.doStream`) contain the following attributes:

- `resource.name`: the functionId that was set through `telemetry.functionId`
- `ai.model.id`: the id of the model
- `ai.model.provider`: the provider of the model
- `ai.request.headers.*`: the request headers that were passed in through `headers`
- `ai.settings.maxRetries`: the maximum number of retries that were set
- `ai.telemetry.functionId`: the functionId that was set through `telemetry.functionId`
- `ai.telemetry.metadata.*`: the metadata that was passed in through `telemetry.metadata`
- `ai.usage.completionTokens`: the number of completion tokens that were used
- `ai.usage.promptTokens`: the number of prompt tokens that were used

### Call LLM span information

Spans that correspond to individual LLM calls (`ai.generateText.doGenerate`, `ai.streamText.doStream`, `ai.generateObject.doGenerate`, `ai.streamObject.doStream`) contain
[basic LLM span information](#basic-llm-span-information) and the following attributes:

- `ai.response.model`: the model that was used to generate the response. This can be different from the model that was requested if the provider supports aliases.
- `ai.response.id`: the id of the response. Uses the ID from the provider when available.
- `ai.response.timestamp`: the timestamp of the response. Uses the timestamp from the provider when available.
- [Semantic Conventions for GenAI operations](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/)
  - `gen_ai.system`: the provider that was used
  - `gen_ai.request.model`: the model that was requested
  - `gen_ai.request.temperature`: the temperature that was set
  - `gen_ai.request.max_tokens`: the maximum number of tokens that were set
  - `gen_ai.request.frequency_penalty`: the frequency penalty that was set
  - `gen_ai.request.presence_penalty`: the presence penalty that was set
  - `gen_ai.request.top_k`: the topK parameter value that was set
  - `gen_ai.request.top_p`: the topP parameter value that was set
  - `gen_ai.request.stop_sequences`: the stop sequences
  - `gen_ai.response.finish_reasons`: the finish reasons that were returned by the provider
  - `gen_ai.response.model`: the model that was used to generate the response. This can be different from the model that was requested if the provider supports aliases.
  - `gen_ai.response.id`: the id of the response. Uses the ID from the provider when available.
  - `gen_ai.usage.input_tokens`: the number of prompt tokens that were used
  - `gen_ai.usage.output_tokens`: the number of completion tokens that were used

### Basic embedding span information

Many spans that use embedding models (`ai.embed`, `ai.embed.doEmbed`, `ai.embedMany`, `ai.embedMany.doEmbed`) contain the following attributes:

- `ai.model.id`: the id of the model
- `ai.model.provider`: the provider of the model
- `ai.request.headers.*`: the request headers that were passed in through `headers`
- `ai.settings.maxRetries`: the maximum number of retries that were set
- `ai.telemetry.functionId`: the functionId that was set through `telemetry.functionId`
- `ai.telemetry.metadata.*`: the metadata that was passed in through `telemetry.metadata`
- `ai.usage.tokens`: the number of tokens that were used
- `resource.name`: the functionId that was set through `telemetry.functionId`

### Tool call spans

Tool call spans (`ai.toolCall`) contain the following attributes:

- `operation.name`: `"ai.toolCall"`
- `ai.operationId`: `"ai.toolCall"`
- `ai.toolCall.name`: the name of the tool
- `ai.toolCall.id`: the id of the tool call
- `ai.toolCall.args`: the parameters of the tool call
- `ai.toolCall.result`: the result of the tool call. Only available if the tool call is successful and the result is serializable.

---
title: AI SDK Core
description: Learn about AI SDK Core.
---

# AI SDK Core

<IndexCards
  cards={[
    {
      title: 'Overview',
      description:
        'Learn about AI SDK Core and how to work with Large Language Models (LLMs).',
      href: '/docs/ai-sdk-core/overview',
    },
    {
      title: 'Generating Text',
      description: 'Learn how to generate text.',
      href: '/docs/ai-sdk-core/generating-text',
    },
    {
      title: 'Generating Structured Data',
      description: 'Learn how to generate structured data.',
      href: '/docs/ai-sdk-core/generating-structured-data',
    },
    {
      title: 'Tool Calling',
      description: 'Learn how to do tool calling with AI SDK Core.',
      href: '/docs/ai-sdk-core/tools-and-tool-calling',
    },
    {
      title: 'Agents',
      description: 'Learn how to use agents with AI SDK Core.',
      href: '/docs/ai-sdk-core/agents',
    },
    {
      title: 'Prompt Engineering',
      description: 'Learn how to write prompts with AI SDK Core.',
      href: '/docs/ai-sdk-core/prompt-engineering',
    },
    {
      title: 'Settings',
      description:
        'Learn how to set up settings for language models generations.',
      href: '/docs/ai-sdk-core/settings',
    },
    {
      title: 'Embeddings',
      description: 'Learn how to use embeddings with AI SDK Core.',
      href: '/docs/ai-sdk-core/embeddings',
    },
    {
      title: 'Provider Management',
      description: 'Learn how to work with multiple providers.',
      href: '/docs/ai-sdk-core/provider-management',
    },
    {
      title: 'Middleware',
      description: 'Learn how to use middleware with AI SDK Core.',
      href: '/docs/ai-sdk-core/middleware',
    },
    {
      title: 'Error Handling',
      description: 'Learn how to handle errors with AI SDK Core.',
      href: '/docs/ai-sdk-core/error-handling',
    },
    {
      title: 'Testing',
      description: 'Learn how to test with AI SDK Core.',
      href: '/docs/ai-sdk-core/testing',
    },
    {
      title: 'Telemetry',
      description: 'Learn how to use telemetry with AI SDK Core.',
      href: '/docs/ai-sdk-core/telemetry',
    },
  ]}
/>

---
title: Overview
description: An overview of AI SDK UI.
---

# AI SDK UI

AI SDK UI is designed to help you build interactive chat, completion, and assistant applications with ease. It is a **framework-agnostic toolkit**, streamlining the integration of advanced AI functionalities into your applications.

AI SDK UI provides robust abstractions that simplify the complex tasks of managing chat streams and UI updates on the frontend, enabling you to develop dynamic AI-driven interfaces more efficiently. With four main hooks — **`useChat`**, **`useCompletion`**, **`useObject`**, and **`useAssistant`** — you can incorporate real-time chat capabilities, text completions, streamed JSON, and interactive assistant features into your app.

- **[`useChat`](/docs/ai-sdk-ui/chatbot)** offers real-time streaming of chat messages, abstracting state management for inputs, messages, loading, and errors, allowing for seamless integration into any UI design.
- **[`useCompletion`](/docs/ai-sdk-ui/completion)** enables you to handle text completions in your applications, managing the prompt input and automatically updating the UI as new completions are streamed.
- **[`useObject`](/docs/ai-sdk-ui/object-generation)** is a hook that allows you to consume streamed JSON objects, providing a simple way to handle and display structured data in your application.
- **[`useAssistant`](/docs/ai-sdk-ui/openai-assistants)** is designed to facilitate interaction with OpenAI-compatible assistant APIs, managing UI state and updating it automatically as responses are streamed.

These hooks are designed to reduce the complexity and time required to implement AI interactions, letting you focus on creating exceptional user experiences.

## UI Framework Support

AI SDK UI supports the following frameworks: [React](https://react.dev/), [Svelte](https://svelte.dev/), [Vue.js](https://vuejs.org/), and [SolidJS](https://www.solidjs.com/).
Here is a comparison of the supported functions across these frameworks:

| Function                                                  | React               | Svelte              | Vue.js              | SolidJS             |
| --------------------------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| [useChat](/docs/reference/ai-sdk-ui/use-chat)             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useChat](/docs/reference/ai-sdk-ui/use-chat) attachments | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useCompletion](/docs/reference/ai-sdk-ui/use-completion) | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useObject](/docs/reference/ai-sdk-ui/use-object)         | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useAssistant](/docs/reference/ai-sdk-ui/use-assistant)   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |

<Note>
  [Contributions](https://github.com/vercel/ai/blob/main/CONTRIBUTING.md) are
  welcome to implement missing features for non-React frameworks.
</Note>

## API Reference

Please check out the [AI SDK UI API Reference](/docs/reference/ai-sdk-ui) for more details on each function.

---
title: Chatbot
description: Learn how to use the useChat hook.
---

# Chatbot

The `useChat` hook makes it effortless to create a conversational user interface for your chatbot application. It enables the streaming of chat messages from your AI provider, manages the chat state, and updates the UI automatically as new messages arrive.

To summarize, the `useChat` hook provides the following features:

- **Message Streaming**: All the messages from the AI provider are streamed to the chat UI in real-time.
- **Managed States**: The hook manages the states for input, messages, loading, error and more for you.
- **Seamless Integration**: Easily integrate your chat AI into any design or layout with minimal effort.

In this guide, you will learn how to use the `useChat` hook to create a chatbot application with real-time message streaming.
Check out our [chatbot with tools guide](/docs/ai-sdk-ui/chatbot-with-tool-calling) to learn how to use tools in your chatbot.
Let's start with the following example first.

## Example

```tsx filename='app/page.tsx'
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({});

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input name="prompt" value={input} onChange={handleInputChange} />
        <button type="submit">Submit</button>
      </form>
    </>
  );
}
```

```ts filename='app/api/chat/route.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    system: 'You are a helpful assistant.',
    messages,
  });

  return result.toDataStreamResponse();
}
```

In the `Page` component, the `useChat` hook will request to your AI provider endpoint whenever the user submits a message.
The messages are then streamed back in real-time and displayed in the chat UI.

This enables a seamless chat experience where the user can see the AI response as soon as it is available,
without having to wait for the entire response to be received.

## Customized UI

`useChat` also provides ways to manage the chat message and input states via code, show loading and error states, and update messages without being triggered by user interactions.

### Loading State

The `isLoading` state returned by the `useChat` hook can be used for several
purposes

- To show a loading spinner while the chatbot is processing the user's message.
- To show a "Stop" button to abort the current message.
- To disable the submit button.

```tsx filename='app/page.tsx' highlight="6,20-27,34"
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, stop } =
    useChat({});

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.content}
        </div>
      ))}

      {isLoading && (
        <div>
          <Spinner />
          <button type="button" onClick={() => stop()}>
            Stop
          </button>
        </div>
      )}

      <form onSubmit={handleSubmit}>
        <input
          name="prompt"
          value={input}
          onChange={handleInputChange}
          disabled={isLoading}
        />
        <button type="submit">Submit</button>
      </form>
    </>
  );
}
```

### Error State

Similarly, the `error` state reflects the error object thrown during the fetch request.
It can be used to display an error message, disable the submit button, or show a retry button:

<Note>
  We recommend showing a generic error message to the user, such as "Something
  went wrong." This is a good practice to avoid leaking information from the
  server.
</Note>

```tsx file="app/page.tsx" highlight="6,18-25,31"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, error, reload } =
    useChat({});

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      {error && (
        <>
          <div>An error occurred.</div>
          <button type="button" onClick={() => reload()}>
            Retry
          </button>
        </>
      )}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          disabled={error != null}
        />
      </form>
    </div>
  );
}
```

Please also see the [error handling](/docs/ai-sdk-ui/error-handling) guide for more information.

### Modify messages

Sometimes, you may want to directly modify some existing messages. For example, a delete button can be added to each message to allow users to remove them from the chat history.

The `setMessages` function can help you achieve these tasks:

```tsx
const { messages, setMessages, ... } = useChat()

const handleDelete = (id) => {
  setMessages(messages.filter(message => message.id !== id))
}

return <>
  {messages.map(message => (
    <div key={message.id}>
      {message.role === 'user' ? 'User: ' : 'AI: '}
      {message.content}
      <button onClick={() => handleDelete(message.id)}>Delete</button>
    </div>
  ))}
  ...
```

You can think of `messages` and `setMessages` as a pair of `state` and `setState` in React.

### Controlled input

In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.

The following example demonstrates how to use more granular APIs like `setInput` and `append` with your custom input and submit button components:

```tsx
const { input, setInput, append } = useChat()

return <>
  <MyCustomInput value={input} onChange={value => setInput(value)} />
  <MySubmitButton onClick={() => {
    // Send a new message to the AI provider
    append({
      role: 'user',
      content: input,
    })
  }}/>
  ...
```

### Cancelation and regeneration

It's also a common use case to abort the response message while it's still streaming back from the AI provider. You can do this by calling the `stop` function returned by the `useChat` hook.

```tsx
const { stop, isLoading, ... } = useChat()

return <>
  <button onClick={stop} disabled={!isLoading}>Stop</button>
  ...
```

When the user clicks the "Stop" button, the fetch request will be aborted. This avoids consuming unnecessary resources and improves the UX of your chatbot application.

Similarly, you can also request the AI provider to reprocess the last message by calling the `reload` function returned by the `useChat` hook:

```tsx
const { reload, isLoading, ... } = useChat()

return <>
  <button onClick={reload} disabled={isLoading}>Regenerate</button>
  ...
</>
```

When the user clicks the "Regenerate" button, the AI provider will regenerate the last message and replace the current one correspondingly.

### Throttling UI Updates

<Note>This feature is currently only available for React.</Note>

By default, the `useChat` hook will trigger a render every time a new chunk is received.
You can throttle the UI updates with the `experimental_throttle` option.

```tsx filename="page.tsx" highlight="2-3"
const { messages, ... } = useChat({
  // Throttle the messages and data updates to 50ms:
  experimental_throttle: 50
})
```

## Event Callbacks

`useChat` provides optional event callbacks that you can use to handle different stages of the chatbot lifecycle:

- `onFinish`: Called when the assistant message is completed
- `onError`: Called when an error occurs during the fetch request.
- `onResponse`: Called when the response from the API is received.

These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx
import { Message } from 'ai/react';

const {
  /* ... */
} = useChat({
  onFinish: (message, { usage, finishReason }) => {
    console.log('Finished streaming message:', message);
    console.log('Token usage:', usage);
    console.log('Finish reason:', finishReason);
  },
  onError: error => {
    console.error('An error occurred:', error);
  },
  onResponse: response => {
    console.log('Received HTTP response from server:', response);
  },
});
```

It's worth noting that you can abort the processing by throwing an error in the `onResponse` callback. This will trigger the `onError` callback and stop the message from being appended to the chat UI. This can be useful for handling unexpected responses from the AI provider.

## Request Configuration

### Custom headers, body, and credentials

By default, the `useChat` hook sends a HTTP POST request to the `/api/chat` endpoint with the message list as the request body. You can customize the request by passing additional options to the `useChat` hook:

```tsx
const { messages, input, handleInputChange, handleSubmit } = useChat({
  api: '/api/custom-chat',
  headers: {
    Authorization: 'your_token',
  },
  body: {
    user_id: '123',
  },
  credentials: 'same-origin',
});
```

In this example, the `useChat` hook sends a POST request to the `/api/custom-chat` endpoint with the specified headers, additional body fields, and credentials for that fetch request. On your server side, you can handle the request with these additional information.

### Setting custom body fields per request

You can configure custom `body` fields on a per-request basis using the `body` option of the `handleSubmit` function.
This is useful if you want to pass in additional information to your backend that is not part of the message list.

```tsx filename="app/page.tsx" highlight="18-20"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      <form
        onSubmit={event => {
          handleSubmit(event, {
            body: {
              customKey: 'customValue',
            },
          });
        }}
      >
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

You can retrieve these custom fields on your server side by destructuring the request body:

```ts filename="app/api/chat/route.ts" highlight="3"
export async function POST(req: Request) {
  // Extract addition information ("customKey") from the body of the request:
  const { messages, customKey } = await req.json();
  //...
}
```

## Controlling the response stream

With `streamText`, you can control how error messages and usage information are sent back to the client.

### Error Messages

By default, the error message is suppressed for security reasons. You can forward it or send your own error message by providing a `getErrorMessage` function:

```ts filename="app/api/chat/route.ts" highlight="13-27"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages,
  });

  return result.toDataStreamResponse({
    getErrorMessage: error => {
      if (error == null) {
        return 'unknown error';
      }

      if (typeof error === 'string') {
        return error;
      }

      if (error instanceof Error) {
        return error.message;
      }

      return JSON.stringify(error);
    },
  });
}
```

### Usage Information

By default, the usage information is sent back to the client. You can disable it by setting the `sendUsage` option to `false`:

```ts filename="app/api/chat/route.ts" highlight="13"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages,
  });

  return result.toDataStreamResponse({
    sendUsage: false,
  });
}
```

### Text Streams

`useChat` can handle plain text streams by setting the `streamProtocol` option to `text`:

```tsx filename="app/page.tsx" highlight="7"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages } = useChat({
    streamProtocol: 'text',
  });

  return <>...</>;
}
```

This configuration also works with other backend servers that stream plain text.
Check out the [stream protocol guide](/docs/ai-sdk-ui/stream-protocol) for more information.

<Note>
  When using `streamProtocol: 'text'`, tool calls, usage information and finish
  reasons are not available.
</Note>

## Empty Submissions

You can configure the `useChat` hook to allow empty submissions by setting the `allowEmptySubmit` option to `true`.

```tsx filename="app/page.tsx" highlight="18"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      <form
        onSubmit={event => {
          handleSubmit(event, {
            allowEmptySubmit: true,
          });
        }}
      >
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

## Attachments (Experimental)

The `useChat` hook supports sending attachments along with a message as well as rendering them on the client. This can be useful for building applications that involve sending images, files, or other media content to the AI provider.

> **Note:** Attachments is currently only available for React frameworks.

There are two ways to send attachments with a message, either by providing a `FileList` object or a list of URLs to the `handleSubmit` function:

### FileList

By using `FileList`, you can send multiple files as attachments along with a message using the file input element. The `useChat` hook will automatically convert them into data URLs and send them to the AI provider.

> **Note:** Currently, only `image/*` and `text/*` content types get automatically converted into [multi-modal content parts](https://sdk.vercel.ai/docs/foundations/prompts#multi-modal-messages). You will need to handle other content types manually.

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';
import { useRef, useState } from 'react';

export default function Page() {
  const { messages, input, handleSubmit, handleInputChange, isLoading } =
    useChat();

  const [files, setFiles] = useState<FileList | undefined>(undefined);
  const fileInputRef = useRef<HTMLInputElement>(null);

  return (
    <div>
      <div>
        {messages.map(message => (
          <div key={message.id}>
            <div>{`${message.role}: `}</div>

            <div>
              {message.content}

              <div>
                {message.experimental_attachments
                  ?.filter(attachment =>
                    attachment.contentType.startsWith('image/'),
                  )
                  .map((attachment, index) => (
                    <img
                      key={`${message.id}-${index}`}
                      src={attachment.url}
                      alt={attachment.name}
                    />
                  ))}
              </div>
            </div>
          </div>
        ))}
      </div>

      <form
        onSubmit={event => {
          handleSubmit(event, {
            experimental_attachments: files,
          });

          setFiles(undefined);

          if (fileInputRef.current) {
            fileInputRef.current.value = '';
          }
        }}
      >
        <input
          type="file"
          onChange={event => {
            if (event.target.files) {
              setFiles(event.target.files);
            }
          }}
          multiple
          ref={fileInputRef}
        />
        <input
          value={input}
          placeholder="Send message..."
          onChange={handleInputChange}
          disabled={isLoading}
        />
      </form>
    </div>
  );
}
```

### URLs

You can also send URLs as attachments along with a message. This can be useful for sending links to external resources or media content.

> **Note:** The URL can also be a data URL, which is a base64-encoded string that represents the content of a file. Currently, only `image/*` content types get automatically converted into [multi-modal content parts](https://sdk.vercel.ai/docs/foundations/prompts#multi-modal-messages). You will need to handle other content types manually.

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';
import { Attachment } from '@ai-sdk/ui-utils';

export default function Page() {
  const { messages, input, handleSubmit, handleInputChange, isLoading } =
    useChat();

  const [attachments] = useState<Attachment[]>([
    {
      name: 'earth.png',
      contentType: 'image/png',
      url: 'https://example.com/earth.png',
    },
    {
      name: 'moon.png',
      contentType: 'image/png',
      url: 'data:image/png;base64,iVBORw0KGgo...',
    },
  ]);

  return (
    <div>
      <div>
        {messages.map(message => (
          <div key={message.id}>
            <div>{`${message.role}: `}</div>

            <div>
              {message.content}

              <div>
                {message.experimental_attachments
                  ?.filter(attachment =>
                    attachment.contentType?.startsWith('image/'),
                  )
                  .map((attachment, index) => (
                    <img
                      key={`${message.id}-${index}`}
                      src={attachment.url}
                      alt={attachment.name}
                    />
                  ))}
              </div>
            </div>
          </div>
        ))}
      </div>

      <form
        onSubmit={event => {
          handleSubmit(event, {
            experimental_attachments: attachments,
          });
        }}
      >
        <input
          value={input}
          placeholder="Send message..."
          onChange={handleInputChange}
          disabled={isLoading}
        />
      </form>
    </div>
  );
}
```

---
title: Chatbot with Tools
description: Learn how to use tools with the useChat hook.
---

# Chatbot with Tools

With [`useChat`](/docs/reference/ai-sdk-ui/use-chat) and [`streamText`](/docs/reference/ai-sdk-core/stream-text), you can use tools in your chatbot application.
The AI SDK supports three types of tools in this context:

1. Automatically executed server-side tools
2. Automatically executed client-side tools
3. Tools that require user interaction, such as confirmation dialogs

The flow is as follows:

1. The user enters a message in the chat UI.
1. The message is sent to the API route.
1. In your server side route, the language model generates tool calls during the `streamText` call.
1. All tool calls are forwarded to the client.
1. Server-side tools are executed using their `execute` method and their results are forwarded to the client.
1. Client-side tools that should be automatically executed are handled with the `onToolCall` callback.
   You can return the tool result from the callback.
1. Client-side tool that require user interactions can be displayed in the UI.
   The tool calls and results are available in the `toolInvocations` property of the last assistant message.
1. When the user interaction is done, `addToolResult` can be used to add the tool result to the chat.
1. When there are tool calls in the last assistant message and all tool results are available, the client sends the updated messages back to the server.
   This triggers another iteration of this flow.

The tool call and tool executions are integrated into the assistant message as `toolInvocations`.
A tool invocation is at first a tool call, and then it becomes a tool result when the tool is executed.
The tool result contains all information about the tool call as well as the result of the tool execution.

<Note>
  In order to automatically send another request to the server when all tool
  calls are server-side, you need to set
  [`maxSteps`](/docs/reference/ai-sdk-ui/use-chat#max-steps) to a value greater
  than 1 in the `useChat` options. It is disabled by default for backward
  compatibility.
</Note>

## Example

In this example, we'll use three tools:

- `getWeatherInformation`: An automatically executed server-side tool that returns the weather in a given city.
- `askForConfirmation`: A user-interaction client-side tool that asks the user for confirmation.
- `getLocation`: An automatically executed client-side tool that returns a random city.

### API route

```tsx filename='app/api/chat/route.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      // server-side tool with execute function:
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        parameters: z.object({ city: z.string() }),
        execute: async ({}: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
      // client-side tool that starts user interaction:
      askForConfirmation: {
        description: 'Ask the user for confirmation.',
        parameters: z.object({
          message: z.string().describe('The message to ask for confirmation.'),
        }),
      },
      // client-side tool that is automatically executed on the client:
      getLocation: {
        description:
          'Get the user location. Always ask for confirmation before using this tool.',
        parameters: z.object({}),
      },
    },
  });

  return result.toDataStreamResponse();
}
```

### Client-side page

The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming.
Tool invocations are displayed in the chat UI.

There are three things worth mentioning:

1. The [`onToolCall`](/docs/reference/ai-sdk-ui/use-chat#on-tool-call) callback is used to handle client-side tools that should be automatically executed.
   In this example, the `getLocation` tool is a client-side tool that returns a random city.

2. The `toolInvocations` property of the last assistant message contains all tool calls and results.
   The client-side tool `askForConfirmation` is displayed in the UI.
   It asks the user for confirmation and displays the result once the user confirms or denies the execution.
   The result is added to the chat using `addToolResult`.

3. The [`maxSteps`](/docs/reference/ai-sdk-ui/use-chat#max-steps) option is set to 5.
   This enables several tool use iterations between the client and the server.

```tsx filename='app/page.tsx' highlight="9,12,31"
'use client';

import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, addToolResult } =
    useChat({
      maxSteps: 5,

      // run client-side tools that are automatically executed:
      async onToolCall({ toolCall }) {
        if (toolCall.toolName === 'getLocation') {
          const cities = [
            'New York',
            'Los Angeles',
            'Chicago',
            'San Francisco',
          ];
          return cities[Math.floor(Math.random() * cities.length)];
        }
      },
    });

  return (
    <>
      {messages?.map((m: Message) => (
        <div key={m.id}>
          <strong>{m.role}:</strong>
          {m.content}
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            const toolCallId = toolInvocation.toolCallId;
            const addResult = (result: string) =>
              addToolResult({ toolCallId, result });

            // render confirmation tool (client-side tool with user interaction)
            if (toolInvocation.toolName === 'askForConfirmation') {
              return (
                <div key={toolCallId}>
                  {toolInvocation.args.message}
                  <div>
                    {'result' in toolInvocation ? (
                      <b>{toolInvocation.result}</b>
                    ) : (
                      <>
                        <button onClick={() => addResult('Yes')}>Yes</button>
                        <button onClick={() => addResult('No')}>No</button>
                      </>
                    )}
                  </div>
                </div>
              );
            }

            // other tools:
            return 'result' in toolInvocation ? (
              <div key={toolCallId}>
                Tool call {`${toolInvocation.toolName}: `}
                {toolInvocation.result}
              </div>
            ) : (
              <div key={toolCallId}>Calling {toolInvocation.toolName}...</div>
            );
          })}
          <br />
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </>
  );
}
```

## Tool call streaming

<Note type="warning">This feature is experimental.</Note>

You can stream tool calls while they are being generated by enabling the
`experimental_toolCallStreaming` option in `streamText`.

```tsx filename='app/api/chat/route.ts' highlight="5"
export async function POST(req: Request) {
  // ...

  const result = streamText({
    experimental_toolCallStreaming: true,
    // ...
  });

  return result.toDataStreamResponse();
}
```

When the flag is enabled, partial tool calls will be streamed as part of the data stream.
They are available through the `useChat` hook.
The `toolInvocations` property of assistant messages will also contain partial tool calls.
You can use the `state` property of the tool invocation to render the correct UI.

```tsx filename='app/page.tsx' highlight="9,10"
export default function Chat() {
  // ...
  return (
    <>
      {messages?.map((m: Message) => (
        <div key={m.id}>
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            switch (toolInvocation.state) {
              case 'partial-call':
                return <>render partial tool call</>;
              case 'call':
                return <>render full tool call</>;
              case 'result':
                return <>render tool result</>;
            }
          })}
        </div>
      ))}
    </>
  );
}
```

## Server-side Multi-Step Calls

You can also use multi-step calls on the server-side with `streamText`.
This works when all invoked tools have an `execute` function on the server side.

```tsx filename='app/api/chat/route.ts' highlight="15-21,24"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        parameters: z.object({ city: z.string() }),
        // tool has execute function:
        execute: async ({}: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
    },
    maxSteps: 5,
  });

  return result.toDataStreamResponse();
}
```

---
title: Generative User Interfaces
description: Learn how to build Generative UI with AI SDK UI.
---

# Generative User Interfaces

Generative user interfaces (generative UI) is the process of allowing a large language model (LLM) to go beyond text and "generate UI". This creates a more engaging and AI-native experience for users.

<WeatherSearch />

At the core of generative UI are [ tools ](/docs/ai-sdk-core/tools-and-tool-calling), which are functions you provide to the model to perform specialized tasks like getting the weather in a location. The model can decide when and how to use these tools based on the context of the conversation.

Generative UI is the process of connecting the results of a tool call to a React component. Here's how it works:

1. You provide the model with a prompt or conversation history, along with a set of tools.
2. Based on the context, the model may decide to call a tool.
3. If a tool is called, it will execute and return data.
4. This data can then be passed to a React component for rendering.

By passing the tool results to React components, you can create a generative UI experience that's more engaging and adaptive to your needs.

## Build a Generative UI Chat Interface

Let's create a chat interface that handles text-based conversations and incorporates dynamic UI elements based on model responses.

### Basic Chat Implementation

Start with a basic chat implementation using the `useChat` hook:

```tsx filename="app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

To handle the chat requests and model responses, set up an API route:

```ts filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(request: Request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'You are a friendly assistant!',
    messages,
    maxSteps: 5,
  });

  return result.toDataStreamResponse();
}
```

This API route uses the `streamText` function to process chat messages and stream the model's responses back to the client.

### Create a Tool

Before enhancing your chat interface with dynamic UI elements, you need to create a tool and corresponding React component. A tool will allow the model to perform a specific action, such as fetching weather information.

Create a new file called `ai/tools.ts` with the following content:

```ts filename="ai/tools.ts"
import { tool as createTool } from 'ai';
import { z } from 'zod';

export const weatherTool = createTool({
  description: 'Display the weather for a location',
  parameters: z.object({
    location: z.string(),
  }),
  execute: async function ({ location }) {
    await new Promise(resolve => setTimeout(resolve, 2000));
    return { weather: 'Sunny', temperature: 75, location };
  },
});

export const tools = {
  displayWeather: weatherTool,
};
```

In this file, you've created a tool called `weatherTool`. This tool simulates fetching weather information for a given location. This tool will return simulated data after a 2-second delay. In a real-world application, you would replace this simulation with an actual API call to a weather service.

### Update the API Route

Update the API route to include the tool you've defined:

```ts filename="app/api/chat/route.ts" highlight="3,13"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { tools } from '@/ai/tools';

export async function POST(request: Request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'You are a friendly assistant!',
    messages,
    maxSteps: 5,
    tools,
  });

  return result.toDataStreamResponse();
}
```

Now that you've defined the tool and added it to your `streamText` call, let's build a React component to display the weather information it returns.

### Create UI Components

Create a new file called `components/weather.tsx`:

```tsx filename="components/weather.tsx"
type WeatherProps = {
  temperature: number;
  weather: string;
  location: string;
};

export const Weather = ({ temperature, weather, location }: WeatherProps) => {
  return (
    <div>
      <h2>Current Weather for {location}</h2>
      <p>Condition: {weather}</p>
      <p>Temperature: {temperature}°C</p>
    </div>
  );
};
```

This component will display the weather information for a given location. It takes three props: `temperature`, `weather`, and `location` (exactly what the `weatherTool` returns).

### Render the Weather Component

Now that you have your tool and corresponding React component, let's integrate them into your chat interface. You'll render the Weather component when the model calls the weather tool.

To check if the model has called a tool, you can use the `toolInvocations` property of the message object. This property contains information about any tools that were invoked in that generation including `toolCallId`, `toolName`, `args`, `toolState`, and `result`.

Update your `page.tsx` file:

```tsx filename="app/page.tsx" highlight="4,16-39"
'use client';

import { useChat } from 'ai/react';
import { Weather } from '@/components/weather';

export default function Page() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role === 'user' ? 'User: ' : 'AI: '}</div>
          <div>{message.content}</div>

          <div>
            {message.toolInvocations?.map(toolInvocation => {
              const { toolName, toolCallId, state } = toolInvocation;

              if (state === 'result') {
                if (toolName === 'displayWeather') {
                  const { result } = toolInvocation;
                  return (
                    <div key={toolCallId}>
                      <Weather {...result} />
                    </div>
                  );
                }
              } else {
                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <div>Loading weather...</div>
                    ) : null}
                  </div>
                );
              }
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

In this updated code snippet, you:

1. Check if the message has `toolInvocations`.
2. Check if the tool invocation state is 'result'.
3. If it's a result and the tool name is 'displayWeather', render the Weather component.
4. If the tool invocation state is not 'result', show a loading message.

This approach allows you to dynamically render UI components based on the model's responses, creating a more interactive and context-aware chat experience.

## Expanding Your Generative UI Application

You can enhance your chat application by adding more tools and components, creating a richer and more versatile user experience. Here's how you can expand your application:

### Adding More Tools

To add more tools, simply define them in your `ai/tools.ts` file:

```ts
// Add a new stock tool
export const stockTool = createTool({
  description: 'Get price for a stock',
  parameters: z.object({
    symbol: z.string(),
  }),
  execute: async function ({ symbol }) {
    // Simulated API call
    await new Promise(resolve => setTimeout(resolve, 2000));
    return { symbol, price: 100 };
  },
});

// Update the tools object
export const tools = {
  displayWeather: weatherTool,
  getStockPrice: stockTool,
};
```

Now, create a new file called `components/stock.tsx`:

```tsx
type StockProps = {
  price: number;
  symbol: string;
};

export const Stock = ({ price, symbol }: StockProps) => {
  return (
    <div>
      <h2>Stock Information</h2>
      <p>Symbol: {symbol}</p>
      <p>Price: ${price}</p>
    </div>
  );
};
```

Finally, update your `page.tsx` file to include the new Stock component:

```tsx
'use client';

import { useChat } from 'ai/react';
import { Weather } from '@/components/weather';
import { Stock } from '@/components/stock';

export default function Page() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>

          <div>
            {message.toolInvocations?.map(toolInvocation => {
              const { toolName, toolCallId, state } = toolInvocation;

              if (state === 'result') {
                if (toolName === 'displayWeather') {
                  const { result } = toolInvocation;
                  return (
                    <div key={toolCallId}>
                      <Weather {...result} />
                    </div>
                  );
                } else if (toolName === 'getStockPrice') {
                  const { result } = toolInvocation;
                  return <Stock {...result} />;
                }
              } else {
                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <div>Loading weather...</div>
                    ) : toolName === 'getStockPrice' ? (
                      <div>Loading stock price...</div>
                    ) : (
                      <div>Loading...</div>
                    )}
                  </div>
                );
              }
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

By following this pattern, you can continue to add more tools and components, expanding the capabilities of your Generative UI application.

---
title: Completion
description: Learn how to use the useCompletion hook.
---

# Completion

The `useCompletion` hook allows you to create a user interface to handle text completions in your application. It enables the streaming of text completions from your AI provider, manages the state for chat input, and updates the UI automatically as new messages are received.

In this guide, you will learn how to use the `useCompletion` hook in your application to generate text completions and stream them in real-time to your users.

## Example

```tsx filename='app/page.tsx'
'use client';

import { useCompletion } from 'ai/react';

export default function Page() {
  const { completion, input, handleInputChange, handleSubmit } = useCompletion({
    api: '/api/completion',
  });

  return (
    <form onSubmit={handleSubmit}>
      <input
        name="prompt"
        value={input}
        onChange={handleInputChange}
        id="input"
      />
      <button type="submit">Submit</button>
      <div>{completion}</div>
    </form>
  );
}
```

```ts filename='app/api/completion/route.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = streamText({
    model: openai('gpt-3.5-turbo'),
    prompt,
  });

  return result.toDataStreamResponse();
}
```

In the `Page` component, the `useCompletion` hook will request to your AI provider endpoint whenever the user submits a message. The completion is then streamed back in real-time and displayed in the UI.

This enables a seamless text completion experience where the user can see the AI response as soon as it is available, without having to wait for the entire response to be received.

## Customized UI

`useCompletion` also provides ways to manage the prompt via code, show loading and error states, and update messages without being triggered by user interactions.

### Loading and error states

To show a loading spinner while the chatbot is processing the user's message, you can use the `isLoading` state returned by the `useCompletion` hook:

```tsx
const { isLoading, ... } = useCompletion()

return(
  <>
    {isLoading ? <Spinner /> : null}
  </>
)
```

Similarly, the `error` state reflects the error object thrown during the fetch request. It can be used to display an error message, or show a toast notification:

```tsx
const { error, ... } = useCompletion()

useEffect(() => {
  if (error) {
    toast.error(error.message)
  }
}, [error])

// Or display the error message in the UI:
return (
  <>
    {error ? <div>{error.message}</div> : null}
  </>
)
```

### Controlled input

In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.

The following example demonstrates how to use more granular APIs like `setInput` with your custom input and submit button components:

```tsx
const { input, setInput } = useCompletion();

return (
  <>
    <MyCustomInput value={input} onChange={value => setInput(value)} />
  </>
);
```

### Cancelation

It's also a common use case to abort the response message while it's still streaming back from the AI provider. You can do this by calling the `stop` function returned by the `useCompletion` hook.

```tsx
const { stop, isLoading, ... } = useCompletion()

return (
  <>
    <button onClick={stop} disabled={!isLoading}>Stop</button>
  </>
)
```

When the user clicks the "Stop" button, the fetch request will be aborted. This avoids consuming unnecessary resources and improves the UX of your application.

### Throttling UI Updates

<Note>This feature is currently only available for React.</Note>

By default, the `useCompletion` hook will trigger a render every time a new chunk is received.
You can throttle the UI updates with the `experimental_throttle` option.

```tsx filename="page.tsx" highlight="2-3"
const { completion, ... } = useCompletion({
  // Throttle the completion and data updates to 50ms:
  experimental_throttle: 50
})
```

## Event Callbacks

`useCompletion` also provides optional event callbacks that you can use to handle different stages of the chatbot lifecycle. These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx
const { ... } = useCompletion({
  onResponse: (response: Response) => {
    console.log('Received response from server:', response)
  },
  onFinish: (message: Message) => {
    console.log('Finished streaming message:', message)
  },
  onError: (error: Error) => {
    console.error('An error occurred:', error)
  },
})
```

It's worth noting that you can abort the processing by throwing an error in the `onResponse` callback. This will trigger the `onError` callback and stop the message from being appended to the chat UI. This can be useful for handling unexpected responses from the AI provider.

## Configure Request Options

By default, the `useCompletion` hook sends a HTTP POST request to the `/api/completion` endpoint with the prompt as part of the request body. You can customize the request by passing additional options to the `useCompletion` hook:

```tsx
const { messages, input, handleInputChange, handleSubmit } = useCompletion({
  api: '/api/custom-completion',
  headers: {
    Authorization: 'your_token',
  },
  body: {
    user_id: '123',
  },
  credentials: 'same-origin',
});
```

In this example, the `useCompletion` hook sends a POST request to the `/api/completion` endpoint with the specified headers, additional body fields, and credentials for that fetch request. On your server side, you can handle the request with these additional information.

---
title: Object Generation
description: Learn how to use the useObject hook.
---

# Object Generation

<Note>`useObject` is an experimental feature and only available in React.</Note>

The [`useObject`](/docs/reference/ai-sdk-ui/use-object) hook allows you to create interfaces that represent a structured JSON object that is being streamed.

In this guide, you will learn how to use the `useObject` hook in your application to generate UIs for structured data on the fly.

## Example

The example shows a small notfications demo app that generates fake notifications in real-time.

### Schema

It is helpful to set up the schema in a separate file that is imported on both the client and server.

```ts filename='app/api/notifications/schema.ts'
import { z } from 'zod';

// define a schema for the notifications
export const notificationSchema = z.object({
  notifications: z.array(
    z.object({
      name: z.string().describe('Name of a fictional person.'),
      message: z.string().describe('Message. Do not use emojis or links.'),
    }),
  ),
});
```

### Client

The client uses [`useObject`](/docs/reference/ai-sdk-ui/use-object) to stream the object generation process.

The results are partial and are displayed as they are received.
Please note the code for handling `undefined` values in the JSX.

```tsx filename='app/page.tsx'
'use client';

import { experimental_useObject as useObject } from 'ai/react';
import { notificationSchema } from './api/notifications/schema';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

### Server

On the server, we use [`streamObject`](/docs/reference/ai-sdk-core/stream-object) to stream the object generation process.

```typescript filename='app/api/notifications/route.ts'
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { notificationSchema } from './schema';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamObject({
    model: openai('gpt-4-turbo'),
    schema: notificationSchema,
    prompt:
      `Generate 3 notifications for a messages app in this context:` + context,
  });

  return result.toTextStreamResponse();
}
```

## Customized UI

`useObject` also provides ways to show loading and error states:

### Loading State

The `isLoading` state returned by the `useObject` hook can be used for several
purposes

- To show a loading spinner while the object is generated.
- To show a "Stop" button to abort the current message.
- To disable the submit button.

```tsx filename='app/page.tsx' highlight="6,13-20,24"
'use client';

import { useObject } from 'ai/react';

export default function Page() {
  const { isLoading, stop, object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      {isLoading && (
        <div>
          <Spinner />
          <button type="button" onClick={() => stop()}>
            Stop
          </button>
        </div>
      )}

      <button
        onClick={() => submit('Messages during finals week.')}
        disabled={isLoading}
      >
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

### Error State

Similarly, the `error` state reflects the error object thrown during the fetch request.
It can be used to display an error message, or to disable the submit button:

<Note>
  We recommend showing a generic error message to the user, such as "Something
  went wrong." This is a good practice to avoid leaking information from the
  server.
</Note>

```tsx file="app/page.tsx" highlight="6,13"
'use client';

import { useObject } from 'ai/react';

export default function Page() {
  const { error, object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      {error && <div>An error occurred.</div>}

      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

## Event Callbacks

`useObject` provides optional event callbacks that you can use to handle life-cycle events.

- `onFinish`: Called when the object generation is completed.
- `onError`: Called when an error occurs during the fetch request.

These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx filename='app/page.tsx' highlight="10-20"
'use client';

import { experimental_useObject as useObject } from 'ai/react';
import { notificationSchema } from './api/notifications/schema';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
    onFinish({ object, error }) {
      // typed object, undefined if schema validation fails:
      console.log('Object generation completed:', object);

      // error, undefined if schema validation succeeds:
      console.log('Schema validation error:', error);
    },
    onError(error) {
      // error during fetch request:
      console.error('An error occurred:', error);
    },
  });

  return (
    <div>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

## Configure Request Options

You can configure the API endpoint and optional headers using the `api` and `headers` settings.

```tsx highlight="2-5"
const { submit, object } = useObject({
  api: '/api/use-object',
  headers: {
    'X-Custom-Header': 'CustomValue',
  },
  schema: yourSchema,
});
```

---
title: OpenAI Assistants
description: Learn how to use the useAssistant hook.
---

# OpenAI Assistants

The `useAssistant` hook allows you to handle the client state when interacting with an OpenAI compatible assistant API.
This hook is useful when you want to integrate assistant capabilities into your application,
with the UI updated automatically as the assistant is streaming its execution.

The `useAssistant` hook is supported in `ai/react`, `ai/svelte`, and `ai/vue`.

## Example

```tsx filename='app/page.tsx'
'use client';

import { Message, useAssistant } from 'ai/react';

export default function Chat() {
  const { status, messages, input, submitMessage, handleInputChange } =
    useAssistant({ api: '/api/assistant' });

  return (
    <div>
      {messages.map((m: Message) => (
        <div key={m.id}>
          <strong>{`${m.role}: `}</strong>
          {m.role !== 'data' && m.content}
          {m.role === 'data' && (
            <>
              {(m.data as any).description}
              <br />
              <pre className={'bg-gray-200'}>
                {JSON.stringify(m.data, null, 2)}
              </pre>
            </>
          )}
        </div>
      ))}

      {status === 'in_progress' && <div />}

      <form onSubmit={submitMessage}>
        <input
          disabled={status !== 'awaiting_message'}
          value={input}
          placeholder="What is the temperature in the living room?"
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

```tsx filename='app/api/assistant/route.ts'
import { AssistantResponse } from 'ai';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || '',
});

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  // Parse the request body
  const input: {
    threadId: string | null;
    message: string;
  } = await req.json();

  // Create a thread if needed
  const threadId = input.threadId ?? (await openai.beta.threads.create({})).id;

  // Add a message to the thread
  const createdMessage = await openai.beta.threads.messages.create(threadId, {
    role: 'user',
    content: input.message,
  });

  return AssistantResponse(
    { threadId, messageId: createdMessage.id },
    async ({ forwardStream, sendDataMessage }) => {
      // Run the assistant on the thread
      const runStream = openai.beta.threads.runs.stream(threadId, {
        assistant_id:
          process.env.ASSISTANT_ID ??
          (() => {
            throw new Error('ASSISTANT_ID is not set');
          })(),
      });

      // forward run status would stream message deltas
      let runResult = await forwardStream(runStream);

      // status can be: queued, in_progress, requires_action, cancelling, cancelled, failed, completed, or expired
      while (
        runResult?.status === 'requires_action' &&
        runResult.required_action?.type === 'submit_tool_outputs'
      ) {
        const tool_outputs =
          runResult.required_action.submit_tool_outputs.tool_calls.map(
            (toolCall: any) => {
              const parameters = JSON.parse(toolCall.function.arguments);

              switch (toolCall.function.name) {
                // configure your tool calls here

                default:
                  throw new Error(
                    `Unknown tool call function: ${toolCall.function.name}`,
                  );
              }
            },
          );

        runResult = await forwardStream(
          openai.beta.threads.runs.submitToolOutputsStream(
            threadId,
            runResult.id,
            { tool_outputs },
          ),
        );
      }
    },
  );
}
```

## Customized UI

`useAssistant` also provides ways to manage the chat message and input states via code and show loading and error states.

### Loading and error states

To show a loading spinner while the assistant is running the thread, you can use the `status` state returned by the `useAssistant` hook:

```tsx
const { status, ... } = useAssistant()

return(
  <>
    {status === "in_progress" ? <Spinner /> : null}
  </>
)
```

Similarly, the `error` state reflects the error object thrown during the fetch request. It can be used to display an error message, or show a toast notification:

```tsx
const { error, ... } = useAssistant()

useEffect(() => {
  if (error) {
    toast.error(error.message)
  }
}, [error])

// Or display the error message in the UI:
return (
  <>
    {error ? <div>{error.message}</div> : null}
  </>
)
```

### Controlled input

In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.

The following example demonstrates how to use more granular APIs like `append` with your custom input and submit button components:

```tsx
const { append } = useAssistant();

return (
  <>
    <MySubmitButton
      onClick={() => {
        // Send a new message to the AI provider
        append({
          role: 'user',
          content: input,
        });
      }}
    />
  </>
);
```

## Configure Request Options

By default, the `useAssistant` hook sends a HTTP POST request to the `/api/assistant` endpoint with the prompt as part of the request body. You can customize the request by passing additional options to the `useAssistant` hook:

```tsx
const { messages, input, handleInputChange, handleSubmit } = useAssistant({
  api: '/api/custom-completion',
  headers: {
    Authorization: 'your_token',
  },
  body: {
    user_id: '123',
  },
  credentials: 'same-origin',
});
```

In this example, the `useAssistant` hook sends a POST request to the `/api/custom-completion` endpoint with the specified headers, additional body fields, and credentials for that fetch request. On your server side, you can handle the request with these additional information.

---
title: Storing Messages
description: Welcome to the AI SDK documentation!
---

# Storing Messages

The ability to store message history is essential for chatbot use cases.
The AI SDK simplifies the process of storing chat history through the `onFinish` callback of the `streamText` function.

`onFinish` is called after the model's response and all tool executions have completed.
It provides the final text, tool calls, tool results, and usage information,
making it an ideal place to e.g. store the chat history in a database.

## Implementing Persistent Chat History

To implement persistent chat storage, you can utilize the `onFinish` callback on the `streamText` function.
This callback is triggered upon the completion of the model's response and all tool executions,
making it an ideal place to handle the storage of each interaction.

### API Route Example

```tsx highlight="13-16"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    async onFinish({ text, toolCalls, toolResults, usage, finishReason }) {
      // implement your own storage logic:
      await saveChat({ text, toolCalls, toolResults });
    },
  });

  return result.toDataStreamResponse();
}
```

### Server Action Example

```tsx highlight="10-13"
'use server';

import { openai } from '@ai-sdk/openai';
import { CoreMessage, streamText } from 'ai';

export async function continueConversation(messages: CoreMessage[]) {
  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    async onFinish({ text, toolCalls, toolResults, finishReason, usage }) {
      // implement your own storage logic:
      await saveChat({ text, toolCalls, toolResults });
    },
  });

  return result.toDataStreamResponse();
}
```

---
title: Streaming Data
description: Welcome to the AI SDK documentation!
---

# Streaming Data

It is often useful to send additional data alongside the model's response.
For example, you may want to send status information, the message ids after storing them,
or references to content that the language model is referring to.

The AI SDK provides a `StreamData` helper that allows you to stream additional data to the client
and attach it either to the `Message` or to the `data` object of the `useChat` hook.
The data is streamed as part of the response stream.

## Sending Stream Data

In your server-side route handler, you can use `StreamData` in combination with `streamText`.
You need to:

1. Initialize a `StreamData` object.
2. Append data to it, which can either be message annotations (`appendMessageAnnotation`) or call annotations (`append`) .
3. Close the `StreamData` object.
4. Return the `StreamData` object in the `toDataStreamResponse` call.

Here is an example:

```tsx highlight="7,8,10,11, 16-20,22,23,25,26,31,32"
import { openai } from '@ai-sdk/openai';
import { streamText, StreamData } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  // Create a new StreamData object
  const data = new StreamData();

  // Append to general streamed data
  data.append({ test: 'initialized calls' });

  const result = streamText({
    model: openai('gpt-4-turbo'),
    onFinish() {
      // message annotation:
      data.appendMessageAnnotation({
        id: generateId(), // e.g. id from saved DB record
        other: 'information',
      });

      // call annotation (can be any JSON serializable value)
      data.append('call completed');

      // close the StreamData object
      data.close();
    },
    messages,
  });

  // Respond with the stream and additional StreamData
  return result.toDataStreamResponse({ data });
}
```

<Note>
  You can also send stream data from custom backends, e.g. Python / FastAPI,
  using the [Data Stream
  Protocol](/docs/ai-sdk-ui/stream-protocol#data-stream-protocol).
</Note>

## Stream Data in useChat

The `useChat` hook automatically processes the streamed data and makes it available to you.

### Accessing Data

On the client, you can destructure `data` from the `useChat` hook which stores all `StreamData`
as a `JSONValue[]`.

```tsx
import { useChat } from 'ai/react';

const { data } = useChat();
```

### Accessing Message Annotations

Each message from the `useChat` hook has an optional `annotations` property that contains
the message annotations sent from the server.

Since the shape of the annotations depends on what you send from the server,
you have to destructure them in a type-safe way on the client side.

Here we just show the annotations as a JSON string:

```tsx highlight="9"
import { Message, useChat } from 'ai/react';

const { messages } = useChat();

const result = (
  <>
    {messages?.map((m: Message) => (
      <div key={m.id}>
        {m.annotations && <>{JSON.stringify(m.annotations)}</>}
      </div>
    ))}
  </>
);
```

### Updating and Clearing Data

You can update and clear the `data` object of the `useChat` hook using the `setData` function.

```tsx
const { setData } = useChat();

// clear existing data
setData(undefined);

// set new data
setData([{ test: 'value' }]);

// transform existing data, e.g. adding additional values:
setData(currentData => [...currentData, { test: 'value' }]);
```

#### Example: Clear on Submit

```tsx highlight="18-21"
'use client';

import { Message, useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, data, setData } =
    useChat();

  return (
    <>
      {data && <pre>{JSON.stringify(data, null, 2)}</pre>}

      {messages?.map((m: Message) => (
        <div key={m.id}>{`${m.role}: ${m.content}`}</div>
      ))}

      <form
        onSubmit={e => {
          setData(undefined); // clear stream data
          handleSubmit(e);
        }}
      >
        <input value={input} onChange={handleInputChange} />
      </form>
    </>
  );
}
```

---
title: Error Handling
description: Learn how to handle errors in the AI SDK UI
---

# Error Handling

### Error Helper Object

Each AI SDK UI hook also returns an [error](/docs/reference/ai-sdk-ui/use-chat#error) object that you can use to render the error in your UI.
You can use the error object to show an error message, disable the submit button, or show a retry button.

<Note>
  We recommend showing a generic error message to the user, such as "Something
  went wrong." This is a good practice to avoid leaking information from the
  server.
</Note>

```tsx file="app/page.tsx" highlight="7,17-24,30"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, error, reload } =
    useChat({});

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      {error && (
        <>
          <div>An error occurred.</div>
          <button type="button" onClick={() => reload()}>
            Retry
          </button>
        </>
      )}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          disabled={error != null}
        />
      </form>
    </div>
  );
}
```

#### Alternative: replace last message

Alternatively you can write a custom submit handler that replaces the last message when an error is present.

```tsx file="app/page.tsx" highlight="15-21,33"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const {
    handleInputChange,
    handleSubmit,
    error,
    input,
    messages,
    setMesages,
  } = useChat({});

  function customSubmit(event: React.FormEvent<HTMLFormElement>) {
    if (error != null) {
      setMessages(messages.slice(0, -1)); // remove last message
    }

    handleSubmit(event);
  }

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      {error && <div>An error occurred.</div>}

      <form onSubmit={customSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

### Error Handling Callback

Errors can be processed by passing an [`onError`](/docs/reference/ai-sdk-ui/use-chat#on-error) callback function as an option to the [`useChat`](/docs/reference/ai-sdk-ui/use-chat), [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion) or [`useAssistant`](/docs/reference/ai-sdk-ui/use-assistant) hooks.
The callback function receives an error object as an argument.

```tsx file="app/page.tsx" highlight="8-11"
import { useChat } from 'ai/react';

export default function Page() {
  const {
    /* ... */
  } = useChat({
    // handle error:
    onError: error => {
      console.error(error);
    },
  });
}
```

### Injecting Errors for Testing

You might want to create errors for testing.
You can easily do so by throwing an error in your route handler:

```ts file="app/api/chat/route.ts"
export async function POST(req: Request) {
  throw new Error('This is a test error');
}
```

---
title: Stream Protocols
description: Learn more about the supported stream protocols in the AI SDK.
---

# Stream Protocols

AI SDK UI functions such as `useChat` and `useCompletion` support both text streams and data streams.
The stream protocol defines how the data is streamed to the frontend on top of the HTTP protocol.

This page describes both protocols and how to use them in the backend and frontend.

You can use this information to develop custom backends and frontends for your use case, e.g.,
to provide compatible API endpoints that are implemented in a different language such as Python.

For instance, here's an example using [FastAPI](https://github.com/vercel/ai/tree/main/examples/next-fastapi) as a backend.

## Text Stream Protocol

A text stream contains chunks in plain text, that are streamed to the frontend.
Each chunk is then appended together to form a full text response.

Text streams are supported by `useChat`, `useCompletion`, and `useObject`.
When you use `useChat` or `useCompletion`, you need to enable text streaming
by setting the `streamProtocol` options to `text`.

You can generate text streams with `streamText` in the backend.
When you call `toTextStreamResponse()` on the result object,
a streaming HTTP response is returned.

<Note>
  Text streams only support basic text data. If you need to stream other types
  of data such as tool calls, use data streams.
</Note>

### Text Stream Example

Here is a Next.js example that uses the text stream protocol:

```tsx filename='app/page.tsx' highlight="7"
'use client';

import { useCompletion } from 'ai/react';

export default function Page() {
  const { completion, input, handleInputChange, handleSubmit } = useCompletion({
    streamProtocol: 'text',
  });

  return (
    <form onSubmit={handleSubmit}>
      <input name="prompt" value={input} onChange={handleInputChange} />
      <button type="submit">Submit</button>
      <div>{completion}</div>
    </form>
  );
}
```

```ts filename='app/api/completion/route.ts' highlight="15"
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    prompt,
  });

  return result.toTextStreamResponse();
}
```

## Data Stream Protocol

A data stream follows a special protocol that the AI SDK provides to send information to the frontend.

Each stream part has the format `TYPE_ID:CONTENT_JSON\n`.

<MDXImage
  srcLight="/images/data-stream-protocol.png"
  srcDark="/images/data-stream-protocol.png"
  width={3200}
  height={1712}
/>

<Note>
  When you provide data streams from a custom backend, you need to set the
  `x-vercel-ai-data-stream` header to `v1`.
</Note>

The following stream parts are currently supported:

### Text Part

The text parts are appended to the message as they are received.

Format: `0:string\n`

Example: `0:"example"\n`

<MDXImage
  srcLight="/images/text-part.png"
  srcDark="/images/text-part.png"
  width={3200}
  height={1148}
/>

### Data Part

The data parts are parsed as JSON and appended to the message as they are received. The data part is available through `data`.

Format: `2:Array<JSONValue>\n`

Example: `2:[{"key":"object1"},{"anotherKey":"object2"}]\n`

<MDXImage
  srcLight="/images/data-part.png"
  srcDark="/images/data-part.png"
  width={3200}
  height={1148}
/>

### Message Annotation Part

The message annotation parts are appended to the message as they are received.

Format: `8:JSONValue\n`

Example: `8:{"id":"message-123","other":"annotation"}\n`

### Error Part

The error parts are appended to the message as they are received.

Format: `3:string\n`

Example: `3:"error message"\n`

<MDXImage
  srcLight="/images/error-part.png"
  srcDark="/images/error-part.png"
  width={3200}
  height={1148}
/>

### Tool Call Streaming Start Part

A part indicating the start of a streaming tool call. This part needs to be sent before any tool call delta for that tool call. Tool call streaming is optional, you can use tool call and tool result parts without it.

Format: `b:{toolCallId:string; toolName:string}\n`

Example: `b:{"toolCallId":"call-456","toolName":"streaming-tool"}\n`

<MDXImage
  srcLight="/images/tool-call-streaming-start-part.png"
  srcDark="/images/tool-call-streaming-start-part.png"
  width={3200}
  height={1148}
/>

### Tool Call Delta Part

A part representing a delta update for a streaming tool call.

Format: `c:{toolCallId:string; argsTextDelta:string}\n`

Example: `c:{"toolCallId":"call-456","argsTextDelta":"partial arg"}\n`

<MDXImage
  srcLight="/images/tool-call-delta-part.png"
  srcDark="/images/tool-call-delta-part.png"
  width={3200}
  height={1148}
/>

### Tool Call Part

A part representing a tool call. When there are streamed tool calls, the tool call part needs to come after the tool call streaming is finished.

Format: `9:{toolCallId:string; toolName:string; args:object}\n`

Example: `9:{"toolCallId":"call-123","toolName":"my-tool","args":{"some":"argument"}}\n`

<MDXImage
  srcLight="/images/tool-call-part.png"
  srcDark="/images/tool-call-part.png"
  width={3200}
  height={1148}
/>

### Tool Result Part

A part representing a tool result. The result part needs to be sent after the tool call part for that tool call.

Format: `a:{toolCallId:string; result:object}\n`

Example: `a:{"toolCallId":"call-123","result":"tool output"}\n`

<MDXImage
  srcLight="/images/tool-result-part.png"
  srcDark="/images/tool-result-part.png"
  width={3200}
  height={1148}
/>

### Finish Step Part

A part indicating that a step (i.e., one LLM API call in the backend) has been completed.

This part is necessary to correctly process multiple stitched assistant calls, e.g. when calling tools in the backend, and using steps in `useChat` at the same time.

It includes the following metadata:

- [`FinishReason`](/docs/reference/ai-sdk-ui/use-chat#on-finish-finish-reason)
- [`Usage`](/docs/reference/ai-sdk-ui/use-chat#on-finish-usage) for that step.
- `isContinued` to indicate if the step text will be continued in the next step.

The finish step part needs to come at the end of a step.

Format: `e:{finishReason:'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown';usage:{promptTokens:number; completionTokens:number;},isContinued:boolean}\n`

Example: `e:{"finishReason":"stop","usage":{"promptTokens":10,"completionTokens":20},"isContinued":false}\n`

### Finish Message Part

A part indicating the completion of a message with additional metadata, such as [`FinishReason`](/docs/reference/ai-sdk-ui/use-chat#on-finish-finish-reason) and [`Usage`](/docs/reference/ai-sdk-ui/use-chat#on-finish-usage). This part needs to be the last part in the stream.

Format: `d:{finishReason:'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown';usage:{promptTokens:number; completionTokens:number;}}\n`

Example: `d:{"finishReason":"stop","usage":{"promptTokens":10,"completionTokens":20}}\n`

<MDXImage
  srcLight="/images/finish-message-part.png"
  srcDark="/images/finish-message-part.png"
  width={3200}
  height={1148}
/>

The data stream protocol is supported
by `useChat` and `useCompletion` on the frontend and used by default.
`useCompletion` only supports the `text` and `data` stream parts.

On the backend, you can use `toDataStreamResponse()` from the `streamText` result object to return a streaming HTTP response.

### Data Stream Example

Here is a Next.js example that uses the data stream protocol:

```tsx filename='app/page.tsx' highlight="7"
'use client';

import { useCompletion } from 'ai/react';

export default function Page() {
  const { completion, input, handleInputChange, handleSubmit } = useCompletion({
    streamProtocol: 'data', // optional, this is the default
  });

  return (
    <form onSubmit={handleSubmit}>
      <input name="prompt" value={input} onChange={handleInputChange} />
      <button type="submit">Submit</button>
      <div>{completion}</div>
    </form>
  );
}
```

```ts filename='app/api/completion/route.ts' highlight="15"
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    prompt,
  });

  return result.toDataStreamResponse();
}
```

---
title: AI SDK UI
description: Learn about the AI SDK UI.
---

# AI SDK UI

<IndexCards
  cards={[
    {
      title: 'Overview',
      description: 'Get an overview about the AI SDK UI.',
      href: '/docs/ai-sdk-ui/overview',
    },
    {
      title: 'Chatbot',
      description: 'Learn how to integrate an interface for a chatbot.',
      href: '/docs/ai-sdk-ui/chatbot',
    },
    {
      title: 'Chatbot with tool calling',
      description:
        'Learn how to integrate an interface for a chatbot with tool calling.',
      href: '/docs/ai-sdk-ui/chatbot-with-tool-calling',
    },
    {
      title: 'Completion',
      description: 'Learn how to integrate an interface for text completion.',
      href: '/docs/ai-sdk-ui/completion',
    },
    {
      title: 'Object Generation',
      description: 'Learn how to integrate an interface for object generation.',
      href: '/docs/ai-sdk-ui/object-generation',
    },
    {
      title: 'OpenAI Assistants',
      description: 'Learn how to integrate an interface for OpenAI Assistants.',
      href: '/docs/ai-sdk-ui/openai-assistants',
    },
    {
      title: 'Storing Messages',
      description: 'Learn how to store messages.',
      href: '/docs/ai-sdk-ui/storing-messages',
    },
    {
      title: 'Streaming Data',
      description: 'Learn how to stream data.',
      href: '/docs/ai-sdk-ui/streaming-data',
    },
    {
      title: 'Error Handling',
      description: 'Learn how to handle errors.',
      href: '/docs/ai-sdk-ui/error-handling',
    },
    {
      title: 'Stream Protocol',
      description:
        'The stream protocol defines how data is sent from the backend to the AI SDK UI frontend.',
      href: '/docs/ai-sdk-ui/stream-protocol',
    },
  ]}
/>

---
title: Overview
description: An overview of AI SDK RSC.
---

# AI SDK RSC

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

<Note>
  The `ai/rsc` package is compatible with frameworks that support React Server
  Components.
</Note>

[React Server Components](https://nextjs.org/docs/app/building-your-application/rendering/server-components) (RSC) allow you to write UI that can be rendered on the server and streamed to the client. RSCs enable [ Server Actions ](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#with-client-components), a new way to call server-side code directly from the client just like any other function with end-to-end type-safety. This combination opens the door to a new way of building AI applications, allowing the large language model (LLM) to generate and stream UI directly from the server to the client.

## AI SDK RSC Functions

AI SDK RSC has various functions designed to help you build AI-native applications with React Server Components. These functions:

1. Provide abstractions for building Generative UI applications.
   - [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui): calls a model and allows it to respond with React Server Components.
   - [`useUIState`](/docs/reference/ai-sdk-rsc/use-ui-state): returns the current UI state and a function to update the UI State (like React's `useState`). UI State is the visual representation of the AI state.
   - [`useAIState`](/docs/reference/ai-sdk-rsc/use-ai-state): returns the current AI state and a function to update the AI State (like React's `useState`). The AI state is intended to contain context and information shared with the AI model, such as system messages, function responses, and other relevant data.
   - [`useActions`](/docs/reference/ai-sdk-rsc/use-actions): provides access to your Server Actions from the client. This is particularly useful for building interfaces that require user interactions with the server.
   - [`createAI`](/docs/reference/ai-sdk-rsc/create-ai): creates a client-server context provider that can be used to wrap parts of your application tree to easily manage both UI and AI states of your application.
2. Make it simple to work with streamable values between the server and client.
   - [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value): creates a stream that sends values from the server to the client. The value can be any serializable data.
   - [`readStreamableValue`](/docs/reference/ai-sdk-rsc/read-streamable-value): reads a streamable value from the client that was originally created using `createStreamableValue`.
   - [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui): creates a stream that sends UI from the server to the client.
   - [`useStreamableValue`](/docs/reference/ai-sdk-rsc/use-streamable-value): accepts a streamable value created using `createStreamableValue` and returns the current value, error, and pending state.

## Templates

Check out the following templates to see AI SDK RSC in action.

<Templates type="generative-ui" />

## API Reference

Please check out the [AI SDK RSC API Reference](/docs/reference/ai-sdk-rsc) for more details on each function.

---
title: Streaming React Components
description: Overview of streaming RSCs
---

import { UIPreviewCard, Card } from '@/components/home/card';
import { EventPlanning } from '@/components/home/event-planning';
import { Searching } from '@/components/home/searching';
import { Weather } from '@/components/home/weather';

# Streaming React Components

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

The RSC API allows you to stream React components from the server to the client with the [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function. This is useful when you want to go beyond raw text and stream components to the client in real-time.

Similar to [ AI SDK Core ](/docs/ai-sdk-core/overview) APIs (like [ `streamText` ](/docs/reference/ai-sdk-core/stream-text) and [ `streamObject` ](/docs/reference/ai-sdk-core/stream-object)), `streamUI` provides a single function to call a model and allow it to respond with React Server Components.
It supports the same model interfaces as AI SDK Core APIs.

### Concepts

To give the model the ability to respond to a user's prompt with a React component, you can leverage [tools](/docs/ai-sdk-core/tools-and-tool-calling).

<Note>
  Remember, tools are like programs you can give to the model, and the model can
  decide as and when to use based on the context of the conversation.
</Note>

With the `streamUI` function, **you provide tools that return React components**. With the ability to stream components, the model is akin to a dynamic router that is able to understand the user's intention and display relevant UI.

At a high level, the `streamUI` works like other AI SDK Core functions: you can provide the model with a prompt or some conversation history and, optionally, some tools. If the model decides, based on the context of the conversation, to call a tool, it will generate a tool call. The `streamUI` function will then run the respective tool, returning a React component. If the model doesn't have a relevant tool to use, it will return a text generation, which will be passed to the `text` function, for you to handle (render and return as a React component).

<Note>Remember, the `streamUI` function must return a React component. </Note>

```tsx
const result = await streamUI({
  model: openai('gpt-4o'),
  prompt: 'Get the weather for San Francisco',
  text: ({ content }) => <div>{content}</div>,
  tools: {},
});
```

This example calls the `streamUI` function using OpenAI's `gpt-4o` model, passes a prompt, specifies how the model's plain text response (`content`) should be rendered, and then provides an empty object for tools. Even though this example does not define any tools, it will stream the model's response as a `div` rather than plain text.

### Adding A Tool

Using tools with `streamUI` is similar to how you use tools with `generateText` and `streamText`.
A tool is an object that has:

- `description`: a string telling the model what the tool does and when to use it
- `parameters`: a Zod schema describing what the tool needs in order to run
- `generate`: an asynchronous function that will be run if the model calls the tool. This must return a React component

Let's expand the previous example to add a tool.

```tsx highlight="6-14"
const result = await streamUI({
  model: openai('gpt-4o'),
  prompt: 'Get the weather for San Francisco',
  text: ({ content }) => <div>{content}</div>,
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      parameters: z.object({ location: z.string() }),
      generate: async function* ({ location }) {
        yield <LoadingComponent />;
        const weather = await getWeather(location);
        return <WeatherComponent weather={weather} location={location} />;
      },
    },
  },
});
```

This tool would be run if the user asks for the weather for their location. If the user hasn't specified a location, the model will ask for it before calling the tool. When the model calls the tool, the generate function will initially return a loading component. This component will show until the awaited call to `getWeather` is resolved, at which point, the model will stream the `<WeatherComponent />` to the user.

<Note>
  Note: This example uses a [ generator function
  ](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*)
  (`function*`), which allows you to pause its execution and return a value,
  then resume from where it left off on the next call. This is useful for
  handling data streams, as you can fetch and return data from an asynchronous
  source like an API, then resume the function to fetch the next chunk when
  needed. By yielding values one at a time, generator functions enable efficient
  processing of streaming data without blocking the main thread.
</Note>

## Using `streamUI` with Next.js

Let's see how you can use the example above in a Next.js application.

To use `streamUI` in a Next.js application, you will need two things:

1. A Server Action (where you will call `streamUI`)
2. A page to call the Server Action and render the resulting components

### Step 1: Create a Server Action

<Note>
  Server Actions are server-side functions that you can call directly from the
  frontend. For more info, see [the
  documentation](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#with-client-components).
</Note>

Create a Server Action at `app/actions.tsx` and add the following code:

```tsx filename="app/actions.tsx"
'use server';

import { streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const LoadingComponent = () => (
  <div className="animate-pulse p-4">getting weather...</div>
);

const getWeather = async (location: string) => {
  await new Promise(resolve => setTimeout(resolve, 2000));
  return '82°F️ ☀️';
};

interface WeatherProps {
  location: string;
  weather: string;
}

const WeatherComponent = (props: WeatherProps) => (
  <div className="border border-neutral-200 p-4 rounded-lg max-w-fit">
    The weather in {props.location} is {props.weather}
  </div>
);

export async function streamComponent() {
  const result = await streamUI({
    model: openai('gpt-4o'),
    prompt: 'Get the weather for San Francisco',
    text: ({ content }) => <div>{content}</div>,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        parameters: z.object({
          location: z.string(),
        }),
        generate: async function* ({ location }) {
          yield <LoadingComponent />;
          const weather = await getWeather(location);
          return <WeatherComponent weather={weather} location={location} />;
        },
      },
    },
  });

  return result.value;
}
```

The `getWeather` tool should look familiar as it is identical to the example in the previous section. In order for this tool to work:

1. First define a `LoadingComponent`, which renders a pulsing `div` that will show some loading text.
2. Next, define a `getWeather` function that will timeout for 2 seconds (to simulate fetching the weather externally) before returning the "weather" for a `location`. Note: you could run any asynchronous TypeScript code here.
3. Finally, define a `WeatherComponent` which takes in `location` and `weather` as props, which are then rendered within a `div`.

Your Server Action is an asynchronous function called `streamComponent` that takes no inputs, and returns a `ReactNode`. Within the action, you call the `streamUI` function, specifying the model (`gpt-4o`), the prompt, the component that should be rendered if the model chooses to return text, and finally, your `getWeather` tool. Last but not least, you return the resulting component generated by the model with `result.value`.

To call this Server Action and display the resulting React Component, you will need a page.

### Step 2: Create a Page

Create or update your root page (`app/page.tsx`) with the following code:

```tsx filename="app/page.tsx"
'use client';

import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { streamComponent } from './actions';

export default function Page() {
  const [component, setComponent] = useState<React.ReactNode>();

  return (
    <div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setComponent(await streamComponent());
        }}
      >
        <Button>Stream Component</Button>
      </form>
      <div>{component}</div>
    </div>
  );
}
```

This page is first marked as a client component with the `"use client";` directive given it will be using hooks and interactivity. On the page, you render a form. When that form is submitted, you call the `streamComponent` action created in the previous step (just like any other function). The `streamComponent` action returns a `ReactNode` that you can then render on the page using React state (`setComponent`).

## Going beyond a single prompt

You can now allow the model to respond to your prompt with a React component. However, this example is limited to a static prompt that is set within your Server Action. You could make this example interactive by turning it into a chatbot.

Learn how to stream React components with the Next.js App Router using `streamUI` with this [example](https://sdk.vercel.ai/examples/next-app/interface/route-components).

---
title: Managing Generative UI State
description: Overview of the AI and UI states
---

# Managing Generative UI State

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

State is an essential part of any application. State is particularly important in AI applications as it is passed to large language models (LLMs) on each request to ensure they have the necessary context to produce a great generation. Traditional chatbots are text-based and have a structure that mirrors that of any chat application.

For example, in a chatbot, state is an array of `messages` where each `message` has:

- `id`: a unique identifier
- `role`: who sent the message (user/assistant/system/tool)
- `content`: the content of the message

This state can be rendered in the UI and sent to the model without any modifications.

With Generative UI, the model can now return a React component, rather than a plain text message. The client can render that component without issue, but that state can't be sent back to the model because React components aren't serialisable. So, what can you do?

**The solution is to split the state in two, where one (AI State) becomes a proxy for the other (UI State)**.

One way to understand this concept is through a Lego analogy. Imagine a 10,000 piece Lego model that, once built, cannot be easily transported because it is fragile. By taking the model apart, it can be easily transported, and then rebuilt following the steps outlined in the instructions pamphlet. In this way, the instructions pamphlet is a proxy to the physical structure. Similarly, AI State provides a serialisable (JSON) representation of your UI that can be passed back and forth to the model.

## What is AI and UI State?

The RSC API simplifies how you manage AI State and UI State, providing a robust way to keep them in sync between your database, server and client.

### AI State

AI State refers to the state of your application in a serialisable format that will be used on the server and can be shared with the language model.

For a chat app, the AI State is the conversation history (messages) between the user and the assistant. Components generated by the model would be represented in a JSON format as a tool alongside any necessary props. AI State can also be used to store other values and meta information such as `createdAt` for each message and `chatId` for each conversation. The LLM reads this history so it can generate the next message. This state serves as the source of truth for the current application state.

<Note>
  **Note**: AI state can be accessed/modified from both the server and the
  client.
</Note>

### UI State

UI State refers to the state of your application that is rendered on the client. It is a fully client-side state (similar to `useState`) that can store anything from Javascript values to React elements. UI state is a list of actual UI elements that are rendered on the client.

<Note>**Note**: UI State can only be accessed client-side.</Note>

## Using AI / UI State

### Creating the AI Context

AI SDK RSC simplifies managing AI and UI state across your application by providing several hooks. These hooks are powered by [ React context ](https://react.dev/reference/react/hooks#context-hooks) under the hood.

Notably, this means you do not have to pass the message history to the server explicitly for each request. You also can access and update your application state in any child component of the context provider. As you begin building [multistep generative interfaces](/docs/ai-sdk-rsc/multistep-interfaces), this will be particularly helpful.

To use `ai/rsc` to manage AI and UI State in your application, you can create a React context using [`createAI`](/docs/reference/ai-sdk-rsc/create-ai):

```tsx filename='app/actions.tsx'
// Define the AI state and UI state types
export type ServerMessage = {
  role: 'user' | 'assistant';
  content: string;
};

export type ClientMessage = {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
};

export const sendMessage = async (input: string): Promise<ClientMessage> => {
  "use server"
  ...
}
```

```tsx filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ClientMessage, ServerMessage, sendMessage } from './actions';

export type AIState = ServerMessage[];
export type UIState = ClientMessage[];

// Create the AI provider with the initial states and allowed actions
export const AI = createAI<AIState, UIState>({
  initialAIState: [],
  initialUIState: [],
  actions: {
    sendMessage,
  },
});
```

<Note>You must pass Server Actions to the `actions` object.</Note>

In this example, you define types for AI State and UI State, respectively.

Next, wrap your application with your newly created context. With that, you can get and set AI and UI State across your entire application.

```tsx filename='app/layout.tsx'
import { type ReactNode } from 'react';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  return (
    <AI>
      <html lang="en">
        <body>{children}</body>
      </html>
    </AI>
  );
}
```

## Reading UI State in Client

The UI state can be accessed in Client Components using the [`useUIState`](/docs/reference/ai-sdk-rsc/use-ui-state) hook provided by the RSC API. The hook returns the current UI state and a function to update the UI state like React's `useState`.

```tsx filename='app/page.tsx'
'use client';

import { useUIState } from 'ai/rsc';

export default function Page() {
  const [messages, setMessages] = useUIState();

  return (
    <ul>
      {messages.map(message => (
        <li key={message.id}>{message.display}</li>
      ))}
    </ul>
  );
}
```

## Reading AI State in Client

The AI state can be accessed in Client Components using the [`useAIState`](/docs/reference/ai-sdk-rsc/use-ai-state) hook provided by the RSC API. The hook returns the current AI state.

```tsx filename='app/page.tsx'
'use client';

import { useAIState } from 'ai/rsc';

export default function Page() {
  const [messages, setMessages] = useAIState();

  return (
    <ul>
      {messages.map(message => (
        <li key={message.id}>{message.content}</li>
      ))}
    </ul>
  );
}
```

## Reading AI State on Server

The AI State can be accessed within any Server Action provided to the `createAI` context using the [`getAIState`](/docs/reference/ai-sdk-rsc/get-ai-state) function. It returns the current AI state as a read-only value:

```tsx filename='app/actions.ts'
import { getAIState } from 'ai/rsc';

export async function sendMessage(message: string) {
  'use server';

  const history = getAIState();

  const response = await generateText({
    model: openai('gpt-3.5-turbo'),
    messages: [...history, { role: 'user', content: message }],
  });

  return response;
}
```

<Note>
  Remember, you can only access state within actions that have been passed to
  the `createAI` context within the `actions` key.
</Note>

## Updating AI State on Server

The AI State can also be updated from within your Server Action with the [`getMutableAIState`](/docs/reference/ai-sdk-rsc/get-mutable-ai-state) function. This function is similar to `getAIState`, but it returns the state with methods to read and update it:

```tsx filename='app/actions.ts'
import { getMutableAIState } from 'ai/rsc';

export async function sendMessage(message: string) {
  'use server';

  const history = getMutableAIState();

  // Update the AI state with the new user message.
  history.update([...history.get(), { role: 'user', content: message }]);

  const response = await generateText({
    model: openai('gpt-3.5-turbo'),
    messages: history.get(),
  });

  // Update the AI state again with the response from the model.
  history.done([...history.get(), { role: 'assistant', content: response }]);

  return response;
}
```

<Note>
  It is important to update the AI State with new responses using `.update()`
  and `.done()` to keep the conversation history in sync.
</Note>

## Calling Server Actions from the Client

To call the `sendMessage` action from the client, you can use the [`useActions`](/docs/reference/ai-sdk-rsc/use-actions) hook. The hook returns all the available Actions that were provided to `createAI`:

```tsx filename='app/page.tsx'
'use client';

import { useActions, useUIState } from 'ai/rsc';
import { AI } from './ai';

export default function Page() {
  const { sendMessage } = useActions<typeof AI>();
  const [messages, setMessages] = useUIState();

  const handleSubmit = async event => {
    event.preventDefault();

    setMessages([
      ...messages,
      { id: Date.now(), role: 'user', display: event.target.message.value },
    ]);

    const response = await sendMessage(event.target.message.value);

    setMessages([
      ...messages,
      { id: Date.now(), role: 'assistant', display: response },
    ]);
  };

  return (
    <>
      <ul>
        {messages.map(message => (
          <li key={message.id}>{message.display}</li>
        ))}
      </ul>
      <form onSubmit={handleSubmit}>
        <input type="text" name="message" />
        <button type="submit">Send</button>
      </form>
    </>
  );
}
```

When the user submits a message, the `sendMessage` action is called with the message content. The response from the action is then added to the UI state, updating the displayed messages.

<Note>
  Important! Don't forget to update the UI State after you call your Server
  Action otherwise the streamed component will not show in the UI.
</Note>

To learn more, check out this [example](/examples/next-app/state-management/ai-ui-states) on managing AI and UI state using `ai/rsc`.

---

Next, you will learn how you can save and restore state with `ai/rsc`.

---
title: Saving and Restoring States
description: Saving and restoring AI and UI states with onGetUIState and onSetAIState
---

# Saving and Restoring States

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

AI SDK RSC provides convenient methods for saving and restoring AI and UI state. This is useful for saving the state of your application after every model generation, and restoring it when the user revisits the generations.

## AI State

### Saving AI state

The AI state can be saved using the [`onSetAIState`](/docs/reference/ai-sdk-rsc/create-ai#on-set-ai-state) callback, which gets called whenever the AI state is updated. In the following example, you save the chat history to a database whenever the generation is marked as done.

```tsx filename='app/ai.ts'
export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  onSetAIState: async ({ state, done }) => {
    'use server';

    if (done) {
      saveChatToDB(state);
    }
  },
});
```

### Restoring AI state

The AI state can be restored using the [`initialAIState`](/docs/reference/ai-sdk-rsc/create-ai#initial-ai-state) prop passed to the context provider created by the [`createAI`](/docs/reference/ai-sdk-rsc/create-ai) function. In the following example, you restore the chat history from a database when the component is mounted.

```tsx file='app/layout.tsx'
import { ReactNode } from 'react';
import { AI } from './ai';

export default async function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  const chat = await loadChatFromDB();

  return (
    <html lang="en">
      <body>
        <AI initialAIState={chat}>{children}</AI>
      </body>
    </html>
  );
}
```

## UI State

### Saving UI state

The UI state cannot be saved directly, since the contents aren't yet serializable. Instead, you can use the AI state as proxy to store details about the UI state and use it to restore the UI state when needed.

### Restoring UI state

The UI state can be restored using the AI state as a proxy. In the following example, you restore the chat history from the AI state when the component is mounted. You use the [`onGetUIState`](/docs/reference/ai-sdk-rsc/create-ai#on-get-ui-state) callback to listen for SSR events and restore the UI state.

```tsx filename='app/ai.ts'
export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  onGetUIState: async () => {
    'use server';

    const historyFromDB: ServerMessage[] = await loadChatFromDB();
    const historyFromApp: ServerMessage[] = getAIState();

    // If the history from the database is different from the
    // history in the app, they're not in sync so return the UIState
    // based on the history from the database

    if (historyFromDB.length !== historyFromApp.length) {
      return historyFromDB.map(({ role, content }) => ({
        id: generateId(),
        role,
        display:
          role === 'function' ? (
            <Component {...JSON.parse(content)} />
          ) : (
            content
          ),
      }));
    }
  },
});
```

To learn more, check out this [example](/examples/next-app/state-management/save-and-restore-states) that persists and restores states in your Next.js application.

---

Next, you will learn how you can use `ai/rsc` functions like `useActions` and `useUIState` to create interactive, multistep interfaces.

---
title: Multistep Interfaces
description: Overview of Building Multistep Interfaces with AI SDK RSC
---

# Designing Multistep Interfaces

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Multistep interfaces refer to user interfaces that require multiple independent steps to be executed in order to complete a specific task.

For example, if you wanted to build a Generative UI chatbot capable of booking flights, it could have three steps:

- Search all flights
- Pick flight
- Check availability

To build this kind of application you will leverage two concepts, **tool composition** and **application context**.

**Tool composition** is the process of combining multiple [tools](/docs/ai-sdk-core/tools-and-tool-calling) to create a new tool. This is a powerful concept that allows you to break down complex tasks into smaller, more manageable steps. In the example above, _"search all flights"_, _"pick flight"_, and _"check availability"_ come together to create a holistic _"book flight"_ tool.

**Application context** refers to the state of the application at any given point in time. This includes the user's input, the output of the language model, and any other relevant information. In the example above, the flight selected in _"pick flight"_ would be used as context necessary to complete the _"check availability"_ task.

## Overview

In order to build a multistep interface with `ai/rsc`, you will need a few things:

- A Server Action that calls and returns the result from the `streamUI` function
- Tool(s) (sub-tasks necessary to complete your overall task)
- React component(s) that should be rendered when the tool is called
- A page to render your chatbot

The general flow that you will follow is:

- User sends a message (calls your Server Action with `useActions`, passing the message as an input)
- Message is appended to the AI State and then passed to the model alongside a number of tools
- Model can decide to call a tool, which will render the `<SomeTool />` component
- Within that component, you can add interactivity by using `useActions` to call the model with your Server Action and `useUIState` to append the model's response (`<SomeOtherTool />`) to the UI State
- And so on...

## Implementation

The turn-by-turn implementation is the simplest form of multistep interfaces. In this implementation, the user and the model take turns during the conversation. For every user input, the model generates a response, and the conversation continues in this turn-by-turn fashion.

In the following example, you specify two tools (`searchFlights` and `lookupFlight`) that the model can use to search for flights and lookup details for a specific flight.

```tsx filename="app/actions.tsx"
import { streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const searchFlights = async (
  source: string,
  destination: string,
  date: string,
) => {
  return [
    {
      id: '1',
      flightNumber: 'AA123',
    },
    {
      id: '2',
      flightNumber: 'AA456',
    },
  ];
};

const lookupFlight = async (flightNumber: string) => {
  return {
    flightNumber: flightNumber,
    departureTime: '10:00 AM',
    arrivalTime: '12:00 PM',
  };
};

export async function submitUserMessage(input: string) {
  'use server';

  const ui = await streamUI({
    model: openai('gpt-4o'),
    system: 'you are a flight booking assistant',
    prompt: input,
    text: async ({ content }) => <div>{content}</div>,
    tools: {
      searchFlights: {
        description: 'search for flights',
        parameters: z.object({
          source: z.string().describe('The origin of the flight'),
          destination: z.string().describe('The destination of the flight'),
          date: z.string().describe('The date of the flight'),
        }),
        generate: async function* ({ source, destination, date }) {
          yield `Searching for flights from ${source} to ${destination} on ${date}...`;
          const results = await searchFlights(source, destination, date);

          return (
            <div>
              {results.map(result => (
                <div key={result.id}>
                  <div>{result.flightNumber}</div>
                </div>
              ))}
            </div>
          );
        },
      },
      lookupFlight: {
        description: 'lookup details for a flight',
        parameters: z.object({
          flightNumber: z.string().describe('The flight number'),
        }),
        generate: async function* ({ flightNumber }) {
          yield `Looking up details for flight ${flightNumber}...`;
          const details = await lookupFlight(flightNumber);

          return (
            <div>
              <div>Flight Number: {details.flightNumber}</div>
              <div>Departure Time: {details.departureTime}</div>
              <div>Arrival Time: {details.arrivalTime}</div>
            </div>
          );
        },
      },
    },
  });

  return ui.value;
}
```

Next, create an AI context that will hold the UI State and AI State.

```ts filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { submitUserMessage } from './actions';

export const AI = createAI<any[], React.ReactNode[]>({
  initialUIState: [],
  initialAIState: [],
  actions: {
    submitUserMessage,
  },
});
```

Next, wrap your application with your newly created context.

```tsx filename='app/layout.tsx'
import { type ReactNode } from 'react';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  return (
    <AI>
      <html lang="en">
        <body>{children}</body>
      </html>
    </AI>
  );
}
```

To call your Server Action, update your root page with the following:

```tsx filename="app/page.tsx"
'use client';

import { useState } from 'react';
import { AI } from './ai';
import { useActions, useUIState } from 'ai/rsc';

export default function Page() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState<typeof AI>();
  const { submitUserMessage } = useActions();

  const handleSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    setInput('');
    setConversation(currentConversation => [
      ...currentConversation,
      <div>{input}</div>,
    ]);
    const message = await submitUserMessage(input);
    setConversation(currentConversation => [...currentConversation, message]);
  };

  return (
    <div>
      <div>
        {conversation.map((message, i) => (
          <div key={i}>{message}</div>
        ))}
      </div>
      <div>
        <form onSubmit={handleSubmit}>
          <input
            type="text"
            value={input}
            onChange={e => setInput(e.target.value)}
          />
          <button>Send Message</button>
        </form>
      </div>
    </div>
  );
}
```

This page pulls in the current UI State using the `useUIState` hook, which is then mapped over and rendered in the UI. To access the Server Action, you use the `useActions` hook which will return all actions that were passed to the `actions` key of the `createAI` function in your `actions.tsx` file. Finally, you call the `submitUserMessage` function like any other TypeScript function. This function returns a React component (`message`) that is then rendered in the UI by updating the UI State with `setConversation`.

In this example, to call the next tool, the user must respond with plain text. **Given you are streaming a React component, you can add a button to trigger the next step in the conversation**.

To add user interaction, you will have to convert the component into a client component and use the `useAction` hook to trigger the next step in the conversation.

```tsx filename="components/flights.tsx"
'use client';

import { useActions, useUIState } from 'ai/rsc';
import { ReactNode } from 'react';

interface FlightsProps {
  flights: { id: string; flightNumber: string }[];
}

export const Flights = ({ flights }: FlightsProps) => {
  const { submitUserMessage } = useActions();
  const [_, setMessages] = useUIState();

  return (
    <div>
      {flights.map(result => (
        <div key={result.id}>
          <div
            onClick={async () => {
              const display = await submitUserMessage(
                `lookupFlight ${result.flightNumber}`,
              );

              setMessages((messages: ReactNode[]) => [...messages, display]);
            }}
          >
            {result.flightNumber}
          </div>
        </div>
      ))}
    </div>
  );
};
```

Now, update your `searchFlights` tool to render the new `<Flights />` component.

```tsx filename="actions.tsx"
...
searchFlights: {
  description: 'search for flights',
  parameters: z.object({
    source: z.string().describe('The origin of the flight'),
    destination: z.string().describe('The destination of the flight'),
    date: z.string().describe('The date of the flight'),
  }),
  generate: async function* ({ source, destination, date }) {
    yield `Searching for flights from ${source} to ${destination} on ${date}...`;
    const results = await searchFlights(source, destination, date);
    return (<Flights flights={results} />);
  },
}
...
```

In the above example, the `Flights` component is used to display the search results. When the user clicks on a flight number, the `lookupFlight` tool is called with the flight number as a parameter. The `submitUserMessage` action is then called to trigger the next step in the conversation.

Learn more about tool calling in Next.js App Router by checking out examples [here](/examples/next-app/tools).

---
title: Streaming Values
description: Overview of streaming RSCs
---

import { UIPreviewCard, Card } from '@/components/home/card';
import { EventPlanning } from '@/components/home/event-planning';
import { Searching } from '@/components/home/searching';
import { Weather } from '@/components/home/weather';

# Streaming Values

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

The RSC API provides several utility functions to allow you to stream values from the server to the client. This is useful when you need more granular control over what you are streaming and how you are streaming it.

<Note>
  These utilities can also be paired with [AI SDK Core](/docs/ai-sdk-core)
  functions like [`streamText`](/docs/reference/ai-sdk-core/stream-text) and
  [`streamObject`](/docs/reference/ai-sdk-core/stream-object) to easily stream
  LLM generations from the server to the client.
</Note>

There are two functions provided by the RSC API that allow you to create streamable values:

- [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value) - creates a streamable (serializable) value, with full control over how you create, update, and close the stream.
- [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) - creates a streamable React component, with full control over how you create, update, and close the stream.

## `createStreamableValue`

The RSC API allows you to stream serializable Javascript values from the server to the client using [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value), such as strings, numbers, objects, and arrays.

This is useful when you want to stream:

- Text generations from the language model in real-time.
- Buffer values of image and audio generations from multi-modal models.
- Progress updates from multi-step agent runs.

## Creating a Streamable Value

You can import `createStreamableValue` from `ai/rsc` and use it to create a streamable value.

```tsx file='app/actions.ts'
'use server';

import { createStreamableValue } from 'ai/rsc';

export const runThread = async () => {
  const streamableStatus = createStreamableValue('thread.init');

  setTimeout(() => {
    streamableStatus.update('thread.run.create');
    streamableStatus.update('thread.run.update');
    streamableStatus.update('thread.run.end');
    streamableStatus.done('thread.end');
  }, 1000);

  return {
    status: streamableStatus.value,
  };
};
```

## Reading a Streamable Value

You can read streamable values on the client using `readStreamableValue`. It returns an async iterator that yields the value of the streamable as it is updated:

```tsx file='app/page.tsx'
import { readStreamableValue } from 'ai/rsc';
import { runThread } from '@/actions';

export default function Page() {
  return (
    <button
      onClick={async () => {
        const { status } = await runThread();

        for await (const value of readStreamableValue(status)) {
          console.log(value);
        }
      }}
    >
      Ask
    </button>
  );
}
```

Learn how to stream a text generation (with `streamText`) using the Next.js App Router and `createStreamableValue` in this [example](/examples/next-app/basics/streaming-text-generation).

## `createStreamableUI`

`createStreamableUI` creates a stream that holds a React component. Unlike AI SDK Core APIs, this function does not call a large language model. Instead, it provides a primitive that can be used to have granular control over streaming a React component.

## Using `createStreamableUI`

Let's look at how you can use the `createStreamableUI` function with a Server Action.

```tsx filename='app/actions.tsx'
'use server';

import { createStreamableUI } from 'ai/rsc';

export async function getWeather() {
  const weatherUI = createStreamableUI();

  weatherUI.update(<div style={{ color: 'gray' }}>Loading...</div>);

  setTimeout(() => {
    weatherUI.done(<div>It&apos;s a sunny day!</div>);
  }, 1000);

  return weatherUI.value;
}
```

First, you create a streamable UI with an empty state and then update it with a loading message. After 1 second, you mark the stream as done passing in the actual weather information as it's final value. The `.value` property contains the actual UI that can be sent to the client.

## Reading a Streamable UI

On the client side, you can call the `getWeather` Server Action and render the returned UI like any other React component.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { getWeather } from '@/actions';

export default function Page() {
  const [weather, setWeather] = useState<React.ReactNode | null>(null);

  return (
    <div>
      <button
        onClick={async () => {
          const weatherUI = await getWeather();
          setWeather(weatherUI);
        }}
      >
        What&apos;s the weather?
      </button>

      {weather}
    </div>
  );
}
```

When the button is clicked, the `getWeather` function is called, and the returned UI is set to the `weather` state and rendered on the page. Users will see the loading message first and then the actual weather information after 1 second.

Learn more about handling multiple streams in a single request in the [Multiple Streamables](/docs/advanced/multiple-streamables) guide.

Learn more about handling state for more complex use cases with [ AI/UI State ](/docs/ai-sdk-rsc/generative-ui-state).

---
title: Handling Loading State
description: Overview of handling loading state with AI SDK RSC
---

# Handling Loading State

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Given that responses from language models can often take a while to complete, it's crucial to be able to show loading state to users. This provides visual feedback that the system is working on their request and helps maintain a positive user experience.

There are three approaches you can take to handle loading state with the AI SDK RSC:

- Managing loading state similar to how you would in a traditional Next.js application. This involves setting a loading state variable in the client and updating it when the response is received.
- Streaming loading state from the server to the client. This approach allows you to track loading state on a more granular level and provide more detailed feedback to the user.
- Streaming loading component from the server to the client. This approach allows you to stream a React Server Component to the client while awaiting the model's response.

## Handling Loading State on the Client

### Client

Let's create a simple Next.js page that will call the `generateResponse` function when the form is submitted. The function will take in the user's prompt (`input`) and then generate a response (`response`). To handle the loading state, use the `loading` state variable. When the form is submitted, set `loading` to `true`, and when the response is received, set it back to `false`. While the response is being streamed, the input field will be disabled.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<string>('');
  const [loading, setLoading] = useState<boolean>(false);

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setLoading(true);
          const response = await generateResponse(input);

          let textContent = '';

          for await (const delta of readStreamableValue(response)) {
            textContent = `${textContent}${delta}`;
            setGeneration(textContent);
          }
          setInput('');
          setLoading(false);
        }}
      >
        <input
          type="text"
          value={input}
          disabled={loading}
          className="disabled:opacity-50"
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

### Server

Now let's implement the `generateResponse` function. Use the `streamText` function to generate a response to the input.

```typescript filename='app/actions.ts'
'use server';

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';

export async function generateResponse(prompt: string) {
  const stream = createStreamableValue();

  (async () => {
    const { textStream } = streamText({
      model: openai('gpt-4o'),
      prompt,
    });

    for await (const text of textStream) {
      stream.update(text);
    }

    stream.done();
  })();

  return stream.value;
}
```

## Streaming Loading State from the Server

If you are looking to track loading state on a more granular level, you can create a new streamable value to store a custom variable and then read this on the frontend. Let's update the example to create a new streamable value for tracking loading state:

### Server

```typescript filename='app/actions.ts' highlight='9,22,25'
'use server';

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';

export async function generateResponse(prompt: string) {
  const stream = createStreamableValue();
  const loadingState = createStreamableValue({ loading: true });

  (async () => {
    const { textStream } = streamText({
      model: openai('gpt-4o'),
      prompt,
    });

    for await (const text of textStream) {
      stream.update(text);
    }

    stream.done();
    loadingState.done({ loading: false });
  })();

  return { response: stream.value, loadingState: loadingState.value };
}
```

### Client

```tsx filename='app/page.tsx' highlight="22,30-34"
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<string>('');
  const [loading, setLoading] = useState<boolean>(false);

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setLoading(true);
          const { response, loadingState } = await generateResponse(input);

          let textContent = '';

          for await (const responseDelta of readStreamableValue(response)) {
            textContent = `${textContent}${responseDelta}`;
            setGeneration(textContent);
          }
          for await (const loadingDelta of readStreamableValue(loadingState)) {
            if (loadingDelta) {
              setLoading(loadingDelta.loading);
            }
          }
          setInput('');
          setLoading(false);
        }}
      >
        <input
          type="text"
          value={input}
          disabled={loading}
          className="disabled:opacity-50"
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

This allows you to provide more detailed feedback about the generation process to your users.

## Streaming Loading Components with `streamUI`

If you are using the [ `streamUI` ](/docs/reference/ai-sdk-rsc/stream-ui) function, you can stream the loading state to the client in the form of a React component. `streamUI` supports the usage of [ JavaScript generator functions ](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*), which allow you to yield some value (in this case a React component) while some other blocking work completes.

## Server

```ts
'use server';

import { openai } from '@ai-sdk/openai';
import { streamUI } from 'ai/rsc';

export async function generateResponse(prompt: string) {
  const result = await streamUI({
    model: openai('gpt-4o'),
    prompt,
    text: async function* ({ content }) {
      yield <div>loading...</div>;
      return <div>{content}</div>;
    },
  });

  return result.value;
}
```

<Note>
  Remember to update the file from `.ts` to `.tsx` because you are defining a
  React component in the `streamUI` function.
</Note>

## Client

```tsx
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<React.ReactNode>();

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          const result = await generateResponse(input);
          setGeneration(result);
          setInput('');
        }}
      >
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

---
title: Error Handling
description: Learn how to handle errors with the AI SDK.
---

# Error Handling

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Two categories of errors can occur when working with the RSC API: errors while streaming user interfaces and errors while streaming other values.

## Handling UI Errors

To handle errors while generating UI, the [`streamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) object exposes an `error()` method.

```tsx filename='app/actions.tsx'
'use server';

import { createStreamableUI } from 'ai/rsc';

export async function getStreamedUI() {
  const ui = createStreamableUI();

  (async () => {
    ui.update(<div>loading</div>);
    const data = await fetchData();
    ui.done(<div>{data}</div>);
  })().catch(e => {
    ui.error(<div>Error: {e.message}</div>);
  });

  return ui.value;
}
```

With this method, you can catch any error with the stream, and return relevant UI. On the client, you can also use a [React Error Boundary](https://react.dev/reference/react/Component#catching-rendering-errors-with-an-error-boundary) to wrap the streamed component and catch any additional errors.

```tsx filename='app/page.tsx'
import { getStreamedUI } from '@/actions';
import { useState } from 'react';
import { ErrorBoundary } from './ErrorBoundary';

export default function Page() {
  const [streamedUI, setStreamedUI] = useState(null);

  return (
    <div>
      <button
        onClick={async () => {
          const newUI = await getStreamedUI();
          setStreamedUI(newUI);
        }}
      >
        What does the new UI look like?
      </button>
      <ErrorBoundary>{streamedUI}</ErrorBoundary>
    </div>
  );
}
```

## Handling Other Errors

To handle other errors while streaming, you can return an error object that the reciever can use to determine why the failure occurred.

```tsx filename='app/actions.tsx'
'use server';

import { createStreamableValue } from 'ai/rsc';
import { fetchData, emptyData } from '../utils/data';

export const getStreamedData = async () => {
  const streamableData = createStreamableValue<string>(emptyData);

  try {
    (() => {
      const data1 = await fetchData();
      streamableData.update(data1);

      const data2 = await fetchData();
      streamableData.update(data2);

      const data3 = await fetchData();
      streamableData.done(data3);
    })();

    return { data: streamableData.value };
  } catch (e) {
    return { error: e.message };
  }
};
```

---
title: Handling Authentication
description: Learn how to authenticate with the AI SDK.
---

# Authentication

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

The RSC API makes extensive use of [`Server Actions`](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations) to power streaming values and UI from the server.

Server Actions are exposed as public, unprotected endpoints. As a result, you should treat Server Actions as you would public-facing API endpoints and ensure that the user is authorized to perform the action before returning any data.

```tsx filename="app/actions.tsx"
'use server';

import { cookies } from 'next/headers';
import { createStremableUI } from 'ai/rsc';
import { validateToken } from '../utils/auth';

export const getWeather = async () => {
  const token = cookies().get('token');

  if (!token || !validateToken(token)) {
    return {
      error: 'This action requires authentication',
    };
  }
  const streamableDisplay = createStreamableUI(null);

  streamableDisplay.update(<Skeleton />);
  streamableDisplay.done(<Weather />);

  return {
    display: streamableDisplay.value,
  };
};
```

---
title: Migrating from RSC to UI
description: Learn how to migrate from AI SDK RSC to AI SDK UI.
---

# Migrating from RSC to UI

This guide helps you migrate from AI SDK RSC to AI SDK UI.

## Background

The AI SDK has two packages that help you build the frontend for your applications – [AI SDK UI](/docs/ai-sdk-ui) and [AI SDK RSC](/docs/ai-sdk-rsc).

We introduced support for using [React Server Components](https://react.dev/reference/rsc/server-components) (RSC) within the AI SDK to simplify building generative user interfaces for frameworks that support RSC.

However, given we're pushing the boundaries of this technology, AI SDK RSC currently faces significant limitations that make it unsuitable for stable production use.

- It is not possible to abort a stream using server actions. This will be improved in future releases of React and Next.js [(1122)](https://github.com/vercel/ai/issues/1122).
- When using `createStreamableUI` and `streamUI`, components remount on `.done()`, causing them to flicker [(2939)](https://github.com/vercel/ai/issues/2939).
- Many suspense boundaries can lead to crashes [(2843)](https://github.com/vercel/ai/issues/2843).
- Using `createStreamableUI` can lead to quadratic data transfer. You can avoid this using createStreamableValue instead, and rendering the component client-side.
- Closed RSC streams cause update issues [(3007)](https://github.com/vercel/ai/issues/3007).

Due to these limitations, AI SDK RSC is marked as experimental, and we do not recommend using it for stable production environments.

As a result, we strongly recommend migrating to AI SDK UI, which has undergone extensive development to provide a more stable and production grade experience.

In building [v0](https://v0.dev), we have invested considerable time exploring how to create the best chat experience on the web. AI SDK UI ships with many of these best practices and commonly used patterns like [language model middleware](/docs/ai-sdk-core/middleware), [multi-step tool calls](/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls), [attachments](/docs/ai-sdk-ui/chatbot#attachments-experimental), [telemetry](/docs/ai-sdk-core/telemetry), [provider registry](/docs/ai-sdk-core/provider-management#provider-registry), and many more. These features have been considerately designed into a neat abstraction that you can use to reliably integrate AI into your applications.

## Streaming Chat Completions

### Basic Setup

The `streamUI` function executes as part of a server action as illustrated below.

#### Before: Handle generation and rendering in a single server action

```tsx filename="@/app/actions.tsx"
import { openai } from '@ai-sdk/openai';
import { getMutableAIState, streamUI } from 'ai/rsc';

export async function sendMessage(message: string) {
  'use server';

  const messages = getMutableAIState('messages');

  messages.update([...messages.get(), { role: 'user', content: message }]);

  const { value: stream } = await streamUI({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages: messages.get(),
    text: async function* ({ content, done }) {
      // process text
    },
    tools: {
      // tool definitions
    },
  });

  return stream;
}
```

#### Before: Call server action and update UI state

The chat interface calls the server action. The response is then saved using the `useUIState` hook.

```tsx filename="@/app/page.tsx"
'use client';

import { useState, ReactNode } from 'react';
import { useActions, useUIState } from 'ai/rsc';

export default function Page() {
  const { sendMessage } = useActions();
  const [input, setInput] = useState('');
  const [messages, setMessages] = useUIState();

  return (
    <div>
      {messages.map(message => message)}

      <form
        onSubmit={async () => {
          const response: ReactNode = await sendMessage(input);
          setMessages(msgs => [...msgs, response]);
        }}
      >
        <input type="text" />
        <button type="submit">Submit</button>
      </form>
    </div>
  );
}
```

The `streamUI` function combines generating text and rendering the user interface. To migrate to AI SDK UI, you need to **separate these concerns** – streaming generations with `streamText` and rendering the UI with `useChat`.

#### After: Replace server action with route handler

The `streamText` function executes as part of a route handler and streams the response to the client. The `useChat` hook on the client decodes this stream and renders the response within the chat interface.

```ts filename="@/app/api/chat/route.ts"
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages,
    tools: {
      // tool definitions
    },
  });

  return result.toDataStreamResponse();
}
```

#### After: Update client to use chat hook

```tsx filename="@/app/page.tsx"
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

### Parallel Tool Calls

In AI SDK RSC, `streamUI` does not support parallel tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.

With AI SDK UI, `useChat` comes with built-in support for parallel tool calls. You can define multiple tools in the `streamText` and have them called them in parallel. The `useChat` hook will then handle the parallel tool calls for you automatically.

### Multi-Step Tool Calls

In AI SDK RSC, `streamUI` does not support multi-step tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.

With AI SDK UI, `useChat` comes with built-in support for multi-step tool calls. You can set `maxSteps` in the `streamText` function to define the number of steps the language model can make in a single call. The `useChat` hook will then handle the multi-step tool calls for you automatically.

### Generative User Interfaces

The `streamUI` function uses `tools` as a way to execute functions based on user input and renders React components based on the function output to go beyond text in the chat interface.

#### Before: Render components within the server action and stream to client

```tsx filename="@/app/actions.tsx"
import { z } from 'zod';
import { streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import { Weather } from '@/components/weather';

const { value: stream } = await streamUI({
  model: openai('gpt-4o'),
  system: 'you are a friendly assistant!',
  messages,
  text: async function* ({ content, done }) {
    // process text
  },
  tools: {
    displayWeather: {
      description: 'Display the weather for a location',
      parameters: z.object({
        latitude: z.number(),
        longitude: z.number(),
      }),
      generate: async function* ({ latitude, longitude }) {
        yield <div>Loading weather...</div>;

        const { value, unit } = await getWeather({ latitude, longitude });

        return <Weather value={value} unit={unit} />;
      },
    },
  },
});
```

As mentioned earlier, `streamUI` generates text and renders the React component in a single server action call.

#### After: Replace with route handler and stream props data to client

The `streamText` function streams the props data as response to the client, while `useChat` decode the stream as `toolInvocations` and renders the chat interface.

```ts filename="@/app/api/chat/route.ts"
import { z } from 'zod';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import { streamText } from 'ai';

export async function POST(request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages,
    tools: {
      displayWeather: {
        description: 'Display the weather for a location',
        parameters: z.object({
          latitude: z.number(),
          longitude: z.number(),
        }),
        execute: async function ({ latitude, longitude }) {
          const props = await getWeather({ latitude, longitude });
          return props;
        },
      },
    },
  });

  return result.toDataStreamResponse();
}
```

#### After: Update client to use chat hook and render components using tool invocations

```tsx filename="@/app/page.tsx"
'use client';

import { useChat } from 'ai/react';
import { Weather } from '@/components/weather';

export default function Page() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>

          <div>
            {message.toolInvocations.map(toolInvocation => {
              const { toolName, toolCallId, state } = toolInvocation;

              if (state === 'result') {
                const { result } = toolInvocation;

                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <Weather weatherAtLocation={result} />
                    ) : null}
                  </div>
                );
              } else {
                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <div>Loading weather...</div>
                    ) : null}
                  </div>
                );
              }
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

### Handling Client Interactions

With AI SDK RSC, components streamed to the client can trigger subsequent generations by calling the relevant server action using the `useActions` hooks. This is possible as long as the component is a descendant of the `<AI/>` context provider.

#### Before: Use actions hook to send messages

```tsx filename="@/app/components/list-flights.tsx"
'use client';

import { useActions, useUIState } from 'ai/rsc';

export function ListFlights({ flights }) {
  const { sendMessage } = useActions();
  const [_, setMessages] = useUIState();

  return (
    <div>
      {flights.map(flight => (
        <div
          key={flight.id}
          onClick={async () => {
            const response = await sendMessage(
              `I would like to choose flight ${flight.id}!`,
            );

            setMessages(msgs => [...msgs, response]);
          }}
        >
          {flight.name}
        </div>
      ))}
    </div>
  );
}
```

#### After: Use another chat hook with same ID from the component

After switching to AI SDK UI, these messages are synced by initializing the `useChat` hook in the component with the same `id` as the parent component.

```tsx filename="@/app/components/list-flights.tsx"
'use client';

import { useChat } from 'ai/react';

export function ListFlights({ chatId, flights }) {
  const { append } = useChat({
    id: chatId,
    body: { id: chatId },
    maxSteps: 5,
  });

  return (
    <div>
      {flights.map(flight => (
        <div
          key={flight.id}
          onClick={async () => {
            await append({
              role: 'user',
              content: `I would like to choose flight ${flight.id}!`,
            });
          }}
        >
          {flight.name}
        </div>
      ))}
    </div>
  );
}
```

### Loading Indicators

In AI SDK RSC, you can use the `initial` parameter of `streamUI` to define the component to display while the generation is in progress.

#### Before: Use `loading` to show loading indicator

```tsx filename="@/app/actions.tsx"
import { openai } from '@ai-sdk/openai';
import { streamUI } from 'ai/rsc';

const { value: stream } = await streamUI({
  model: openai('gpt-4o'),
  system: 'you are a friendly assistant!',
  messages,
  initial: <div>Loading...</div>,
  text: async function* ({ content, done }) {
    // process text
  },
  tools: {
    // tool definitions
  },
});

return stream;
```

With AI SDK UI, you can use the tool invocation state to show a loading indicator while the tool is executing.

#### After: Use tool invocation state to show loading indicator

```tsx filename="@/app/components/message.tsx"
'use client';

export function Message({ role, content, toolInvocations }) {
  return (
    <div>
      <div>{role}</div>
      <div>{content}</div>

      {toolInvocations && (
        <div>
          {toolInvocations.map(toolInvocation => {
            const { toolName, toolCallId, state } = toolInvocation;

            if (state === 'result') {
              const { result } = toolInvocation;

              return (
                <div key={toolCallId}>
                  {toolName === 'getWeather' ? (
                    <Weather weatherAtLocation={result} />
                  ) : null}
                </div>
              );
            } else {
              return (
                <div key={toolCallId}>
                  {toolName === 'getWeather' ? (
                    <Weather isLoading={true} />
                  ) : (
                    <div>Loading...</div>
                  )}
                </div>
              );
            }
          })}
        </div>
      )}
    </div>
  );
}
```

### Saving Chats

Before implementing `streamUI` as a server action, you should create an `<AI/>` provider and wrap your application at the root layout to sync the AI and UI states. During initialization, you typically use the `onSetAIState` callback function to track updates to the AI state and save it to the database when `done(...)` is called.

#### Before: Save chats using callback function of context provider

```ts filename="@/app/actions.ts"
import { createAI } from 'ai/rsc';
import { saveChat } from '@/utils/queries';

export const AI = createAI({
  initialAIState: {},
  initialUIState: {},
  actions: {
    // server actions
  },
  onSetAIState: async ({ state, done }) => {
    'use server';

    if (done) {
      await saveChat(state);
    }
  },
});
```

#### After: Save chats using callback function of `streamText`

With AI SDK UI, you will save chats using the `onFinish` callback function of `streamText` in your route handler.

```ts filename="@/app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { saveChat } from '@/utils/queries';
import { streamText, convertToCoreMessages } from 'ai';

export async function POST(request) {
  const { id, messages } = await request.json();

  const coreMessages = convertToCoreMessages(messages);

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages: coreMessages,
    onFinish: async ({ responseMessages }) => {
      try {
        await saveChat({
          id,
          messages: [...coreMessages, ...responseMessages],
        });
      } catch (error) {
        console.error('Failed to save chat');
      }
    },
  });

  return result.toDataStreamResponse();
}
```

### Restoring Chats

When using AI SDK RSC, the `useUIState` hook contains the UI state of the chat. When restoring a previously saved chat, the UI state needs to be loaded with messages.

Similar to how you typically save chats in AI SDK RSC, you should use the `onGetUIState` callback function to retrieve the chat from the database, convert it into UI state, and return it to be accessible through `useUIState`.

#### Before: Load chat from database using callback function of context provider

```ts filename="@/app/actions.ts"
import { createAI } from 'ai/rsc';
import { loadChatFromDB, convertToUIState } from '@/utils/queries';

export const AI = createAI({
  actions: {
    // server actions
  },
  onGetUIState: async () => {
    'use server';

    const chat = await loadChatFromDB();
    const uiState = convertToUIState(chat);

    return uiState;
  },
});
```

AI SDK UI uses the `messages` field of `useChat` to store messages. To load messages when `useChat` is mounted, you should use `initialMessages`.

As messages are typically loaded from the database, we can use a server actions inside a Page component to fetch an older chat from the database during static generation and pass the messages as props to the `<Chat/>` component.

#### After: Load chat from database during static generation of page

```tsx filename="@/app/chat/[id]/page.tsx"
import { Chat } from '@/app/components/chat';
import { getChatById } from '@/utils/queries';

// link to example implementation: https://github.com/vercel/ai-chatbot/blob/00b125378c998d19ef60b73fe576df0fe5a0e9d4/lib/utils.ts#L87-L127
import { convertToUIMessages } from '@/utils/functions';

export default async function Page({ params }: { params: any }) {
  const { id } = params;
  const chatFromDb = await getChatById({ id });

  const chat: Chat = {
    ...chatFromDb,
    messages: convertToUIMessages(chatFromDb.messages),
  };

  return <Chat key={id} id={chat.id} initialMessages={chat.messages} />;
}
```

#### After: Pass chat messages as props and load into chat hook

```tsx filename="@/app/components/chat.tsx"
'use client';

import { Message } from 'ai';
import { useChat } from 'ai/react';

export function Chat({
  id,
  initialMessages,
}: {
  id;
  initialMessages: Array<Message>;
}) {
  const { messages } = useChat({
    id,
    initialMessages,
  });

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>
        </div>
      ))}
    </div>
  );
}
```

## Streaming Object Generation

The `createStreamableValue` function streams any serializable data from the server to the client. As a result, this function allows you to stream object generations from the server to the client when paired with `streamObject`.

#### Before: Use streamable value to stream object generations

```ts filename="@/app/actions.ts"
import { streamObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';
import { notificationsSchema } from '@/utils/schemas';

export async function generateSampleNotifications() {
  'use server';

  const stream = createStreamableValue();

  (async () => {
    const { partialObjectStream } = streamObject({
      model: openai('gpt-4o'),
      system: 'generate sample ios messages for testing',
      prompt: 'messages from a family group chat during diwali, max 4',
      schema: notificationsSchema,
    });

    for await (const partialObject of partialObjectStream) {
      stream.update(partialObject);
    }
  })();

  stream.done();

  return { partialNotificationsStream: stream.value };
}
```

#### Before: Read streamable value and update object

```tsx filename="@/app/page.tsx"
'use client';

import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { generateSampleNotifications } from '@/app/actions';

export default function Page() {
  const [notifications, setNotifications] = useState(null);

  return (
    <div>
      <button
        onClick={async () => {
          const { partialNotificationsStream } =
            await generateSampleNotifications();

          for await (const partialNotifications of readStreamableValue(
            partialNotificationsStream,
          )) {
            if (partialNotifications) {
              setNotifications(partialNotifications.notifications);
            }
          }
        }}
      >
        Generate
      </button>
    </div>
  );
}
```

To migrate to AI SDK UI, you should use the `useObject` hook and implement `streamObject` within your route handler.

#### After: Replace with route handler and stream text response

```ts filename="@/app/api/object/route.ts"
import { streamObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { notificationSchema } from '@/utils/schemas';

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamObject({
    model: openai('gpt-4-turbo'),
    schema: notificationSchema,
    prompt:
      `Generate 3 notifications for a messages app in this context:` + context,
  });

  return result.toTextStreamResponse();
}
```

#### After: Use object hook to decode stream and update object

```tsx filename="@/app/page.tsx"
'use client';

import { useObject } from 'ai/react';
import { notificationSchema } from '@/utils/schemas';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/object',
    schema: notificationSchema,
  });

  return (
    <div>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

---
title: AI SDK RSC
description: Learn about AI SDK RSC.
collapsed: true
---

# AI SDK RSC

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

<IndexCards
  cards={[
    {
      title: 'Overview',
      description: 'Learn about AI SDK RSC.',
      href: '/docs/ai-sdk-rsc/overview',
    },
    {
      title: 'Streaming React Components',
      description: 'Learn how to stream React components.',
      href: '/docs/ai-sdk-rsc/streaming-react-components',
    },
    {
      title: 'Managing Generative UI State',
      description: 'Learn how to manage generative UI state.',
      href: '/docs/ai-sdk-rsc/generative-ui-state',
    },
    {
      title: 'Saving and Restoring States',
      description: 'Learn how to save and restore states.',
      href: '/docs/ai-sdk-rsc/saving-and-restoring-states',
    },
    {
      title: 'Multi-step Interfaces',
      description: 'Learn how to build multi-step interfaces.',
      href: '/docs/ai-sdk-rsc/multistep-interfaces',
    },
    {
      title: 'Streaming Values',
      description: 'Learn how to stream values with AI SDK RSC.',
      href: '/docs/ai-sdk-rsc/streaming-values',
    },
    {
      title: 'Error Handling',
      description: 'Learn how to handle errors.',
      href: '/docs/ai-sdk-rsc/error-handling',
    },
    {
      title: 'Authentication',
      description: 'Learn how to authenticate users.',
      href: '/docs/ai-sdk-rsc/authentication',
    },
  ]}
/>

---
title: Prompt Engineering
description: Learn how to engineer prompts for LLMs with the AI SDK
---

# Prompt Engineering

## What is a Large Language Model (LLM)?

A Large Language Model is essentially a prediction engine that takes a sequence of words as input and aims to predict the most likely sequence to follow. It does this by assigning probabilities to potential next sequences and then selecting one. The model continues to generate sequences until it meets a specified stopping criterion.

These models learn by training on massive text corpuses, which means they will be better suited to some use cases than others. For example, a model trained on GitHub data would understand the probabilities of sequences in source code particularly well. However, it's crucial to understand that the generated sequences, while often seeming plausible, can sometimes be random and not grounded in reality. As these models become more accurate, many surprising abilities and applications emerge.

## What is a prompt?

Prompts are the starting points for LLMs. They are the inputs that trigger the model to generate text. The scope of prompt engineering involves not just crafting these prompts but also understanding related concepts such as hidden prompts, tokens, token limits, and the potential for prompt hacking, which includes phenomena like jailbreaks and leaks.

## Why is prompt engineering needed?

Prompt engineering currently plays a pivotal role in shaping the responses of LLMs. It allows us to tweak the model to respond more effectively to a broader range of queries. This includes the use of techniques like semantic search, command grammars, and the ReActive model architecture. The performance, context window, and cost of LLMs varies between models and model providers which adds further constraints to the mix. For example, the GPT-4 model is more expensive than GPT-3.5-turbo and significantly slower, but it can also be more effective at certain tasks. And so, like many things in software engineering, there is a trade-offs between cost and performance.

To assist with comparing and tweaking LLMs, we've built an AI playground that allows you to compare the performance of different models side-by-side online. When you're ready, you can even generate code with the AI SDK to quickly use your prompt and your selected model into your own applications.

## Example: Build a Slogan Generator

### Start with an instruction

Imagine you want to build a slogan generator for marketing campaigns. Creating catchy slogans isn't always straightforward!

First, you'll need a prompt that makes it clear what you want. Let's start with an instruction. Submit this prompt to generate your first completion.

<InlinePrompt initialInput="Create a slogan for a coffee shop." />

Not bad! Now, try making your instruction more specific.

<InlinePrompt initialInput="Create a slogan for an organic coffee shop." />

Introducing a single descriptive term to our prompt influences the completion. Essentially, crafting your prompt is the means by which you "instruct" or "program" the model.

### Include examples

Clear instructions are key for quality outcomes, but that might not always be enough. Let's try to enhance your instruction further.

<InlinePrompt initialInput="Create three slogans for a coffee shop with live music." />

These slogans are fine, but could be even better. It appears the model overlooked the 'live' part in our prompt. Let's change it slightly to generate more appropriate suggestions.

Often, it's beneficial to both demonstrate and tell the model your requirements. Incorporating examples in your prompt can aid in conveying patterns or subtleties. Test this prompt that carries a few examples.

<InlinePrompt
  initialInput={`Create three slogans for a business with unique features.

Business: Bookstore with cats
Slogans: "Purr-fect Pages", "Books and Whiskers", "Novels and Nuzzles"
Business: Gym with rock climbing
Slogans: "Peak Performance", "Reach New Heights", "Climb Your Way Fit"
Business: Coffee shop with live music
Slogans:`}
/>

Great! Incorporating examples of expected output for a certain input prompted the model to generate the kind of names we aimed for.

### Tweak your settings

Apart from designing prompts, you can influence completions by tweaking model settings. A crucial setting is the **temperature**.

You might have seen that the same prompt, when repeated, yielded the same or nearly the same completions. This happens when your temperature is at 0.

Attempt to re-submit the identical prompt a few times with temperature set to 1.

<InlinePrompt
  initialInput={`Create three slogans for a business with unique features.

Business: Bookstore with cats
Slogans: "Purr-fect Pages", "Books and Whiskers", "Novels and Nuzzles"
Business: Gym with rock climbing
Slogans: "Peak Performance", "Reach New Heights", "Climb Your Way Fit"
Business: Coffee shop with live music
Slogans:`}
showTemp={true}
initialTemperature={1}
/>

Notice the difference? With a temperature above 0, the same prompt delivers varied completions each time.

Keep in mind that the model forecasts the text most likely to follow the preceding text. Temperature, a value from 0 to 1, essentially governs the model's confidence level in making these predictions. A lower temperature implies lesser risks, leading to more precise and deterministic completions. A higher temperature yields a broader range of completions.

For your slogan generator, you might want a large pool of name suggestions. A moderate temperature of 0.6 should serve well.

## Recommended Resources

Prompt Engineering is evolving rapidly, with new methods and research papers surfacing every week. Here are some resources that we've found useful for learning about and experimenting with prompt engineering:

- [The Vercel AI Playground](/playground)
- [Brex Prompt Engineering](https://github.com/brexhq/prompt-engineering)
- [Prompt Engineering Guide by Dair AI](https://www.promptingguide.ai/)

---
title: Stopping Streams
description: Learn how to cancel streams with the AI SDK
---

# Stopping Streams

Cancelling ongoing streams is often needed.
For example, users might want to stop a stream when they realize that the response is not what they want.

The different parts of the AI SDK support cancelling streams in different ways.

## AI SDK Core

The AI SDK functions have an `abortSignal` argument that you can use to cancel a stream.
You would use this if you want to cancel a stream from the server side to the LLM API, e.g. by
forwarding the `abortSignal` from the request.

```tsx highlight="10,11"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    prompt,
    // forward the abort signal:
    abortSignal: req.signal,
  });

  return result.toTextStreamResponse();
}
```

## AI SDK UI

The hooks, e.g. `useChat` or `useCompletion`, provide a `stop` helper function that can be used to cancel a stream.
This will cancel the stream from the client side to the server.

```tsx file="app/page.tsx" highlight="9,18-20"
'use client';

import { useCompletion } from 'ai/react';

export default function Chat() {
  const {
    input,
    completion,
    stop,
    isLoading,
    handleSubmit,
    handleInputChange,
  } = useCompletion();

  return (
    <div>
      {isLoading && (
        <button type="button" onClick={() => stop()}>
          Stop
        </button>
      )}
      {completion}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

## AI SDK RSC

<Note type="warning">
  The AI SDK RSC does not currently support stopping streams.
</Note>

---
title: Backpressure
description: How to handle backpressure and cancellation when working with the AI SDK
---

# Stream Back-pressure and Cancellation

This page focuses on understanding back-pressure and cancellation when working with streams. You do not need to know this information to use the AI SDK, but for those interested, it offers a deeper dive on why and how the SDK optimally streams responses.

In the following sections, we'll explore back-pressure and cancellation in the context of a simple example program. We'll discuss the issues that can arise from an eager approach and demonstrate how a lazy approach can resolve them.

## Back-pressure and Cancellation with Streams

Let's begin by setting up a simple example program:

```jsx
// A generator that will yield positive integers
async function* integers() {
  let i = 1;
  while (true) {
    console.log(`yielding ${i}`);
    yield i++;

    await sleep(100);
  }
}
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async start(controller) {
      for await (const v of iterator) {
        controller.enqueue(v);
      }
      controller.close();
    },
  });
}

// Collect data from stream
async function run() {
  // Set up a stream of integers
  const stream = createStream(integers());

  // Read values from our stream
  const reader = stream.getReader();
  for (let i = 0; i < 10_000; i++) {
    // we know our stream is infinite, so there's no need to check `done`.
    const { value } = await reader.read();
    console.log(`read ${value}`);

    await sleep(1_000);
  }
}
run();
```

In this example, we create an async-generator that yields positive integers, a `ReadableStream` that wraps our integer generator, and a reader which will read values out of our stream. Notice, too, that our integer generator logs out `"yielding ${i}"`, and our reader logs out `"read ${value}"`. Both take an arbitrary amount of time to process data, represented with a 100ms sleep in our generator, and a 1sec sleep in our reader.

## Back-pressure

If you were to run this program, you'd notice something funny. We'll see roughly 10 "yield" logs for every "read" log. This might seem obvious, the generator can push values 10x faster than the reader can pull them out. But it represents a problem, our `stream` has to maintain an ever expanding queue of items that have been pushed in but not pulled out.

The problem stems from the way we wrap our generator into a stream. Notice the use of `for await (…)` inside our `start` handler. This is an **eager** for-loop, and it is constantly running to get the next value from our generator to be enqueued in our stream. This means our stream does not respect back-pressure, the signal from the consumer to the producer that more values aren't needed _yet_. We've essentially spawned a thread that will perpetually push more data into the stream, one that runs as fast as possible to push new data immediately. Worse, there's no way to signal to this thread to stop running when we don't need additional data.

To fix this, `ReadableStream` allows a `pull` handler. `pull` is called every time the consumer attempts to read more data from our stream (if there's no data already queued internally). But it's not enough to just move the `for await(…)` into `pull`, we also need to convert from an eager enqueuing to a **lazy** one. By making these 2 changes, we'll be able to react to the consumer. If they need more data, we can easily produce it, and if they don't, then we don't need to spend any time doing unnecessary work.

```jsx
function createStream(iterator) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await iterator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}
```

Our `createStream` is a little more verbose now, but the new code is important. First, we need to manually call our `iterator.next()` method. This returns a `Promise` for an object with the type signature `{ done: boolean, value: T }`. If `done` is `true`, then we know that our iterator won't yield any more values and we must `close` the stream (this allows the consumer to know that the stream is also finished producing values). Else, we need to `enqueue` our newly produced value.

When we run this program, we see that our "yield" and "read" logs are now paired. We're no longer yielding 10x integers for every read! And, our stream now only needs to maintain 1 item in its internal buffer. We've essentially given control to the consumer, so that it's responsible for producing new values as it needs it. Neato!

## Cancellation

Let's go back to our initial eager example, with 1 small edit. Now instead of reading 10,000 integers, we're only going to read 3:

```jsx
// A generator that will yield positive integers
async function* integers() {
  let i = 1;
  while (true) {
    console.log(`yielding ${i}`);
    yield i++;

    await sleep(100);
  }
}
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async start(controller) {
      for await (const v of iterator) {
        controller.enqueue(v);
      }
      controller.close();
    },
  });
}
// Collect data from stream
async function run() {
  // Set up a stream that of integers
  const stream = createStream(integers());

  // Read values from our stream
  const reader = stream.getReader();
  // We're only reading 3 items this time:
  for (let i = 0; i < 3; i++) {
    // we know our stream is infinite, so there's no need to check `done`.
    const { value } = await reader.read();
    console.log(`read ${value}`);

    await sleep(1000);
  }
}
run();
```

We're back to yielding 10x the number of values read. But notice now, after we've read 3 values, we're continuing to yield new values. We know that our reader will never read another value, but our stream doesn't! The eager `for await (…)` will continue forever, loudly enqueuing new values into our stream's buffer and increasing our memory usage until it consumes all available program memory.

The fix to this is exactly the same: use `pull` and manual iteration. By producing values _**lazily**_, we tie the lifetime of our integer generator to the lifetime of the reader. Once the reads stop, the yields will stop too:

```jsx
// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await iterator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}
```

Since the solution is the same as implementing back-pressure, it shows that they're just 2 facets of the same problem: Pushing values into a stream should be done **lazily**, and doing it eagerly results in expected problems.

## Tying Stream Laziness to AI Responses

Now let's imagine you're integrating AIBot service into your product. Users will be able to prompt "count from 1 to infinity", the browser will fetch your AI API endpoint, and your servers connect to AIBot to get a response. But "infinity" is, well, infinite. The response will never end!

After a few seconds, the user gets bored and navigates away. Or maybe you're doing local development and a hot-module reload refreshes your page. The browser will have ended its connection to the API endpoint, but will your server end its connection with AIBot?

If you used the eager `for await (...)` approach, then the connection is still running and your server is asking for more and more data from AIBot. Our server spawned a "thread" and there's no signal when we can end the eager pulls. Eventually, the server is going to run out of memory (remember, there's no active fetch connection to read the buffering responses and free them).

{/* When we started writing the streaming code for the AI SDK, we confirm aborting a fetch will end a streamed response from Next.js */}

With the lazy approach, this is taken care of for you. Because the stream will only request new data from AIBot when the consumer requests it, navigating away from the page naturally frees all resources. The fetch connection aborts and the server can clean up the response. The `ReadableStream` tied to that response can now be garbage collected. When that happens, the connection it holds to AIBot can then be freed.

---
title: Caching
description: How to handle caching when working with the AI SDK
---

# Caching Responses

Depending on the type of application you're building, you may want to cache the responses you receive from your AI provider, at least temporarily.

Each stream helper for each provider has special lifecycle callbacks you can use.
The one of interest is likely `onFinish`, which is called when the stream is closed. This is where you can cache the full response.

Here's an example of how you can implement caching using Vercel KV and Next.js to cache the OpenAI response for 1 hour:

## Example: Vercel KV

This example uses [Vercel KV](https://vercel.com/storage/kv) and Next.js to cache the OpenAI response for 1 hour.

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { formatDataStreamPart, streamText } from 'ai';
import kv from '@vercel/kv';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

// simple cache implementation, use Vercel KV or a similar service for production
const cache = new Map<string, string>();

export async function POST(req: Request) {
  const { messages } = await req.json();

  // come up with a key based on the request:
  const key = JSON.stringify(messages);

  // Check if we have a cached response
  const cached = await kv.get(key);
  if (cached != null) {
    return new Response(formatDataStreamPart('text', cached), {
      status: 200,
      headers: { 'Content-Type': 'text/plain' },
    });
  }

  // Call the language model:
  const result = streamText({
    model: openai('gpt-4o'),
    messages,
    async onFinish({ text }) {
      // Cache the response text:
      await kv.set(key, text);
      await kv.expire(key, 60 * 60);
    },
  });

  // Respond with the stream
  return result.toDataStreamResponse();
}
```

---
title: Multiple Streamables
description: Learn to handle multiple streamables in your application.
---

# Multiple Streams

## Multiple Streamable UIs

The AI SDK RSC APIs allow you to compose and return any number of streamable UIs, along with other data, in a single request. This can be useful when you want to decouple the UI into smaller components and stream them separately.

```tsx file='app/actions.tsx'
'use server';

import { createStreamableUI } from 'ai/rsc';

export async function getWeather() {
  const weatherUI = createStreamableUI();
  const forecastUI = createStreamableUI();

  weatherUI.update(<div>Loading weather...</div>);
  forecastUI.update(<div>Loading forecast...</div>);

  getWeatherData().then(weatherData => {
    weatherUI.done(<div>{weatherData}</div>);
  });

  getForecastData().then(forecastData => {
    forecastUI.done(<div>{forecastData}</div>);
  });

  // Return both streamable UIs and other data fields.
  return {
    requestedAt: Date.now(),
    weather: weatherUI.value,
    forecast: forecastUI.value,
  };
}
```

The client side code is similar to the previous example, but the [tool call](/docs/ai-sdk-core/tools-and-tool-calling) will return the new data structure with the weather and forecast UIs. Depending on the speed of getting weather and forecast data, these two components might be updated independently.

## Nested Streamable UIs

You can stream UI components within other UI components. This allows you to create complex UIs that are built up from smaller, reusable components. In the example below, we pass a `historyChart` streamable as a prop to a `StockCard` component. The StockCard can render the `historyChart` streamable, and it will automatically update as the server responds with new data.

```tsx file='app/actions.tsx'
async function getStockHistoryChart({ symbol: string }) {
  'use server';

  const ui = createStreamableUI(<Spinner />);

  // We need to wrap this in an async IIFE to avoid blocking.
  (async () => {
    const price = await getStockPrice({ symbol });

    // Show a spinner as the history chart for now.
    const historyChart = createStreamableUI(<Spinner />);
    ui.done(<StockCard historyChart={historyChart.value} price={price} />);

    // Getting the history data and then update that part of the UI.
    const historyData = await fetch('https://my-stock-data-api.com');
    historyChart.done(<HistoryChart data={historyData} />);
  })();

  return ui;
}
```

---
title: Rate Limiting
description: Learn how to rate limit your application.
---

# Rate Limiting

Rate limiting helps you protect your APIs from abuse. It involves setting a
maximum threshold on the number of requests a client can make within a
specified timeframe. This simple technique acts as a gatekeeper,
preventing excessive usage that can degrade service performance and incur
unnecessary costs.

## Rate Limiting with Vercel KV and Upstash Ratelimit

In this example, you will protect an API endpoint using [Vercel KV](https://vercel.com/storage/kv)
and [Upstash Ratelimit](https://github.com/upstash/ratelimit).

```tsx filename='app/api/generate/route.ts'
import kv from '@vercel/kv';
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { Ratelimit } from '@upstash/ratelimit';
import { NextRequest } from 'next/server';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

// Create Rate limit
const ratelimit = new Ratelimit({
  redis: kv,
  limiter: Ratelimit.fixedWindow(5, '30s'),
});

export async function POST(req: NextRequest) {
  // call ratelimit with request ip
  const ip = req.ip ?? 'ip';
  const { success, remaining } = await ratelimit.limit(ip);

  // block the request if unsuccessfull
  if (!success) {
    return new Response('Ratelimited!', { status: 429 });
  }

  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-3.5-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

## Simplify API Protection

With Vercel KV and Upstash Ratelimit, it is possible to protect your APIs
from such attacks with ease. To learn more about how Ratelimit works and
how it can be configured to your needs, see [Ratelimit Documentation](https://upstash.com/docs/oss/sdks/ts/ratelimit/overview).

---
title: Rendering UI with Language Models
description: Rendering UI with Language Models
---

# Rendering User Interfaces with Language Models

Language models generate text, so at first it may seem like you would only need to render text in your application.

```tsx highlight="16" filename="app/actions.tsx"
const text = generateText({
  model: openai('gpt-3.5-turbo'),
  system: 'You are a friendly assistant',
  prompt: 'What is the weather in SF?',
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      parameters: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in'),
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit });
        return `It is currently ${weather.value}°${unit} and ${weather.description} in ${city}!`;
      },
    },
  },
});
```

Above, the language model is passed a [tool](/docs/ai-sdk-core/tools-and-tool-calling) called `getWeather` that returns the weather information as text. However, instead of returning text, if you return a JSON object that represents the weather information, you can use it to render a React component instead.

```tsx highlight="18-23" filename="app/action.ts"
const text = generateText({
  model: openai('gpt-3.5-turbo'),
  system: 'You are a friendly assistant',
  prompt: 'What is the weather in SF?',
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      parameters: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in'),
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit });
        const { temperature, unit, description, forecast } = weather;

        return {
          temperature,
          unit,
          description,
          forecast,
        };
      },
    },
  },
});
```

Now you can use the object returned by the `getWeather` function to conditionally render a React component `<WeatherCard/>` that displays the weather information by passing the object as props.

```tsx filename="app/page.tsx"
return (
  <div>
    {messages.map(message => {
      if (message.role === 'function') {
        const { name, content } = message
        const { temperature, unit, description, forecast } = content;

        return (
          <WeatherCard
            weather={{
              temperature: 47,
              unit: 'F',
              description: 'sunny'
              forecast,
            }}
          />
        )
      }
    })}
  </div>
)
```

Here's a little preview of what that might look like.

<div className="not-prose flex flex-col2">
  <CardPlayer
    type="weather"
    title="Weather"
    description="An example of an assistant that renders the weather information in a streamed component."
  />
</div>

Rendering interfaces as part of language model generations elevates the user experience of your application, allowing people to interact with language models beyond text.

They also make it easier for you to interpret [sequential tool calls](/docs/ai-sdk-rsc/multistep-interfaces) that take place in multiple steps and help identify and debug where the model reasoned incorrectly.

## Rendering Multiple User Interfaces

To recap, an application has to go through the following steps to render user interfaces as part of model generations:

1. The user prompts the language model.
2. The language model generates a response that includes a tool call.
3. The tool call returns a JSON object that represents the user interface.
4. The response is sent to the client.
5. The client receives the response and checks if the latest message was a tool call.
6. If it was a tool call, the client renders the user interface based on the JSON object returned by the tool call.

Most applications have multiple tools that are called by the language model, and each tool can return a different user interface.

For example, a tool that searches for courses can return a list of courses, while a tool that searches for people can return a list of people. As this list grows, the complexity of your application will grow as well and it can become increasingly difficult to manage these user interfaces.

```tsx filename='app/page.tsx'
{
  message.role === 'tool' ? (
    message.name === 'api-search-course' ? (
      <Courses courses={message.content} />
    ) : message.name === 'api-search-profile' ? (
      <People people={message.content} />
    ) : message.name === 'api-meetings' ? (
      <Meetings meetings={message.content} />
    ) : message.name === 'api-search-building' ? (
      <Buildings buildings={message.content} />
    ) : message.name === 'api-events' ? (
      <Events events={message.content} />
    ) : message.name === 'api-meals' ? (
      <Meals meals={message.content} />
    ) : null
  ) : (
    <div>{message.content}</div>
  );
}
```

## Rendering User Interfaces on the Server

The **AI SDK RSC (`ai/rsc`)** takes advantage of RSCs to solve the problem of managing all your React components on the client side, allowing you to render React components on the server and stream them to the client.

Rather than conditionally rendering user interfaces on the client based on the data returned by the language model, you can directly stream them from the server during a model generation.

```tsx highlight="3,22-31,38" filename="app/action.ts"
import { createStreamableUI } from 'ai/rsc'

const uiStream = createStreamableUI();

const text = generateText({
  model: openai('gpt-3.5-turbo'),
  system: 'you are a friendly assistant'
  prompt: 'what is the weather in SF?'
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      parameters: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in')
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit })
        const { temperature, unit, description, forecast } = weather

        uiStream.done(
          <WeatherCard
            weather={{
              temperature: 47,
              unit: 'F',
              description: 'sunny'
              forecast,
            }}
          />
        )
      }
    }
  }
})

return {
  display: uiStream.value
}
```

The [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) function belongs to the `ai/rsc` module and creates a stream that can send React components to the client.

On the server, you render the `<WeatherCard/>` component with the props passed to it, and then stream it to the client. On the client side, you only need to render the UI that is streamed from the server.

```tsx filename="app/page.tsx" highlight="4"
return (
  <div>
    {messages.map(message => (
      <div>{message.display}</div>
    ))}
  </div>
);
```

Now the steps involved are simplified:

1. The user prompts the language model.
2. The language model generates a response that includes a tool call.
3. The tool call renders a React component along with relevant props that represent the user interface.
4. The response is streamed to the client and rendered directly.

> **Note:** You can also render text on the server and stream it to the client using React Server Components. This way, all operations from language model generation to UI rendering can be done on the server, while the client only needs to render the UI that is streamed from the server.

Check out this [example](/examples/next-app/interface/stream-component-updates) for a full illustration of how to stream component updates with React Server Components in Next.js App Router.

---
title: Language Models as Routers
description: Generative User Interfaces and Language Models as Routers
---

# Generative User Interfaces

Since language models can render user interfaces as part of their generations, the resulting model generations are referred to as generative user interfaces.

In this section we will learn more about generative user interfaces and their impact on the way AI applications are built.

## Deterministic Routes and Probabilistic Routing

Generative user interfaces are not deterministic in nature because they depend on the model's generation output. Since these generations are probabilistic in nature, it is possible for every user query to result in a different user interface.

Users expect their experience using your application to be predictable, so non-deterministic user interfaces can sound like a bad idea at first. However, language models can be set up to limit their generations to a particular set of outputs using their ability to call functions.

When language models are provided with a set of function definitions and instructed to execute any of them based on user query, they do either one of the following things:

- Execute a function that is most relevant to the user query.
- Not execute any function if the user query is out of bounds of the set of functions available to them.

```tsx filename='app/actions.ts'
const sendMessage = (prompt: string) =>
  generateText({
    model: 'gpt-3.5-turbo',
    system: 'you are a friendly weather assistant!',
    prompt,
    tools: {
      getWeather: {
        description: 'Get the weather in a location',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }: { location: string }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      },
    },
  });

sendMessage('What is the weather in San Francisco?'); // getWeather is called
sendMessage('What is the weather in New York?'); // getWeather is called
sendMessage('What events are happening in London?'); // No function is called
```

This way, it is possible to ensure that the generations result in deterministic outputs, while the choice a model makes still remains to be probabilistic.

This emergent ability exhibited by a language model to choose whether a function needs to be executed or not based on a user query is believed to be models emulating "reasoning".

As a result, the combination of language models being able to reason which function to execute as well as render user interfaces at the same time gives you the ability to build applications where language models can be used as a router.

## Language Models as Routers

Historically, developers had to write routing logic that connected different parts of an application to be navigable by a user and complete a specific task.

In web applications today, most of the routing logic takes place in the form of routes:

- `/login` would navigate you to a page with a login form.
- `/user/john` would navigate you to a page with profile details about John.
- `/api/events?limit=5` would display the five most recent events from an events database.

While routes help you build web applications that connect different parts of an application into a seamless user experience, it can also be a burden to manage them as the complexity of applications grow.

Next.js has helped reduce complexity in developing with routes by introducing:

- File-based routing system
- Dynamic routing
- API routes
- Middleware
- App router, and so on...

With language models becoming better at reasoning, we believe that there is a future where developers only write core application specific components while models take care of routing them based on the user's state in an application.

With generative user interfaces, the language model decides which user interface to render based on the user's state in the application, giving users the flexibility to interact with your application in a conversational manner instead of navigating through a series of predefined routes.

### Routing by parameters

For routes like:

- `/profile/[username]`
- `/search?q=[query]`
- `/media/[id]`

that have segments dependent on dynamic data, the language model can generate the correct parameters and render the user interface.

For example, when you're in a search application, you can ask the language model to search for artworks from different artists. The language model will call the search function with the artist's name as a parameter and render the search results.

<div className="not-prose">
  <CardPlayer
    type="media-search"
    title="Media Search"
    description="Let your users see more than words can say by rendering components directly within your search experience."
  />
</div>

### Routing by sequence

For actions that require a sequence of steps to be completed by navigating through different routes, the language model can generate the correct sequence of routes to complete in order to fulfill the user's request.

For example, when you're in a calendar application, you can ask the language model to schedule a happy hour evening with your friends. The language model will then understand your request and will perform the right sequence of [tool calls](/docs/ai-sdk-core/tools-and-tool-calling) to:

1. Lookup your calendar
2. Lookup your friends' calendars
3. Determine the best time for everyone
4. Search for nearby happy hour spots
5. Create an event and send out invites to your friends

<div className="not-prose">
  <CardPlayer
    type="event-planning"
    title="Planning an Event"
    description="The model calls functions and generates interfaces based on user intent, acting like a router."
  />
</div>

Just by defining functions to lookup contacts, pull events from a calendar, and search for nearby locations, the model is able to sequentially navigate the routes for you.

To learn more, check out these [examples](/examples/next-app/interface) using the `streamUI` function to stream generative user interfaces to the client based on the response from the language model.

---
title: Multistep Interfaces
description: Concepts behind building multistep interfaces
---

# Multistep Interfaces

Multistep interfaces refer to user interfaces that require multiple independent steps to be executed in order to complete a specific task.

In order to understand multistep interfaces, it is important to understand two concepts:

- Tool composition
- Application context

**Tool composition** is the process of combining multiple [tools](/docs/ai-sdk-core/tools-and-tool-calling) to create a new tool. This is a powerful concept that allows you to break down complex tasks into smaller, more manageable steps.

**Application context** refers to the state of the application at any given point in time. This includes the user's input, the output of the language model, and any other relevant information.

When designing multistep interfaces, you need to consider how the tools in your application can be composed together to form a coherent user experience as well as how the application context changes as the user progresses through the interface.

## Application Context

The application context can be thought of as the conversation history between the user and the language model. The richer the context, the more information the model has to generate relevant responses.

In the context of multistep interfaces, the application context becomes even more important. This is because **the user's input in one step may affect the output of the model in the next step**.

For example, consider a meal logging application that helps users track their daily food intake. The language model is provided with the following tools:

- `log_meal` takes in parameters like the name of the food, the quantity, and the time of consumption to log a meal.
- `delete_meal` takes in the name of the meal to be deleted.

When the user logs a meal, the model generates a response confirming the meal has been logged.

```txt highlight="2"
User: Log a chicken shawarma for lunch.
Tool: log_meal("chicken shawarma", "250g", "12:00 PM")
Model: Chicken shawarma has been logged for lunch.
```

Now when the user decides to delete the meal, the model should be able to reference the previous step to identify the meal to be deleted.

```txt highlight="7"
User: Log a chicken shawarma for lunch.
Tool: log_meal("chicken shawarma", "250g", "12:00 PM")
Model: Chicken shawarma has been logged for lunch.
...
...
User: I skipped lunch today, can you update my log?
Tool: delete_meal("chicken shawarma")
Model: Chicken shawarma has been deleted from your log.
```

In this example, managing the application context is important for the model to generate the correct response. The model needs to have information about the previous actions in order for it to use generate the parameters for the `delete_meal` tool.

## Tool Composition

Tool composition is the process of combining multiple tools to create a new tool. This involves defining the inputs and outputs of each tool, as well as how they interact with each other.

The design of how these tools can be composed together to form a multistep interface is crucial to both the user experience of your application and the model's ability to generate the correct output.

For example, consider a flight booking assistant that can help users book flights. The assistant can be designed to have the following tools:

- `searchFlights`: Searches for flights based on the user's query.
- `lookupFlight`: Looks up details of a specific flight based on the flight number.
- `bookFlight`: Books a flight based on the user's selection.

The `searchFlights` tool is called when the user wants to lookup flights for a specific route. This would typically mean the tool should be able to take in parameters like the origin and destination of the flight.

The `lookupFlight` tool is called when the user wants to get more details about a specific flight. This would typically mean the tool should be able to take in parameters like the flight number and return information about seat availability.

The `bookFlight` tool is called when the user decides to book a flight. In order to identify the flight to book, the tool should be able to take in parameters like the flight number, trip date, and passenger details.

So the conversation between the user and the model could look like this:

```txt highlight="8"
User: I want to book a flight from New York to London.
Tool: searchFlights("New York", "London")
Model: Here are the available flights from New York to London.
User: I want to book flight number BA123 on 12th December for myself and my wife.
Tool: lookupFlight("BA123") -> "4 seats available"
Model: Sure, there are seats available! Can you provide the names of the passengers?
User: John Doe and Jane Doe.
Tool: bookFlight("BA123", "12th December", ["John Doe", "Jane Doe"])
Model: Your flight has been booked!
```

In the last tool call, the `bookFlight` tool does not include passenger details as it is not available in the application context. As a result, it requests the user to provide the passenger details before proceeding with the booking.

Looking up passenger information could've been another tool that the model could've called before calling the `bookFlight` tool assuming that the user is logged into the application. This way, the model does not have to ask the user for the passenger details and can proceed with the booking.

```txt highlight="5,6"
User: I want to book a flight from New York to London.
Tool: searchFlights("New York", "London")
Model: Here are the available flights from New York to London.
User: I want to book flight number BA123 on 12th December for myself an my wife.
Tool: lookupContacts() -> ["John Doe", "Jane Doe"]
Tool: bookFlight("BA123", "12th December", ["John Doe", "Jane Doe"])
Model: Your flight has been booked!
```

The `lookupContacts` tool is called before the `bookFlight` tool to ensure that the passenger details are available in the application context when booking the flight. This way, the model can reduce the number of steps required from the user and use its ability to call tools that populate its context and use that information to complete the booking process.

Now, let's introduce another tool called `lookupBooking` that can be used to show booking details by taking in the name of the passenger as parameter. This tool can be composed with the existing tools to provide a more complete user experience.

```txt highlight="2-4"
User: What's the status of my wife's upcoming flight?
Tool: lookupContacts() -> ["John Doe", "Jane Doe"]
Tool: lookupBooking("Jane Doe") -> "BA123 confirmed"
Tool: lookupFlight("BA123") -> "Flight BA123 is scheduled to depart on 12th December."
Model: Your wife's flight BA123 is confirmed and scheduled to depart on 12th December.
```

In this example, the `lookupBooking` tool is used to provide the user with the status of their wife's upcoming flight. By composing this tool with the existing tools, the model is able to generate a response that includes the booking status and the departure date of the flight without requiring the user to provide additional information.

As a result, the more tools you design that can be composed together, the more complex and powerful your application can become.

---
title: Sequential Generations
description: Learn how to implement sequential generations ("chains") with the AI SDK
---

# Sequential Generations

When working with the AI SDK, you may want to create sequences of generations (often referred to as "chains" or "pipes"), where the output of one becomes the input for the next. This can be useful for creating more complex AI-powered workflows or for breaking down larger tasks into smaller, more manageable steps.

## Example

In a sequential chain, the output of one generation is directly used as input for the next generation. This allows you to create a series of dependent generations, where each step builds upon the previous one.

Here's an example of how you can implement sequential actions:

```typescript
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

async function sequentialActions() {
  // Generate blog post ideas
  const ideasGeneration = await generateText({
    model: openai('gpt-4o'),
    prompt: 'Generate 10 ideas for a blog post about making spaghetti.',
  });

  console.log('Generated Ideas:\n', ideasGeneration);

  // Pick the best idea
  const bestIdeaGeneration = await generateText({
    model: openai('gpt-4o'),
    prompt: `Here are some blog post ideas about making spaghetti:
${ideasGeneration}

Pick the best idea from the list above and explain why it's the best.`,
  });

  console.log('\nBest Idea:\n', bestIdeaGeneration);

  // Generate an outline
  const outlineGeneration = await generateText({
    model: openai('gpt-4o'),
    prompt: `We've chosen the following blog post idea about making spaghetti:
${bestIdeaGeneration}

Create a detailed outline for a blog post based on this idea.`,
  });

  console.log('\nBlog Post Outline:\n', outlineGeneration);
}

sequentialActions().catch(console.error);
```

In this example, we first generate ideas for a blog post, then pick the best idea, and finally create an outline based on that idea. Each step uses the output from the previous step as input for the next generation.

---
title: Vercel Deployment Guide
description: Learn how to deploy an AI application to production on Vercel
---

# Vercel Deployment Guide

In this guide, you will deploy an AI application to [Vercel](https://vercel.com) using [Next.js](https://nextjs.org) (App Router).

Vercel is a platform for developers that provides the tools, workflows, and infrastructure you need to build and deploy your web apps faster, without the need for additional configuration.

Vercel allows for automatic deployments on every branch push and merges onto the production branch of your GitHub, GitLab, and Bitbucket projects. It is a great option for deploying your AI application.

## Before You Begin

To follow along with this guide, you will need:

- a Vercel account
- an account with a Git provider (this tutorial will use [Github](https://github.com))
- an OpenAI API key

This guide will teach you how to deploy the application you built in the Next.js (App Router) quickstart tutorial to Vercel. If you haven’t completed the quickstart guide, you can start with [this repo](https://github.com/vercel-labs/ai-sdk-deployment-guide).

## Commit Changes

Vercel offers a powerful git-centered workflow that automatically deploys your application to production every time you push to your repository’s main branch.

Before committing your local changes, make sure that you have a `.gitignore`. Within your `.gitignore`, ensure that you are excluding your environment variables (`.env`) and your node modules (`node_modules`).

If you have any local changes, you can commit them by running the following commands:

```bash
git add .
git commit -m "init"
```

## Create Git Repo

You can create a GitHub repository from within your terminal, or on [github.com](https://github.com/). For this tutorial, you will use the GitHub CLI ([more info here](https://cli.github.com/)).

To create your GitHub repository:

1. Navigate to [github.com](http://github.com/)
2. In the top right corner, click the "plus" icon and select "New repository"
3. Pick a name for your repository (this can be anything)
4. Click "Create repository"

Once you have created your repository, GitHub will redirect you to your new repository.

1. Scroll down the page and copy the commands under the title "...or push an existing repository from the command line"
2. Go back to the terminal, paste and then run the commands

Note: if you run into the error "error: remote origin already exists.", this is because your local repository is still linked to the repository you cloned. To "unlink", you can run the following command:

```bash
rm -rf .git
git init
git add .
git commit -m "init"
```

Rerun the code snippet from the previous step.

## Import Project in Vercel

On the [New Project](https://vercel.com/new) page, under the **Import Git Repository** section, select the Git provider that you would like to import your project from. Follow the prompts to sign in to your GitHub account.

Once you have signed in, you should see your newly created repository from the previous step in the "Import Git Repository" section. Click the "Import" button next to that project.

### Add Environment Variables

Your application stores uses environment secrets to store your OpenAI API key using a `.env.local` file locally in development. To add this API key to your production deployment, expand the "Environment Variables" section and paste in your `.env.local` file. Vercel will automatically parse your variables and enter them in the appropriate `key:value` format.

### Deploy

Press the **Deploy** button. Vercel will create the Project and deploy it based on the chosen configurations.

### Enjoy the confetti!

To view your deployment, select the Project in the dashboard and then select the **Domain**. This page is now visible to anyone who has the URL.

## Considerations

When deploying an AI application, there are infrastructure-related considerations to be aware of.

### Function Duration

In most cases, you will call the large language model (LLM) on the server. By default, Vercel serverless functions have a maximum duration of 10 seconds on the Hobby Tier. Depending on your prompt, it can take an LLM more than this limit to complete a response. If the response is not resolved within this limit, the server will throw an error.

You can specify the maximum duration of your Vercel function using [route segment config](https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config). To update your maximum duration, add the following route segment config to the top of your route handler or the page which is calling your server action.

```ts
export const maxDuration = 30;
```

You can increase the max duration to 60 seconds on the Hobby Tier. For other tiers, [see the documentation](https://vercel.com/docs/functions/runtimes#max-duration) for limits.

## Security Considerations

Given the high cost of calling an LLM, it's important to have measures in place that can protect your application from abuse.

### Rate Limit

Rate limiting is a method used to regulate network traffic by defining a maximum number of requests that a client can send to a server within a given time frame.

Follow [this guide](https://vercel.com/guides/securing-ai-app-rate-limiting) to add rate limiting to your application.

### Firewall

A firewall helps protect your applications and websites from DDoS attacks and unauthorized access.

[Vercel Firewall](https://vercel.com/docs/security/vercel-firewall) is a set of tools and infrastructure, created specifically with security in mind. It automatically mitigates DDoS attacks and Enterprise teams can get further customization for their site, including dedicated support and custom rules for IP blocking.

## Troubleshooting

- Streaming not working ([App Router](/docs/troubleshooting/common-issues/streaming-not-working-on-vercel) / [Pages Router](/docs/troubleshooting/common-issues/streaming-not-working-on-vercel-pages-router))
- Experiencing [Timeouts](/docs/troubleshooting/common-issues/timeout-on-vercel)

---
title: Advanced
description: Learn how to use advanced functionality within the AI SDK and RSC API.
collapsed: true
---

# Advanced

This section covers advanced topics and concepts for the AI SDK and RSC API. Working with LLMs often requires a different mental model compared to traditional software development.

After these concepts, you should have a better understanding of the paradigms behind the AI SDK and RSC API, and how to use them to build more AI applications.

---
title: generateText
description: API Reference for generateText.
---

# `generateText()`

Generates text and calls tools for a given prompt using a language model.

It is ideal for non-interactive use cases such as automation tasks where you need to write text (e.g. drafting email or summarizing web pages) and for agents that use tools.

```ts
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const { text } = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Invent a new holiday and describe its traditions.',
});

console.log(text);
```

To see `generateText` in action, check out [these examples](#examples).

## Import

<Snippet text={`import { generateText } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModel',
      description: "The language model to use. Example: openai('gpt-4-turbo')",
    },
    {
      name: 'system',
      type: 'string',
      description:
        'The system prompt to use that specifies the behavior of the model.',
    },
    {
      name: 'prompt',
      type: 'string',
      description: 'The input prompt to generate the text from.',
    },
    {
      name: 'messages',
      type: 'Array<CoreSystemMessage | CoreUserMessage | CoreAssistantMessage | CoreToolMessage> | Array<UIMessage>',
      description:
        'A list of messages that represent a conversation. Automatically converts UI messages from the useChat hook.',
      properties: [
        {
          type: 'CoreSystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'CoreUserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ImagePart | FilePart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ImagePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'image',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The image content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      isOptional: true,
                      description: 'The mime type of the image. Optional.',
                    },
                  ],
                },
                {
                  type: 'FilePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'file'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'data',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The file content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      description: 'The mime type of the file.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreAssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ToolCallPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ToolCallPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on zod schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'Array<ToolResultPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'ToolResultPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'The id of the tool call the result corresponds to.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool the result corresponds to.',
                    },
                    {
                      name: 'result',
                      type: 'unknown',
                      description:
                        'The result returned by the tool after execution.',
                    },
                    {
                      name: 'isError',
                      type: 'boolean',
                      isOptional: true,
                      description:
                        'Whether the result is an error or an error message.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'tools',
      type: 'Record<string, CoreTool>',
      description:
        'Tools that are accessible to and can be called by the model. The model needs to support calling tools.',
      properties: [
        {
          type: 'CoreTool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.',
            },
            {
              name: 'parameters',
              type: 'Zod Schema | JSON Schema',
              description:
                'The schema of the input that the tool expects. The language model will use this to generate the input. It is also used to validate the output of the language model. Use descriptions to make the input understandable for the language model. You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function).',
            },
            {
              name: 'execute',
              isOptional: true,
              type: 'async (parameters: T, options: { abortSignal: AbortSignal }) => JSONValue',
              description:
                'An async function that is called with the arguments from the tool call and produces a result. If not provided, the tool will not be executed automatically.',
            },
          ],
        },
      ],
    },
    {
      name: 'toolChoice',
      isOptional: true,
      type: '"auto" | "none" | "required" | { "type": "tool", "toolName": string }',
      description:
        'The tool choice setting. It specifies how tools are selected for execution. The default is "auto". "none" disables tool execution. "required" requires tools to be executed. { "type": "tool", "toolName": string } specifies a specific tool to execute.',
    },
    {
      name: 'maxTokens',
      type: 'number',
      isOptional: true,
      description: 'Maximum number of tokens to generate.',
    },
    {
      name: 'temperature',
      type: 'number',
      isOptional: true,
      description:
        'Temperature setting. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topP',
      type: 'number',
      isOptional: true,
      description:
        'Nucleus sampling. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topK',
      type: 'number',
      isOptional: true,
      description:
        'Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Recommended for advanced use cases only. You usually only need to use temperature.',
    },
    {
      name: 'presencePenalty',
      type: 'number',
      isOptional: true,
      description:
        'Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'frequencyPenalty',
      type: 'number',
      isOptional: true,
      description:
        'Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'stopSequences',
      type: 'string[]',
      isOptional: true,
      description:
        'Sequences that will stop the generation of the text. If the model generates any of these sequences, it will stop generating further text.',
    },
    {
      name: 'seed',
      type: 'number',
      isOptional: true,
      description:
        'The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'maxSteps',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of sequential LLM calls (steps), e.g. when you use tool calls. A maximum number is required to prevent infinite loops in the case of misconfigured tools. By default, it is set to 1.',
    },
    {
      name: 'experimental_continueSteps',
      type: 'boolean',
      isOptional: true,
      description: 'Enable or disable continue steps. Disabled by default.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      isOptional: true,
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'experimental_activeTools',
      type: 'Array<TOOLNAME> | undefined',
      isOptional: true,
      description:
        'The tools that are currently active. All tools are active by default.',
    },
    {
      name: 'onStepFinish',
      type: '(result: onStepFinishResult) => Promise<void> | void',
      isOptional: true,
      description: 'Callback that is called when a step is finished.',
      properties: [
        {
          type: 'onStepFinishResult',
          parameters: [
            {
              name: 'stepType',
              type: '"initial" | "continue" | "tool-result"',
              description:
                'The type of step. The first step is always an "initial" step, and subsequent steps are either "continue" steps or "tool-result" steps.',
            },
            {
              name: 'finishReason',
              type: '"stop" | "length" | "content-filter" | "tool-calls" | "error" | "other" | "unknown"',
              description:
                'The reason the model finished generating the text for the step.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the step.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'text',
              type: 'string',
              description: 'The full text that has been generated.',
            },
            {
              name: 'toolCalls',
              type: 'ToolCall[]',
              description: 'The tool calls that have been executed.',
            },
            {
              name: 'toolResults',
              type: 'ToolResult[]',
              description: 'The tool results that have been generated.',
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'isContinued',
              type: 'boolean',
              description:
                'True when there will be a continuation step with a continuation text.',
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              isOptional: true,
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'text',
      type: 'string',
      description: 'The generated text by the model.',
    },
    {
      name: 'toolCalls',
      type: 'array',
      description: 'A list of tool calls made by the model.',
    },
    {
      name: 'toolResults',
      type: 'array',
      description:
        'A list of tool results returned as responses to earlier tool calls.',
    },
    {
      name: 'finishReason',
      type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
      description: 'The reason the model finished generating the text.',
    },
    {
      name: 'usage',
      type: 'CompletionTokenUsage',
      description: 'The token usage of the generated text.',
      properties: [
        {
          type: 'CompletionTokenUsage',
          parameters: [
            {
              name: 'promptTokens',
              type: 'number',
              description: 'The total number of tokens in the prompt.',
            },
            {
              name: 'completionTokens',
              type: 'number',
              description: 'The total number of tokens in the completion.',
            },
            {
              name: 'totalTokens',
              type: 'number',
              description: 'The total number of tokens generated.',
            },
          ],
        },
      ],
    },
    {
      name: 'request',
      type: 'RequestMetadata',
      isOptional: true,
      description: 'Request metadata.',
      properties: [
        {
          type: 'RequestMetadata',
          parameters: [
            {
              name: 'body',
              type: 'string',
              description:
                'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
            },
          ],
        },
      ],
    },
    {
      name: 'response',
      type: 'ResponseMetadata',
      isOptional: true,
      description: 'Response metadata.',
      properties: [
        {
          type: 'ResponseMetadata',
          parameters: [
            {
              name: 'id',
              type: 'string',
              description:
                'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
            },
            {
              name: 'model',
              type: 'string',
              description:
                'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
            },
            {
              name: 'timestamp',
              type: 'Date',
              description:
                'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
            },
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Optional response headers.',
            },
            {
              name: 'messages',
              type: 'Array<CoreAssistantMessage | CoreToolMessage>',
              description:
                'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately.',
            },
          ],
        },
      ],
    },
    {
      name: 'warnings',
      type: 'Warning[] | undefined',
      description:
        'Warnings from the model provider (e.g. unsupported settings).',
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'steps',
      type: 'Array<StepResult>',
      description:
        'Response information for every step. You can use this to get information about intermediate steps, such as the tool calls or the response headers.',
      properties: [
        {
          type: 'StepResult',
          parameters: [
            {
              name: 'stepType',
              type: '"initial" | "continue" | "tool-result"',
              description:
                'The type of step. The first step is always an "initial" step, and subsequent steps are either "continue" steps or "tool-result" steps.',
            },
            {
              name: 'text',
              type: 'string',
              description: 'The generated text by the model.',
            },
            {
              name: 'toolCalls',
              type: 'array',
              description: 'A list of tool calls made by the model.',
            },
            {
              name: 'toolResults',
              type: 'array',
              description:
                'A list of tool results returned as responses to earlier tool calls.',
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'CompletionTokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'CompletionTokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'request',
              type: 'RequestMetadata',
              isOptional: true,
              description: 'Request metadata.',
              properties: [
                {
                  type: 'RequestMetadata',
                  parameters: [
                    {
                      name: 'body',
                      type: 'string',
                      description:
                        'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
                    },
                  ],
                },
              ],
            },
            {
              name: 'response',
              type: 'ResponseMetadata',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'ResponseMetadata',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                    {
                      name: 'messages',
                      type: 'Array<CoreAssistantMessage | CoreToolMessage>',
                      description:
                        'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'isContinued',
              type: 'boolean',
              description:
                'True when there will be a continuation step with a continuation text.',
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              isOptional: true,
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
          ],
        },
      ],
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to generate text using a language model in Next.js',
      link: '/examples/next-app/basics/generating-text',
    },
    {
      title:
        'Learn to generate a chat completion using a language model in Next.js',
      link: '/examples/next-app/basics/generating-text',
    },
    {
      title: 'Learn to call tools using a language model in Next.js',
      link: '/examples/next-app/tools/call-tool',
    },
    {
      title:
        'Learn to render a React component as a tool call using a language model in Next.js',
      link: '/examples/next-app/tools/render-interface-during-tool-call',
    },
    {
      title: 'Learn to generate text using a language model in Node.js',
      link: '/examples/node/generating-text/generate-text',
    },
    {
      title:
        'Learn to generate chat completions using a language model in Node.js',
      link: '/examples/node/generating-text/generate-text-with-chat-prompt',
    },
  ]}
/>

---
title: streamText
description: API Reference for streamText.
---

# `streamText()`

Streams text generations from a language model.

You can use the streamText function for interactive use cases such as chat bots and other real-time applications. You can also generate UI components with tools.

```ts
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

const { textStream } = streamText({
  model: openai('gpt-4-turbo'),
  prompt: 'Invent a new holiday and describe its traditions.',
});

for await (const textPart of textStream) {
  process.stdout.write(textPart);
}
```

To see `streamText` in action, check out [these examples](#examples).

## Import

<Snippet text={`import { streamText } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModel',
      description: "The language model to use. Example: openai('gpt-4-turbo')",
    },
    {
      name: 'system',
      type: 'string',
      description:
        'The system prompt to use that specifies the behavior of the model.',
    },
    {
      name: 'prompt',
      type: 'string',
      description: 'The input prompt to generate the text from.',
    },
    {
      name: 'messages',
      type: 'Array<CoreSystemMessage | CoreUserMessage | CoreAssistantMessage | CoreToolMessage> | Array<UIMessage>',
      description:
        'A list of messages that represent a conversation. Automatically converts UI messages from the useChat hook.',
      properties: [
        {
          type: 'CoreSystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'CoreUserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ImagePart | FilePart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ImagePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'image',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The image content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      isOptional: true,
                      description: 'The mime type of the image. Optional.',
                    },
                  ],
                },
                {
                  type: 'FilePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'file'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'data',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The file content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      description: 'The mime type of the file.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreAssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ToolCallPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ToolCallPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on zod schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'Array<ToolResultPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'ToolResultPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'The id of the tool call the result corresponds to.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool the result corresponds to.',
                    },
                    {
                      name: 'result',
                      type: 'unknown',
                      description:
                        'The result returned by the tool after execution.',
                    },
                    {
                      name: 'isError',
                      type: 'boolean',
                      isOptional: true,
                      description:
                        'Whether the result is an error or an error message.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'tools',
      type: 'Record<string, CoreTool>',
      description:
        'Tools that are accessible to and can be called by the model. The model needs to support calling tools.',
      properties: [
        {
          type: 'CoreTool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.',
            },
            {
              name: 'parameters',
              type: 'Zod Schema | JSON Schema',
              description:
                'The schema of the input that the tool expects. The language model will use this to generate the input. It is also used to validate the output of the language model. Use descriptions to make the input understandable for the language model. You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function).',
            },
            {
              name: 'execute',
              isOptional: true,
              type: 'async (parameters: T, options: { abortSignal: AbortSignal }) => JSONValue',
              description:
                'An async function that is called with the arguments from the tool call and produces a result. If not provided, the tool will not be executed automatically.',
            },
          ],
        },
      ],
    },
    {
      name: 'toolChoice',
      isOptional: true,
      type: '"auto" | "none" | "required" | { "type": "tool", "toolName": string }',
      description:
        'The tool choice setting. It specifies how tools are selected for execution. The default is "auto". "none" disables tool execution. "required" requires tools to be executed. { "type": "tool", "toolName": string } specifies a specific tool to execute.',
    },
    {
      name: 'maxTokens',
      type: 'number',
      isOptional: true,
      description: 'Maximum number of tokens to generate.',
    },
    {
      name: 'temperature',
      type: 'number',
      isOptional: true,
      description:
        'Temperature setting. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topP',
      type: 'number',
      isOptional: true,
      description:
        'Nucleus sampling. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topK',
      type: 'number',
      isOptional: true,
      description:
        'Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Recommended for advanced use cases only. You usually only need to use temperature.',
    },
    {
      name: 'presencePenalty',
      type: 'number',
      isOptional: true,
      description:
        'Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'frequencyPenalty',
      type: 'number',
      isOptional: true,
      description:
        'Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'stopSequences',
      type: 'string[]',
      isOptional: true,
      description:
        'Sequences that will stop the generation of the text. If the model generates any of these sequences, it will stop generating further text.',
    },
    {
      name: 'seed',
      type: 'number',
      isOptional: true,
      description:
        'The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'maxSteps',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of sequential LLM calls (steps), e.g. when you use tool calls. A maximum number is required to prevent infinite loops in the case of misconfigured tools. By default, it is set to 1.',
    },
    {
      name: 'experimental_continueSteps',
      type: 'boolean',
      isOptional: true,
      description: 'Enable or disable continue steps. Disabled by default.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_toolCallStreaming',
      type: 'boolean',
      isOptional: true,
      description:
        'Enable streaming of tool call deltas as they are generated. Disabled by default.',
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      isOptional: true,
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'experimental_activeTools',
      type: 'Array<TOOLNAME> | undefined',
      isOptional: true,
      description:
        'The tools that are currently active. All tools are active by default.',
    },
    {
      name: 'onChunk',
      type: '(event: OnChunkResult) => Promise<void> |void',
      isOptional: true,
      description:
        'Callback that is called for each chunk of the stream. The stream processing will pause until the callback promise is resolved.',
      properties: [
        {
          type: 'OnChunkResult',
          parameters: [
            {
              name: 'chunk',
              type: 'TextStreamPart',
              description: 'The chunk of the stream.',
              properties: [
                {
                  type: 'TextStreamPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text-delta'",
                      description:
                        'The type to identify the object as text delta.',
                    },
                    {
                      name: 'textDelta',
                      type: 'string',
                      description: 'The text delta.',
                    },
                  ],
                },
                {
                  type: 'TextStreamPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description:
                        'The type to identify the object as tool call.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on zod schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
                {
                  type: 'TextStreamPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call-streaming-start'",
                      description:
                        'Indicates the start of a tool call streaming. Only available when streaming tool calls.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                  ],
                },
                {
                  type: 'TextStreamPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call-delta'",
                      description:
                        'The type to identify the object as tool call delta. Only available when streaming tool calls.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'argsTextDelta',
                      type: 'string',
                      description: 'The text delta of the tool call arguments.',
                    },
                  ],
                },
                {
                  type: 'TextStreamPart',
                  description: 'The result of a tool call execution.',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description:
                        'The type to identify the object as tool result.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on zod schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                    {
                      name: 'result',
                      type: 'any',
                      description:
                        'The result returned by the tool after execution has completed.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'onStepFinish',
      type: '(result: onStepFinishResult) => Promise<void> | void',
      isOptional: true,
      description: 'Callback that is called when a step is finished.',
      properties: [
        {
          type: 'onStepFinishResult',
          parameters: [
            {
              name: 'stepType',
              type: '"initial" | "continue" | "tool-result"',
              description:
                'The type of step. The first step is always an "initial" step, and subsequent steps are either "continue" steps or "tool-result" steps.',
            },
            {
              name: 'finishReason',
              type: '"stop" | "length" | "content-filter" | "tool-calls" | "error" | "other" | "unknown"',
              description:
                'The reason the model finished generating the text for the step.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the step.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'text',
              type: 'string',
              description: 'The full text that has been generated.',
            },
            {
              name: 'toolCalls',
              type: 'ToolCall[]',
              description: 'The tool calls that have been executed.',
            },
            {
              name: 'toolResults',
              type: 'ToolResult[]',
              description: 'The tool results that have been generated.',
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'isContinued',
              type: 'boolean',
              description:
                'True when there will be a continuation step with a continuation text.',
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              isOptional: true,
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
          ],
        },
      ],
    },
    {
      name: 'onFinish',
      type: '(result: OnFinishResult) => Promise<void> | void',
      isOptional: true,
      description:
        'Callback that is called when the LLM response and all request tool executions (for tools that have an `execute` function) are finished.',
      properties: [
        {
          type: 'OnFinishResult',
          parameters: [
            {
              name: 'finishReason',
              type: '"stop" | "length" | "content-filter" | "tool-calls" | "error" | "other" | "unknown"',
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
            {
              name: 'text',
              type: 'string',
              description: 'The full text that has been generated.',
            },
            {
              name: 'toolCalls',
              type: 'ToolCall[]',
              description: 'The tool calls that have been executed.',
            },
            {
              name: 'toolResults',
              type: 'ToolResult[]',
              description: 'The tool results that have been generated.',
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                    {
                      name: 'messages',
                      type: 'Array<CoreAssistantMessage | CoreToolMessage>',
                      description:
                        'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'steps',
              type: 'Array<StepResult>',
              description:
                'Response information for every step. You can use this to get information about intermediate steps, such as the tool calls or the response headers.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'finishReason',
      type: "Promise<'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'>",
      description:
        'The reason why the generation finished. Resolved when the response is finished.',
    },
    {
      name: 'usage',
      type: 'Promise<CompletionTokenUsage>',
      description:
        'The token usage of the generated text. Resolved when the response is finished.',
      properties: [
        {
          type: 'CompletionTokenUsage',
          parameters: [
            {
              name: 'promptTokens',
              type: 'number',
              description: 'The total number of tokens in the prompt.',
            },
            {
              name: 'completionTokens',
              type: 'number',
              description: 'The total number of tokens in the completion.',
            },
            {
              name: 'totalTokens',
              type: 'number',
              description: 'The total number of tokens generated.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Promise<Record<string,Record<string,JSONValue>> | undefined>',
      description:
        'Optional metadata from the provider. Resolved whe the response is finished. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'responseMessages',
      type: 'Promise<Array<CoreAssistantMessage | CoreToolMessage>>',
      description:
        'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately. Resolved when the response is finished.',
    },
    {
      name: 'text',
      type: 'Promise<string>',
      description:
        'The full text that has been generated. Resolved when the response is finished.',
    },
    {
      name: 'toolCalls',
      type: 'Promise<ToolCall[]>',
      description:
        'The tool calls that have been executed. Resolved when the response is finished.',
    },
    {
      name: 'toolResults',
      type: 'Promise<ToolResult[]>',
      description:
        'The tool results that have been generated. Resolved when the all tool executions are finished.',
    },
    {
      name: 'request',
      type: 'Promise<RequestMetadata>',
      isOptional: true,
      description: 'Request metadata.',
      properties: [
        {
          type: 'RequestMetadata',
          parameters: [
            {
              name: 'body',
              type: 'string',
              description:
                'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
            },
          ],
        },
      ],
    },
    {
      name: 'response',
      type: 'Promise<ResponseMetadata>',
      isOptional: true,
      description: 'Response metadata. Resolved when the response is finished.',
      properties: [
        {
          type: 'ResponseMetadata',
          parameters: [
            {
              name: 'id',
              type: 'string',
              description:
                'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
            },
            {
              name: 'model',
              type: 'string',
              description:
                'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
            },
            {
              name: 'timestamp',
              type: 'Date',
              description:
                'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
            },
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Optional response headers.',
            },
            {
              name: 'messages',
              type: 'Array<CoreAssistantMessage | CoreToolMessage>',
              description:
                'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately.',
            },
          ],
        },
      ],
    },
    {
      name: 'warnings',
      type: 'Warning[] | undefined',
      description:
        'Warnings from the model provider (e.g. unsupported settings).',
    },
    {
      name: 'steps',
      type: 'Promise<Array<StepResult>>',
      description:
        'Response information for every step. You can use this to get information about intermediate steps, such as the tool calls or the response headers.',
      properties: [
        {
          type: 'StepResult',
          parameters: [
            {
              name: 'stepType',
              type: '"initial" | "continue" | "tool-result"',
              description:
                'The type of step. The first step is always an "initial" step, and subsequent steps are either "continue" steps or "tool-result" steps.',
            },
            {
              name: 'text',
              type: 'string',
              description: 'The generated text by the model.',
            },
            {
              name: 'toolCalls',
              type: 'array',
              description: 'A list of tool calls made by the model.',
            },
            {
              name: 'toolResults',
              type: 'array',
              description:
                'A list of tool results returned as responses to earlier tool calls.',
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'CompletionTokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'CompletionTokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'request',
              type: 'RequestMetadata',
              isOptional: true,
              description: 'Request metadata.',
              properties: [
                {
                  type: 'RequestMetadata',
                  parameters: [
                    {
                      name: 'body',
                      type: 'string',
                      description:
                        'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
                    },
                  ],
                },
              ],
            },
            {
              name: 'response',
              type: 'ResponseMetadata',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'ResponseMetadata',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                    {
                      name: 'messages',
                      type: 'Array<CoreAssistantMessage | CoreToolMessage>',
                      description:
                        'The response messages that were generated during the call. It consists of an assistant message, potentially containing tool calls.  When there are tool results, there is an additional tool message with the tool results that are available. If there are tools that do not have execute functions, they are not included in the tool results and need to be added separately.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'isContinued',
              type: 'boolean',
              description:
                'True when there will be a continuation step with a continuation text.',
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              isOptional: true,
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
          ],
        },
      ],
    },
    {
      name: 'textStream',
      type: 'AsyncIterable<string> & ReadableStream<string>',
      description:
        'A text stream that returns only the generated text deltas. You can use it as either an AsyncIterable or a ReadableStream. When an error occurs, the stream will throw the error.',
    },
    {
      name: 'fullStream',
      type: 'AsyncIterable<TextStreamPart> & ReadableStream<TextStreamPart>',
      description:
        'A stream with all events, including text deltas, tool calls, tool results, and errors. You can use it as either an AsyncIterable or a ReadableStream. Only errors that stop the stream, such as network errors, are thrown.',
      properties: [
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'text-delta'",
              description: 'The type to identify the object as text delta.',
            },
            {
              name: 'textDelta',
              type: 'string',
              description: 'The text delta.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'tool-call'",
              description: 'The type to identify the object as tool call.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
            {
              name: 'toolName',
              type: 'string',
              description:
                'The name of the tool, which typically would be the name of the function.',
            },
            {
              name: 'args',
              type: 'object based on zod schema',
              description:
                'Parameters generated by the model to be used by the tool.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'tool-call-streaming-start'",
              description:
                'Indicates the start of a tool call streaming. Only available when streaming tool calls.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
            {
              name: 'toolName',
              type: 'string',
              description:
                'The name of the tool, which typically would be the name of the function.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'tool-call-delta'",
              description:
                'The type to identify the object as tool call delta. Only available when streaming tool calls.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
            {
              name: 'toolName',
              type: 'string',
              description:
                'The name of the tool, which typically would be the name of the function.',
            },
            {
              name: 'argsTextDelta',
              type: 'string',
              description: 'The text delta of the tool call arguments.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          description: 'The result of a tool call execution.',
          parameters: [
            {
              name: 'type',
              type: "'tool-result'",
              description: 'The type to identify the object as tool result.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
            {
              name: 'toolName',
              type: 'string',
              description:
                'The name of the tool, which typically would be the name of the function.',
            },
            {
              name: 'args',
              type: 'object based on zod schema',
              description:
                'Parameters generated by the model to be used by the tool.',
            },
            {
              name: 'result',
              type: 'any',
              description:
                'The result returned by the tool after execution has completed.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'error'",
              description: 'The type to identify the object as error.',
            },
            {
              name: 'error',
              type: 'Error',
              description:
                'Describes the error that may have occurred during execution.',
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'step-finish'",
              description: 'The type to identify the object as step finish.',
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'isContinued',
              type: 'boolean',
              description:
                'True when there will be a continuation step with a continuation text.',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'TextStreamPart',
          parameters: [
            {
              name: 'type',
              type: "'finish'",
              description: 'The type to identify the object as finish.',
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'pipeDataStreamToResponse',
      type: '(response: ServerResponse, options: PipeDataStreamToResponseOptions } => void',
      description:
        'Writes stream data output to a Node.js response-like object. It sets a `Content-Type` header to `text/plain; charset=utf-8` and writes each stream data part as a separate chunk.',
      properties: [
        {
          type: 'PipeDataStreamToResponseOptions',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'HeadersInit',
              isOptional: true,
              description: 'The response headers.',
            },
            {
              name: 'data',
              type: 'StreamData',
              isOptional: true,
              description: 'The stream data object.',
            },
            {
              name: 'getErrorMessage',
              type: '(error: unknown) => string',
              description:
                'A function to get the error message from the error object. By default, all errors are masked as "" for safety reasons.',
              isOptional: true,
            },
            {
              name: 'sendUsage',
              type: 'boolean',
              isOptional: true,
              description:
                'Whether to send the usage information in the stream. Defaults to true.',
            },
          ],
        },
      ],
    },
    {
      name: 'pipeTextStreamToResponse',
      type: '(response: ServerResponse, init?: ResponseInit => void',
      description:
        'Writes text delta output to a Node.js response-like object. It sets a `Content-Type` header to `text/plain; charset=utf-8` and writes each text delta as a separate chunk.',
      properties: [
        {
          type: 'ResponseInit',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              isOptional: true,
              description: 'The response headers.',
            },
          ],
        },
      ],
    },
    {
      name: 'toDataStream',
      type: '(options?: ToDataStreamOptions) => Response',
      description: 'Converts the result to a data stream.',
      properties: [
        {
          type: 'ToDataStreamOptions',
          parameters: [
            {
              name: 'data',
              type: 'StreamData',
              isOptional: true,
              description: 'The stream data object.',
            },
            {
              name: 'getErrorMessage',
              type: '(error: unknown) => string',
              description:
                'A function to get the error message from the error object. By default, all errors are masked as "" for safety reasons.',
              isOptional: true,
            },
            {
              name: 'sendUsage',
              type: 'boolean',
              isOptional: true,
              description:
                'Whether to send the usage information in the stream. Defaults to true.',
            },
          ],
        },
      ],
    },
    {
      name: 'toDataStreamResponse',
      type: '(options?: ToDataStreamResponseOptions) => Response',
      description:
        'Converts the result to a streamed response object with a stream data part stream. It can be used with the `useChat` and `useCompletion` hooks.',
      properties: [
        {
          type: 'ToDataStreamResponseOptions',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              isOptional: true,
              description: 'The response headers.',
            },
            {
              name: 'data',
              type: 'StreamData',
              isOptional: true,
              description: 'The stream data object.',
            },
            {
              name: 'getErrorMessage',
              type: '(error: unknown) => string',
              description:
                'A function to get the error message from the error object. By default, all errors are masked as "" for safety reasons.',
              isOptional: true,
            },
            {
              name: 'sendUsage',
              type: 'boolean',
              isOptional: true,
              description:
                'Whether to send the usage information in the stream. Defaults to true.',
            },
          ],
        },
      ],
    },
    {
      name: 'toTextStreamResponse',
      type: '(init?: ResponseInit) => Response',
      description:
        'Creates a simple text stream response. Each text delta is encoded as UTF-8 and sent as a separate chunk. Non-text-delta events are ignored.',
      properties: [
        {
          type: 'ResponseInit',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              isOptional: true,
              description: 'The response headers.',
            },
          ],
        },
      ],
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to stream text generated by a language model in Next.js',
      link: '/examples/next-app/basics/streaming-text-generation',
    },
    {
      title:
        'Learn to stream chat completions generated by a language model in Next.js',
      link: '/examples/next-app/chat/stream-chat-completion',
    },
    {
      title: 'Learn to stream text generated by a language model in Node.js',
      link: '/examples/node/generating-text/stream-text',
    },
    {
      title:
        'Learn to stream chat completions generated by a language model in Node.js',
      link: '/examples/node/generating-text/stream-text-with-chat-prompt',
    },
  ]}
/>

---
title: generateObject
description: API Reference for generateObject.
---

# `generateObject()`

Generates a typed, structured object for a given prompt and schema using a language model.

It can be used to force the language model to return structured data, e.g. for information extraction, synthetic data generation, or classification tasks.

#### Example: generate an object using a schema

```ts
import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(JSON.stringify(object, null, 2));
```

#### Example: generate an array using a schema

For arrays, you specify the schema of the array items.

```ts highlight="7"
import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  output: 'array',
  schema: z.object({
    name: z.string(),
    class: z
      .string()
      .describe('Character class, e.g. warrior, mage, or thief.'),
    description: z.string(),
  }),
  prompt: 'Generate 3 hero descriptions for a fantasy role playing game.',
});
```

#### Example: generate an enum

When you want to generate a specific enum value, you can set the output strategy to `enum`
and provide the list of possible values in the `enum` parameter.

```ts highlight="5-6"
import { generateObject } from 'ai';

const { object } = await generateObject({
  model: yourModel,
  output: 'enum',
  enum: ['action', 'comedy', 'drama', 'horror', 'sci-fi'],
  prompt:
    'Classify the genre of this movie plot: ' +
    '"A group of astronauts travel through a wormhole in search of a ' +
    'new habitable planet for humanity."',
});
```

#### Example: generate JSON without a schema

```ts highlight="6"
import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  output: 'no-schema',
  prompt: 'Generate a lasagna recipe.',
});
```

To see `generateObject` in action, check out the [additional examples](#more-examples).

## Import

<Snippet text={`import { generateObject } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModel',
      description: "The language model to use. Example: openai('gpt-4-turbo')",
    },
    {
      name: 'output',
      type: "'object' | 'array' | 'enum' | 'no-schema' | undefined",
      description: "The type of output to generate. Defaults to 'object'.",
    },
    {
      name: 'mode',
      type: "'auto' | 'json' | 'tool'",
      description:
        "The mode to use for object generation. Not every model supports all modes. \
        Defaults to 'auto' for 'object' output and to 'json' for 'no-schema' output. \
        Must be 'json' for 'no-schema' output.",
    },
    {
      name: 'schema',
      type: 'Zod Schema | JSON Schema',
      description:
        "The schema that describes the shape of the object to generate. \
        It is sent to the model to generate the object and used to validate the output. \
        You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function). \
        In 'array' mode, the schema is used to describe an array element. \
        Not available with 'no-schema' or 'enum' output.",
    },
    {
      name: 'schemaName',
      type: 'string | undefined',
      description:
        "Optional name of the output that should be generated. \
        Used by some providers for additional LLM guidance, e.g. via tool or schema name. \
        Not available with 'no-schema' or 'enum' output.",
    },
    {
      name: 'schemaDescription',
      type: 'string | undefined',
      description:
        "Optional description of the output that should be generated. \
        Used by some providers for additional LLM guidance, e.g. via tool or schema name. \
        Not available with 'no-schema' or 'enum' output.",
    },
    {
      name: 'enum',
      type: 'string[]',
      description:
        "List of possible values to generate. \
        Only available with 'enum' output.",
    },
    {
      name: 'system',
      type: 'string',
      description:
        'The system prompt to use that specifies the behavior of the model.',
    },
    {
      name: 'prompt',
      type: 'string',
      description: 'The input prompt to generate the text from.',
    },
    {
      name: 'messages',
      type: 'Array<CoreSystemMessage | CoreUserMessage | CoreAssistantMessage | CoreToolMessage> | Array<UIMessage>',
      description:
        'A list of messages that represent a conversation. Automatically converts UI messages from the useChat hook.',
      properties: [
        {
          type: 'CoreSystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'CoreUserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ImagePart | FilePart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ImagePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'image',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The image content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      isOptional: true,
                      description: 'The mime type of the image. Optional.',
                    },
                  ],
                },
                {
                  type: 'FilePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'file'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'data',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The file content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      description: 'The mime type of the file.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreAssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ToolCallPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ToolCallPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'Array<ToolResultPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'ToolResultPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'The id of the tool call the result corresponds to.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool the result corresponds to.',
                    },
                    {
                      name: 'result',
                      type: 'unknown',
                      description:
                        'The result returned by the tool after execution.',
                    },
                    {
                      name: 'isError',
                      type: 'boolean',
                      isOptional: true,
                      description:
                        'Whether the result is an error or an error message.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'maxTokens',
      type: 'number',
      isOptional: true,
      description: 'Maximum number of tokens to generate.',
    },
    {
      name: 'temperature',
      type: 'number',
      isOptional: true,
      description:
        'Temperature setting. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topP',
      type: 'number',
      isOptional: true,
      description:
        'Nucleus sampling. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topK',
      type: 'number',
      isOptional: true,
      description:
        'Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Recommended for advanced use cases only. You usually only need to use temperature.',
    },
    {
      name: 'presencePenalty',
      type: 'number',
      isOptional: true,
      description:
        'Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'frequencyPenalty',
      type: 'number',
      isOptional: true,
      description:
        'Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'seed',
      type: 'number',
      isOptional: true,
      description:
        'The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      isOptional: true,
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'object',
      type: 'based on the schema',
      description:
        'The generated object, validated by the schema (if it supports validation).',
    },
    {
      name: 'finishReason',
      type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
      description: 'The reason the model finished generating the text.',
    },
    {
      name: 'usage',
      type: 'CompletionTokenUsage',
      description: 'The token usage of the generated text.',
      properties: [
        {
          type: 'CompletionTokenUsage',
          parameters: [
            {
              name: 'promptTokens',
              type: 'number',
              description: 'The total number of tokens in the prompt.',
            },
            {
              name: 'completionTokens',
              type: 'number',
              description: 'The total number of tokens in the completion.',
            },
            {
              name: 'totalTokens',
              type: 'number',
              description: 'The total number of tokens generated.',
            },
          ],
        },
      ],
    },
    {
      name: 'request',
      type: 'RequestMetadata',
      isOptional: true,
      description: 'Request metadata.',
      properties: [
        {
          type: 'RequestMetadata',
          parameters: [
            {
              name: 'body',
              type: 'string',
              description:
                'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
            },
          ],
        },
      ],
    },
    {
      name: 'response',
      type: 'ResponseMetadata',
      isOptional: true,
      description: 'Response metadata.',
      properties: [
        {
          type: 'ResponseMetadata',
          parameters: [
            {
              name: 'id',
              type: 'string',
              description:
                'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
            },
            {
              name: 'model',
              type: 'string',
              description:
                'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
            },
            {
              name: 'timestamp',
              type: 'Date',
              description:
                'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
            },
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Optional response headers.',
            },
          ],
        },
      ],
    },
    {
      name: 'warnings',
      type: 'Warning[] | undefined',
      description:
        'Warnings from the model provider (e.g. unsupported settings).',
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'toJsonResponse',
      type: '(init?: ResponseInit) => Response',
      description:
        'Converts the object to a JSON response. The response will have a status code of 200 and a content type of `application/json; charset=utf-8`.',
    },
  ]}
/>

## More Examples

<ExampleLinks
  examples={[
    {
      title:
        'Learn to generate structured data using a language model in Next.js',
      link: '/examples/next-app/basics/generating-object',
    },
    {
      title:
        'Learn to generate structured data using a language model in Node.js',
      link: '/examples/node/generating-structured-data/generate-object',
    },
  ]}
/>

---
title: streamObject
description: API Reference for streamObject
---

# `streamObject()`

Streams a typed, structured object for a given prompt and schema using a language model.

It can be used to force the language model to return structured data, e.g. for information extraction, synthetic data generation, or classification tasks.

#### Example: stream an object using a schema

```ts
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const { partialObjectStream } = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

for await (const partialObject of partialObjectStream) {
  console.clear();
  console.log(partialObject);
}
```

#### Example: stream an array using a schema

For arrays, you specify the schema of the array items.
You can use `elementStream` to get the stream of complete array elements.

```ts highlight="7,18"
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const { elementStream } = streamObject({
  model: openai('gpt-4-turbo'),
  output: 'array',
  schema: z.object({
    name: z.string(),
    class: z
      .string()
      .describe('Character class, e.g. warrior, mage, or thief.'),
    description: z.string(),
  }),
  prompt: 'Generate 3 hero descriptions for a fantasy role playing game.',
});

for await (const hero of elementStream) {
  console.log(hero);
}
```

#### Example: generate JSON without a schema

```ts
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';

const { partialObjectStream } = streamObject({
  model: openai('gpt-4-turbo'),
  output: 'no-schema',
  prompt: 'Generate a lasagna recipe.',
});

for await (const partialObject of partialObjectStream) {
  console.clear();
  console.log(partialObject);
}
```

To see `streamObject` in action, check out the [additional examples](#more-examples).

## Import

<Snippet text={`import { streamObject } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModel',
      description: "The language model to use. Example: openai('gpt-4-turbo')",
    },
    {
      name: 'output',
      type: "'object' | 'array' | 'no-schema' | undefined",
      description: "The type of output to generate. Defaults to 'object'.",
    },
    {
      name: 'mode',
      type: "'auto' | 'json' | 'tool'",
      description:
        "The mode to use for object generation. Not every model supports all modes. \
        Defaults to 'auto' for 'object' output and to 'json' for 'no-schema' output. \
        Must be 'json' for 'no-schema' output.",
    },
    {
      name: 'schema',
      type: 'Zod Schema | JSON Schema',
      description:
        "The schema that describes the shape of the object to generate. \
        It is sent to the model to generate the object and used to validate the output. \
        You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function). \
        In 'array' mode, the schema is used to describe an array element. \
        Not available with 'no-schema' output.",
    },
    {
      name: 'schemaName',
      type: 'string | undefined',
      description:
        "Optional name of the output that should be generated. \
        Used by some providers for additional LLM guidance, e.g. via tool or schema name. \
        Not available with 'no-schema' output.",
    },
    {
      name: 'schemaDescription',
      type: 'string | undefined',
      description:
        "Optional description of the output that should be generated. \
        Used by some providers for additional LLM guidance, e.g. via tool or schema name. \
        Not available with 'no-schema' output.",
    },
    {
      name: 'system',
      type: 'string',
      description:
        'The system prompt to use that specifies the behavior of the model.',
    },
    {
      name: 'prompt',
      type: 'string',
      description: 'The input prompt to generate the text from.',
    },
    {
      name: 'messages',
      type: 'Array<CoreSystemMessage | CoreUserMessage | CoreAssistantMessage | CoreToolMessage> | Array<UIMessage>',
      description:
        'A list of messages that represent a conversation. Automatically converts UI messages from the useChat hook.',
      properties: [
        {
          type: 'CoreSystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'CoreUserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ImagePart | FilePart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ImagePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'image',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The image content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      isOptional: true,
                      description: 'The mime type of the image. Optional.',
                    },
                  ],
                },
                {
                  type: 'FilePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'file'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'data',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The file content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      description: 'The mime type of the file.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreAssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ToolCallPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ToolCallPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'Array<ToolResultPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'ToolResultPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'The id of the tool call the result corresponds to.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool the result corresponds to.',
                    },
                    {
                      name: 'result',
                      type: 'unknown',
                      description:
                        'The result returned by the tool after execution.',
                    },
                    {
                      name: 'isError',
                      type: 'boolean',
                      isOptional: true,
                      description:
                        'Whether the result is an error or an error message.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'maxTokens',
      type: 'number',
      isOptional: true,
      description: 'Maximum number of tokens to generate.',
    },
    {
      name: 'temperature',
      type: 'number',
      isOptional: true,
      description:
        'Temperature setting. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topP',
      type: 'number',
      isOptional: true,
      description:
        'Nucleus sampling. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topK',
      type: 'number',
      isOptional: true,
      description:
        'Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Recommended for advanced use cases only. You usually only need to use temperature.',
    },
    {
      name: 'presencePenalty',
      type: 'number',
      isOptional: true,
      description:
        'Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'frequencyPenalty',
      type: 'number',
      isOptional: true,
      description:
        'Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'seed',
      type: 'number',
      isOptional: true,
      description:
        'The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      isOptional: true,
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'onFinish',
      type: '(result: OnFinishResult) => void',
      isOptional: true,
      description:
        'Callback that is called when the LLM response has finished.',
      properties: [
        {
          type: 'OnFinishResult',
          parameters: [
            {
              name: 'usage',
              type: 'CompletionTokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'CompletionTokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'experimental_providerMetadata',
              type: 'Record<string,Record<string,JSONValue>> | undefined',
              description:
                'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
            },
            {
              name: 'object',
              type: 'T | undefined',
              description:
                'The generated object (typed according to the schema). Can be undefined if the final object does not match the schema.',
            },
            {
              name: 'error',
              type: 'unknown | undefined',
              description:
                'Optional error object. This is e.g. a TypeValidationError when the final object does not match the schema.',
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Optional response headers.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'usage',
      type: 'Promise<CompletionTokenUsage>',
      description:
        'The token usage of the generated text. Resolved when the response is finished.',
      properties: [
        {
          type: 'CompletionTokenUsage',
          parameters: [
            {
              name: 'promptTokens',
              type: 'number',
              description: 'The total number of tokens in the prompt.',
            },
            {
              name: 'completionTokens',
              type: 'number',
              description: 'The total number of tokens in the completion.',
            },
            {
              name: 'totalTokens',
              type: 'number',
              description: 'The total number of tokens generated.',
            },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Promise<Record<string,Record<string,JSONValue>> | undefined>',
      description:
        'Optional metadata from the provider. Resolved whe the response is finished. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'object',
      type: 'Promise<T>',
      description:
        'The generated object (typed according to the schema). Resolved when the response is finished.',
    },
    {
      name: 'partialObjectStream',
      type: 'AsyncIterableStream<DeepPartial<T>>',
      description:
        'Stream of partial objects. It gets more complete as the stream progresses. Note that the partial object is not validated. If you want to be certain that the actual content matches your schema, you need to implement your own validation for partial results.',
    },
    {
      name: 'elementStream',
      type: 'AsyncIterableStream<ELEMENT>',
      description: 'Stream of array elements. Only available in "array" mode.',
    },
    {
      name: 'textStream',
      type: 'AsyncIterableStream<string>',
      description:
        'Text stream of the JSON representation of the generated object. It contains text chunks. When the stream is finished, the object is valid JSON that can be parsed.',
    },
    {
      name: 'fullStream',
      type: 'AsyncIterableStream<ObjectStreamPart<T>>',
      description:
        'Stream of different types of events, including partial objects, errors, and finish events. Only errors that stop the stream, such as network errors, are thrown.',
      properties: [
        {
          type: 'ObjectPart',
          parameters: [
            {
              name: 'type',
              type: "'object'",
            },
            {
              name: 'object',
              type: 'DeepPartial<T>',
              description: 'The partial object that was generated.',
            },
          ],
        },
        {
          type: 'TextDeltaPart',
          parameters: [
            {
              name: 'type',
              type: "'text-delta'",
            },
            {
              name: 'textDelta',
              type: 'string',
              description: 'The text delta for the underlying raw JSON text.',
            },
          ],
        },
        {
          type: 'ErrorPart',
          parameters: [
            {
              name: 'type',
              type: "'error'",
            },
            {
              name: 'error',
              type: 'unknown',
              description: 'The error that occurred.',
            },
          ],
        },
        {
          type: 'FinishPart',
          parameters: [
            {
              name: 'type',
              type: "'finish'",
            },
            {
              name: 'finishReason',
              type: 'FinishReason',
            },
            {
              name: 'logprobs',
              type: 'Logprobs',
              isOptional: true,
            },
            {
              name: 'usage',
              type: 'Usage',
              description: 'Token usage.',
            },
            {
              name: 'response',
              type: 'Response',
              isOptional: true,
              description: 'Response metadata.',
              properties: [
                {
                  type: 'Response',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description:
                        'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
                    },
                    {
                      name: 'model',
                      type: 'string',
                      description:
                        'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
                    },
                    {
                      name: 'timestamp',
                      type: 'Date',
                      description:
                        'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'request',
      type: 'Promise<RequestMetadata>',
      isOptional: true,
      description: 'Request metadata.',
      properties: [
        {
          type: 'RequestMetadata',
          parameters: [
            {
              name: 'body',
              type: 'string',
              description:
                'Raw request HTTP body that was sent to the provider API as a string (JSON should be stringified).',
            },
          ],
        },
      ],
    },
    {
      name: 'response',
      type: 'Promise<ResponseMetadata>',
      isOptional: true,
      description: 'Response metadata. Resolved when the response is finished.',
      properties: [
        {
          type: 'ResponseMetadata',
          parameters: [
            {
              name: 'id',
              type: 'string',
              description:
                'The response identifier. The AI SDK uses the ID from the provider response when available, and generates an ID otherwise.',
            },
            {
              name: 'model',
              type: 'string',
              description:
                'The model that was used to generate the response. The AI SDK uses the response model from the provider response when available, and the model from the function call otherwise.',
            },
            {
              name: 'timestamp',
              type: 'Date',
              description:
                'The timestamp of the response. The AI SDK uses the response timestamp from the provider response when available, and creates a timestamp otherwise.',
            },
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Optional response headers.',
            },
          ],
        },
      ],
    },
    {
      name: 'warnings',
      type: 'Warning[] | undefined',
      description:
        'Warnings from the model provider (e.g. unsupported settings).',
    },
    {
      name: 'pipeTextStreamToResponse',
      type: '(response: ServerResponse, init?: ResponseInit => void',
      description:
        'Writes text delta output to a Node.js response-like object. It sets a `Content-Type` header to `text/plain; charset=utf-8` and writes each text delta as a separate chunk.',
      properties: [
        {
          type: 'ResponseInit',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              isOptional: true,
              description: 'The response headers.',
            },
          ],
        },
      ],
    },
    {
      name: 'toTextStreamResponse',
      type: '(init?: ResponseInit) => Response',
      description:
        'Creates a simple text stream response. Each text delta is encoded as UTF-8 and sent as a separate chunk. Non-text-delta events are ignored.',
      properties: [
        {
          type: 'ResponseInit',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description: 'The response status code.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description: 'The response status text.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              isOptional: true,
              description: 'The response headers.',
            },
          ],
        },
      ],
    },
  ]}
/>

## More Examples

<ExampleLinks
  examples={[
    {
      title: 'Streaming Object Generation with RSC',
      link: '/examples/next-app/basics/streaming-object-generation',
    },
    {
      title: 'Streaming Object Generation with useObject',
      link: '/examples/next-pages/basics/streaming-object-generation',
    },
    {
      title: 'Streaming Partial Objects',
      link: '/examples/node/streaming-structured-data/stream-object',
    },
    {
      title: 'Recording Token Usage',
      link: '/examples/node/streaming-structured-data/token-usage',
    },
    {
      title: 'Recording Final Object',
      link: '/examples/node/streaming-structured-data/object',
    },
  ]}
/>

---
title: embed
description: API Reference for embed.
---

# `embed()`

Generate an embedding for a single value using an embedding model.

This is ideal for use cases where you need to embed a single value to e.g. retrieve similar items or to use the embedding in a downstream task.

```ts
import { openai } from '@ai-sdk/openai';
import { embed } from 'ai';

const { embedding } = await embed({
  model: openai.embedding('text-embedding-3-small'),
  value: 'sunny day at the beach',
});
```

## Import

<Snippet text={`import { embed } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'EmbeddingModel',
      description:
        "The embedding model to use. Example: openai.embedding('text-embedding-3-small')",
    },
    {
      name: 'value',
      type: 'VALUE',
      description: 'The value to embed. The type depends on the model.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'value',
      type: 'VALUE',
      description: 'The value that was embedded.',
    },
    {
      name: 'embedding',
      type: 'number[]',
      description: 'The embedding of the value.',
    },
    {
      name: 'usage',
      type: 'EmbeddingTokenUsage',
      description: 'The token usage for generating the embeddings.',
      properties: [
        {
          type: 'EmbeddingTokenUsage',
          parameters: [
            {
              name: 'tokens',
              type: 'number',
              description: 'The total number of input tokens.',
            },
          ],
        },
      ],
    },
    {
      name: 'rawResponse',
      type: 'RawResponse',
      isOptional: true,
      description: 'Optional raw response data.',
      properties: [
        {
          type: 'RawResponse',
          parameters: [
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Response headers.',
            },
          ],
        },
      ],
    },
  ]}
/>

---
title: embedMany
description: API Reference for embedMany.
---

# `embedMany()`

Embed several values using an embedding model. The type of the value is defined
by the embedding model.

`embedMany` automatically splits large requests into smaller chunks if the model
has a limit on how many embeddings can be generated in a single call.

```ts
import { openai } from '@ai-sdk/openai';
import { embedMany } from 'ai';

const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: [
    'sunny day at the beach',
    'rainy afternoon in the city',
    'snowy night in the mountains',
  ],
});
```

## Import

<Snippet text={`import { embedMany } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'EmbeddingModel',
      description:
        "The embedding model to use. Example: openai.embedding('text-embedding-3-small')",
    },
    {
      name: 'values',
      type: 'Array<VALUE>',
      description: 'The values to embed. The type depends on the model.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'experimental_telemetry',
      type: 'TelemetrySettings',
      isOptional: true,
      description: 'Telemetry configuration. Experimental feature.',
      properties: [
        {
          type: 'TelemetrySettings',
          parameters: [
            {
              name: 'isEnabled',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable telemetry. Disabled by default while experimental.',
            },
            {
              name: 'recordInputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable input recording. Enabled by default.',
            },
            {
              name: 'recordOutputs',
              type: 'boolean',
              isOptional: true,
              description:
                'Enable or disable output recording. Enabled by default.',
            },
            {
              name: 'functionId',
              type: 'string',
              isOptional: true,
              description:
                'Identifier for this function. Used to group telemetry data by function.',
            },
            {
              name: 'metadata',
              isOptional: true,
              type: 'Record<string, string | number | boolean | Array<null | undefined | string> | Array<null | undefined | number> | Array<null | undefined | boolean>>',
              description:
                'Additional information to include in the telemetry data.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'values',
      type: 'Array<VALUE>',
      description: 'The values that were embedded.',
    },
    {
      name: 'embeddings',
      type: 'number[][]',
      description: 'The embeddings. They are in the same order as the values.',
    },
    {
      name: 'usage',
      type: 'EmbeddingTokenUsage',
      description: 'The token usage for generating the embeddings.',
      properties: [
        {
          type: 'EmbeddingTokenUsage',
          parameters: [
            {
              name: 'tokens',
              type: 'number',
              description: 'The total number of input tokens.',
            },
          ],
        },
      ],
    },
  ]}
/>

---
title: tool
description: Helper function for tool type inference
---

# `tool()`

Tool is a helper function that infers the tool parameters for its `execute` method.

It does not have any runtime behavior, but it helps TypeScript infer the types of the parameters for the `execute` method.

Without this helper function, TypeScript is unable to connect the `parameters` property to the `execute` method,
and the argument types of `execute` cannot be inferred.

```ts highlight={"1,4,9,10"}
import { tool } from 'ai';
import { z } from 'zod';

export const weatherTool = tool({
  description: 'Get the weather in a location',
  parameters: z.object({
    location: z.string().describe('The location to get the weather for'),
  }),
  // location below is inferred to be a string:
  execute: async ({ location }) => ({
    location,
    temperature: 72 + Math.floor(Math.random() * 21) - 10,
  }),
});
```

## Import

<Snippet text={`import { tool } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'tool',
      type: 'CoreTool',
      description: 'The tool definition.',
      properties: [
        {
          type: 'CoreTool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.'
            },
            {
              name: 'parameters',
              type: 'Zod Schema | JSON Schema',
              description:
                'The schema of the input that the tool expects. The language model will use this to generate the input. It is also used to validate the output of the language model. Use descriptions to make the input understandable for the language model. You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function).'
            },
            {
              name: 'execute',
              isOptional: true,
              type: 'async (parameters: T, options: { abortSignal: AbortSignal }) => RESULT',
              description:
                'An async function that is called with the arguments from the tool call and produces a result. If not provided, the tool will not be executed automatically.'
            },
            {
              name: 'experimental_toToolResultContent',
              isOptional: true,
              type: '(result: RESULT) => TextToolResultContent | ImageToolResultContent',
              description: 'An optional function that converts the result of the tool call to a content object that can be used in LLM messages.',
              properties: [
                {
                  type: 'TextToolResultContent',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the tool result content.'
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The content of the message.'
                    }
                  ]
                },
                {
                  type: 'ImageToolResultContent',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the tool result content.'
                    },
                    {
                      name: 'data',
                      type: 'string',
                      description: 'The base64 encoded png image.'
                    },
                    {
                      name: 'mimeType',
                      isOptional: true,
                      type: 'string',
                      description: 'The mime type of the image.'
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }

]}
/>

### Returns

The tool that was passed in.

---
title: jsonSchema
description: Helper function for creating JSON schemas
---

# `jsonSchema()`

`jsonSchema` is a helper function that creates a JSON schema object that is compatible with the AI SDK.
It takes the JSON schema and an optional validation function as inputs, and can be typed.

You can use it to [generate structured data](/docs/ai-sdk-core/generating-structured-data) and in [tools](/docs/ai-sdk-core/tools-and-tool-calling).

`jsonSchema` is an alternative to using Zod schemas that provides you with flexibility in dynamic situations
(e.g. when using OpenAPI definitions) or for using other validation libraries.

```ts
import { jsonSchema } from 'ai';

const mySchema = jsonSchema<{
  recipe: {
    name: string;
    ingredients: { name: string; amount: string }[];
    steps: string[];
  };
}>({
  type: 'object',
  properties: {
    recipe: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        ingredients: {
          type: 'array',
          items: {
            type: 'object',
            properties: {
              name: { type: 'string' },
              amount: { type: 'string' },
            },
            required: ['name', 'amount'],
          },
        },
        steps: {
          type: 'array',
          items: { type: 'string' },
        },
      },
      required: ['name', 'ingredients', 'steps'],
    },
  },
  required: ['recipe'],
});
```

## Import

<Snippet text={`import { jsonSchema } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'schema',
      type: 'JSONSchema7',
      description: 'The JSON schema definition.',
    },
    {
      name: 'options',
      type: 'SchemaOptions',
      description: 'Additional options for the JSON schema.',
      properties: [
        {
          type: 'SchemaOptions',
          parameters: [
            {
              name: 'validate',
              isOptional: true,
              type: '(value: unknown) => { success: true; value: OBJECT } | { success: false; error: Error };',
              description:
                'A function that validates the value against the JSON schema. If the value is valid, the function should return an object with a `success` property set to `true` and a `value` property set to the validated value. If the value is invalid, the function should return an object with a `success` property set to `false` and an `error` property set to the error.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A JSON schema object that is compatible with the AI SDK.

---
title: CoreMessage
description: Message types for AI SDK Core (API Reference)
---

# `CoreMessage`

`CoreMessage` represents the fundamental message structure used with AI SDK Core functions.
It encompasses various message types that can be used in the `messages` field of any AI SDK Core functions.

## `CoreMessage` Types

### `CoreSystemMessage`

A system message that can contain system information.

```typescript
type CoreSystemMessage = {
  role: 'system';
  content: string;
};
```

<Note>
  Using the "system" property instead of a system message is recommended to
  enhance resilience against prompt injection attacks.
</Note>

### `CoreUserMessage`

A user message that can contain text or a combination of text, images, and files.

```typescript
type CoreUserMessage = {
  role: 'user';
  content: UserContent;
};

type UserContent = string | Array<TextPart | ImagePart | FilePart>;
```

### `CoreAssistantMessage`

An assistant message that can contain text, tool calls, or a combination of both.

```typescript
type CoreAssistantMessage = {
  role: 'assistant';
  content: AssistantContent;
};

type AssistantContent = string | Array<TextPart | ToolCallPart>;
```

### `CoreToolMessage`

A tool message that contains the result of one or more tool calls.

```typescript
type CoreToolMessage = {
  role: 'tool';
  content: ToolContent;
};

type ToolContent = Array<ToolResultPart>;
```

## `CoreMessage` Parts

### `TextPart`

Represents a text content part of a prompt. It contains a string of text.

```typescript
export interface TextPart {
  type: 'text';
  /**
   * The text content.
   */
  text: string;
}
```

### `ImagePart`

Represents an image part in a user message.

```typescript
export interface ImagePart {
  type: 'image';

  /**
   * Image data. Can either be:
   * - data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
   * - URL: a URL that points to the image
   */
  image: DataContent | URL;

  /**
   * Optional mime type of the image.
   */
  mimeType?: string;
}
```

### `FilePart`

Represents an file part in a user message.

```typescript
export interface FilePart {
  type: 'file';

  /**
   * File data. Can either be:
   * - data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
   * - URL: a URL that points to the file
   */
  data: DataContent | URL;

  /**
   * Mime type of the file.
   */
  mimeType: string;
}
```

### `ToolCallPart`

Represents a tool call content part of a prompt, typically generated by the AI model.

```typescript
export interface ToolCallPart {
  type: 'tool-call';

  /**
   * ID of the tool call. This ID is used to match the tool call with the tool result.
   */
  toolCallId: string;

  /**
   * Name of the tool that is being called.
   */
  toolName: string;

  /**
   * Arguments of the tool call. This is a JSON-serializable object that matches the tool's input schema.
   */
  args: unknown;
}
```

### `ToolResultPart`

Represents the result of a tool call in a tool message.

```typescript
export interface ToolResultPart {
  type: 'tool-result';

  /**
   * ID of the tool call that this result is associated with.
   */
  toolCallId: string;

  /**
   * Name of the tool that generated this result.
   */
  toolName: string;

  /**
   * Result of the tool call. This is a JSON-serializable object.
   */
  result: unknown;

  /**
   * Multi-part content of the tool result. Only for tools that support multipart results.
   */
  experimental_content?: ToolResultContent;

  /**
   * Optional flag if the result is an error or an error message.
   */
  isError?: boolean;
}
```

### `ToolResultContent`

```ts
export type ToolResultContent = Array<
  | {
      type: 'text';
      text: string;
    }
  | {
      type: 'image';
      data: string; // base64 encoded png image, e.g. screenshot
      mimeType?: string; // e.g. 'image/png';
    }
>;
```

---
title: createProviderRegistry
description: Registry for managing multiple providers and models (API Reference)
---

# `createProviderRegistry()`

<Note type="warning">Provider management is an experimental feature.</Note>

When you work with multiple providers and models, it is often desirable to manage them
in a central place and access the models through simple string ids.

`createProviderRegistry` lets you create a registry with multiple providers that you
can access by their ids in the format `providerId:modelId`.

### Setup

You can create a registry with multiple providers and models using `experimental_createProviderRegistry`.

```ts
import { anthropic } from '@ai-sdk/anthropic';
import { createOpenAI } from '@ai-sdk/openai';
import { experimental_createProviderRegistry as createProviderRegistry } from 'ai';

export const registry = createProviderRegistry({
  // register provider with prefix and default setup:
  anthropic,

  // register provider with prefix and custom setup:
  openai: createOpenAI({
    apiKey: process.env.OPENAI_API_KEY,
  }),
});
```

### Language models

You can access language models by using the `languageModel` method on the registry.
The provider id will become the prefix of the model id: `providerId:modelId`.

```ts highlight={"5"}
import { generateText } from 'ai';
import { registry } from './registry';

const { text } = await generateText({
  model: registry.languageModel('openai:gpt-4-turbo'),
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

### Text embedding models

You can access text embedding models by using the `textEmbeddingModel` method on the registry.
The provider id will become the prefix of the model id: `providerId:modelId`.

```ts highlight={"5"}
import { embed } from 'ai';
import { registry } from './registry';

const { embedding } = await embed({
  model: registry.textEmbeddingModel('openai:text-embedding-3-small'),
  value: 'sunny day at the beach',
});
```

## Import

<Snippet
  text={`import { experimental_createProviderRegistry as createProviderRegistry } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'providers',
      type: 'Record<string, Provider>',
      description:
        'The unique identifier for the provider. It should be unique within the registry.',
      properties: [
        {
          type: 'Provider',
          parameters: [
            {
              name: 'languageModel',
              type: '(id: string) => LanguageModel',
              description:
                'A function that returns a language model by its id.',
            },
            {
              name: 'textEmbeddingModel',
              type: '(id: string) => EmbeddingModel<string>',
              description:
                'A function that returns a text embedding model by its id.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

The `createProviderRegistry` function returns a `Provider` instance. It has the following methods:

<PropertiesTable
  content={[
    {
      name: 'languageModel',
      type: '(id: string) => LanguageModel',
      description:
        'A function that returns a language model by its id (format: providerId:modelId)',
    },
    {
      name: 'textEmbeddingModel',
      type: '(id: string) => EmbeddingModel<string>',
      description:
        'A function that returns a text embedding model by its id (format: providerId:modelId)',
    },
  ]}
/>

---
title: customProvider
description: Custom provider that uses models from a different provider (API Reference)
---

# `customProvider()`

<Note type="warning">Provider management is an experimental feature.</Note>

With a custom provider, you can map ids to any model.
This allows you to set up custom model configurations, alias names, and more.
The custom provider also supports a fallback provider, which is useful for
wrapping existing providers and adding additional functionality.

### Example: custom model settings

You can create a custom provider using `experiental_createCustomProvider`.

```ts
import { openai } from '@ai-sdk/openai';
import { experimental_customProvider as customProvider } from 'ai';

// custom provider with different model settings:
export const myOpenAI = customProvider({
  languageModels: {
    // replacement model with custom settings:
    'gpt-4': openai('gpt-4', { structuredOutputs: true }),
    // alias model with custom settings:
    'gpt-4o-structured': openai('gpt-4o', { structuredOutputs: true }),
  },
  fallbackProvider: openai,
});
```

## Import

<Snippet
  text={`import { experimental_customProvider as customProvider } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'languageModels',
      type: 'Record<string, LanguageModel>',
      isOptional: true,
      description:
        'A record of language models, where keys are model IDs and values are LanguageModel instances.',
    },
    {
      name: 'textEmbeddingModels',
      type: 'Record<string, EmbeddingModelV1<string>>',
      isOptional: true,
      description:
        'A record of text embedding models, where keys are model IDs and values are EmbeddingModel<string> instances.',
    },
    {
      name: 'fallbackProvider',
      type: 'Provider',
      isOptional: true,
      description:
        'An optional fallback provider to use when a requested model is not found in the custom provider.',
    },
  ]}
/>

### Returns

The `createCustomProvider` function returns a `Provider` instance. It has the following methods:

<PropertiesTable
  content={[
    {
      name: 'languageModel',
      type: '(id: string) => LanguageModel',
      description:
        'A function that returns a language model by its id (format: providerId:modelId)',
    },
    {
      name: 'textEmbeddingModel',
      type: '(id: string) => EmbeddingModel<string>',
      description:
        'A function that returns a text embedding model by its id (format: providerId:modelId)',
    },
  ]}
/>

---
title: cosineSimilarity
description: Calculate the cosine similarity between two vectors (API Reference)
---

# `cosineSimilarity()`

When you want to compare the similarity of embeddings, standard vector similarity metrics
like cosine similarity are often used.

`cosineSimilarity` calculates the cosine similarity between two vectors.
A high value (close to 1) indicates that the vectors are very similar, while a low value (close to -1) indicates that they are different.

```ts
import { openai } from '@ai-sdk/openai';
import { cosineSimilarity, embedMany } from 'ai';

const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: ['sunny day at the beach', 'rainy afternoon in the city'],
});

console.log(
  `cosine similarity: ${cosineSimilarity(embeddings[0], embeddings[1])}`,
);
```

## Import

<Snippet text={`import { cosineSimilarity } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'vector1',
      type: 'number[]',
      description: 'The first vector to compare',
    },
    {
      name: 'vector2',
      type: 'number[]',
      description: 'The second vector to compare',
    },
  ]}
/>

### Returns

A number between -1 and 1 representing the cosine similarity between the two vectors.

---
title: wrapLanguageModel
description: Function for wrapping a language model with middleware (API Reference)
---

# `wrapLanguageModel()`

<Note type="warning">
  Language model middleware is an experimental feature.
</Note>

The `experimental_wrapLanguageModel` function provides a way to enhance the behavior of language models
by wrapping them with middleware.
See [Language Model Middleware](/docs/ai-sdk-core/middleware) for more information on middleware.

```ts
import { experimental_wrapLanguageModel as wrapLanguageModel } from 'ai';

const wrappedLanguageModel = wrapLanguageModel({
  model: yourModel,
  middleware: yourLanguageModelMiddleware,
});
```

## Import

<Snippet
  text={`import { experimental_wrapLanguageModel as wrapLanguageModel } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModelV1',
      description: 'The original LanguageModelV1 instance to be wrapped.',
    },
    {
      name: 'middleware',
      type: 'Experimental_LanguageModelV1Middleware',
      description: 'The middleware to be applied to the language model.',
    },
    {
      name: 'modelId',
      type: 'string',
      description:
        "Optional custom model ID to override the original model's ID.",
    },
    {
      name: 'providerId',
      type: 'string',
      description:
        "Optional custom provider ID to override the original model's provider.",
    },
  ]}
/>

### Returns

A new `LanguageModelV1` instance with middleware applied.

---
title: LanguageModelV1Middleware
description: Middleware for enhancing language model behavior (API Reference)
---

# `LanguageModelV1Middleware`

<Note type="warning">
  Language model middleware is an experimental feature.
</Note>

Language model middleware provides a way to enhance the behavior of language models
by intercepting and modifying the calls to the language model. It can be used to add
features like guardrails, RAG, caching, and logging in a language model agnostic way.

See [Language Model Middleware](/docs/ai-sdk-core/middleware) for more information.

## Import

<Snippet
  text={`import { Experimental_LanguageModelV1Middleware as LanguageModelV1Middleware } from "ai"`}
  prompt={false}
/>

## API Signature

<PropertiesTable
  content={[
    {
      name: 'transformParams',
      type: '({ type: "generate" | "stream", params: LanguageModelV1CallOptions }) => Promise<LanguageModelV1CallOptions>',
      description:
        'Transforms the parameters before they are passed to the language model.',
    },
    {
      name: 'wrapGenerate',
      type: '({ doGenerate: DoGenerateFunction, params: LanguageModelV1CallOptions, model: LanguageModelV1 }) => Promise<DoGenerateResult>',
      description: 'Wraps the generate operation of the language model.',
    },
    {
      name: 'wrapStream',
      type: '({ doStream: DoStreamFunction, params: LanguageModelV1CallOptions, model: LanguageModelV1 }) => Promise<DoStreamResult>',
      description: 'Wraps the stream operation of the language model.',
    },
  ]}
/>

---
title: AI SDK Core
description: Reference documentation for the AI SDK Core
collapsed: true
---

# AI SDK Core

[AI SDK Core](/docs/ai-sdk-core) is a set of functions that allow you to interact with language models and other AI models.
These functions are designed to be easy-to-use and flexible, allowing you to generate text, structured data,
and embeddings from language models and other AI models.

AI SDK Core contains the following main functions:

<IndexCards
  cards={[
    {
      title: 'generateText()',
      description: 'Generate text and call tools from a language model.',
      href: '/docs/reference/ai-sdk-core/generate-text',
    },
    {
      title: 'streamText()',
      description: 'Stream text and call tools from a language model.',
      href: '/docs/reference/ai-sdk-core/stream-text',
    },
    {
      title: 'generateObject()',
      description: 'Generate structured data from a language model.',
      href: '/docs/reference/ai-sdk-core/generate-object',
    },
    {
      title: 'streamObject()',
      description: 'Stream structured data from a language model.',
      href: '/docs/reference/ai-sdk-core/stream-object',
    },
    {
      title: 'embed()',
      description:
        'Generate an embedding for a single value using an embedding model.',
      href: '/docs/reference/ai-sdk-core/embed',
    },
    {
      title: 'embedMany()',
      description:
        'Generate embeddings for several values using an embedding model (batch embedding).',
      href: '/docs/reference/ai-sdk-core/embed-many',
    },
  ]}
/>

It also contains the following helper functions:

<IndexCards
  cards={[
    {
      title: 'tool()',
      description: 'Type inference helper function for tools.',
      href: '/docs/reference/ai-sdk-core/tool',
    },
    {
      title: 'jsonSchema()',
      description: 'Creates AI SDK compatible JSON schema objects.',
      href: '/docs/reference/ai-sdk-core/json-schema',
    },
    {
      title: 'createProviderRegistry()',
      description:
        'Creates a registry for using models from multiple providers.',
      href: '/docs/reference/ai-sdk-core/provider-registry',
    },
    {
      title: 'cosineSimilarity()',
      description:
        'Calculates the cosine similarity between two vectors, e.g. embeddings.',
      href: '/docs/reference/ai-sdk-core/cosine-similarity',
    },
  ]}
/>

---
title: useChat
description: API reference for the useChat hook.
---

# `useChat()`

Allows you to easily create a conversational user interface for your chatbot application. It enables the streaming of chat messages from your AI provider, manages the state for chat input, and updates the UI automatically as new messages are received.

## Import

<Tabs items={['React', 'Svelte', 'Vue', 'Solid']}>
  <Tab>
    <Snippet text="import { useChat } from 'ai/react'" dark prompt={false} />
  </Tab>
  <Tab>
    <Snippet
      text="import { useChat } from '@ai-sdk/svelte'"
      dark
      prompt={false}
    />
  </Tab>
  <Tab>
    <Snippet text="import { useChat } from '@ai-sdk/vue'" dark prompt={false} />
  </Tab>
  <Tab>
    <Snippet
      text="import { useChat } from '@ai-sdk/solid'"
      dark
      prompt={false}
    />
  </Tab>
</Tabs>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'api',
      type: "string = '/api/chat'",
      isOptional: true,
      description:
        'The API endpoint that is called to generate chat responses. It can be a relative path (starting with `/`) or an absolute URL.',
    },
    {
      name: 'id',
      type: 'string',
      isOptional: true,
      description:
        'An unique identifier for the chat. If not provided, a random one will be generated. When provided, the `useChat` hook with the same `id` will have shared states across components. This is useful when you have multiple components showing the same chat stream.',
    },
    {
      name: 'initialInput',
      type: "string = ''",
      isOptional: true,
      description: 'An optional string for the initial prompt input.',
    },
    {
      name: 'initialMessages',
      type: 'Messages[] = []',
      isOptional: true,
      description: 'An optional array of initial chat messages',
    },
    {
      name: 'onToolCall',
      type: '({toolCall: ToolCall}) => void | unknown| Promise<unknown>',
      isOptional: true,
      description:
        'Optional callback function that is invoked when a tool call is received. Intended for automatic client-side tool execution. You can optionally return a result for the tool call, either synchronously or asynchronously.',
    },
    {
      name: 'onResponse',
      type: '(response: Response) => void',
      isOptional: true,
      description:
        'An optional callback that will be called with the response from the API endpoint. Useful for throwing customized errors or logging',
    },
    {
      name: 'onFinish',
      type: '(message: Message, options: OnFinishOptions) => void',
      isOptional: true,
      description:
        'An optional callback function that is called when the completion stream ends.',
      properties: [
        {
          type: 'OnFinishOptions',
          parameters: [
            {
              name: 'usage',
              type: 'CompletionTokenUsage',
              description: 'The token usage for the completion.',
              properties: [
                {
                  type: 'CompletionTokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason why the generation ended.',
            },
          ],
        },
      ],
    },
    {
      name: 'onError',
      type: '(error: Error) => void',
      isOptional: true,
      description:
        'A callback that will be called when the chat stream encounters an error. Optional.',
    },
    {
      name: 'generateId',
      type: '() => string',
      isOptional: true,
      description: 'A custom id generator for messages. Optional.',
    },
    {
      name: 'headers',
      type: 'Record<string, string> | Headers',
      isOptional: true,
      description:
        'Additional headers to be passed to the API endpoint. Optional.',
    },
    {
      name: 'body',
      type: 'any',
      isOptional: true,
      description:
        'Additional body object to be passed to the API endpoint. Optional.',
    },
    {
      name: 'credentials',
      type: "'omit' | 'same-origin' | 'include'",
      isOptional: true,
      description:
        'An optional literal that sets the mode of credentials to be used on the request. Defaults to same-origin.',
    },
    {
      name: 'sendExtraMessageFields',
      type: 'boolean',
      isOptional: true,
      description:
        "An optional boolean that determines whether to send extra fields you've added to `messages`. Defaults to `false` and only the `content` and `role` fields will be sent to the API endpoint. If set to `true`, the `name`, `data`, and `annotations` fields will also be sent.",
    },
    {
      name: 'maxSteps',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of backend calls to generate a response. A maximum number is required to prevent infinite loops in the case of misconfigured tools. By default, it is set to 1.',
    },
    {
      name: 'streamProtocol',
      type: "'text' | 'data'",
      isOptional: true,
      description:
        'An optional literal that sets the type of stream to be used. Defaults to `data`. If set to `text`, the stream will be treated as a text stream.',
    },
    {
      name: 'fetch',
      type: 'FetchFunction',
      isOptional: true,
      description:
        'Optional. A custom fetch function to be used for the API call. Defaults to the global fetch function.',
    },
    {
      name: 'experimental_prepareRequestBody',
      type: '(options: { messages: Message[]; requestData?: JSONValue; requestBody?: object }) => JSONValue',
      isOptional: true,
      description:
        'Experimental (React only). When a function is provided, it will be used to prepare the request body for the chat API. This can be useful for customizing the request body based on the messages and data in the chat.',
    },
    {
      name: 'experimental_throttle',
      type: 'number',
      isOptional: true,
      description:
        'React only. Custom throttle wait time in milliseconds for the message and data updates. When specified, updates will be throttled using this interval. Defaults to undefined (no throttling).',
    },

]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'messages',
      type: 'Message[]',
      description: 'The current array of chat messages.',
      properties: [
        {
          type: 'Message',
          parameters: [
            {
              name: 'id',
              type: 'string',
              description: 'The unique identifier of the message.',
            },
            {
              name: 'role',
              type: "'system' | 'user' | 'assistant' | 'data'",
              description: 'The role of the message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
            {
              name: 'createdAt',
              type: 'Date',
              isOptional: true,
              description: 'The creation date of the message.',
            },
            {
              name: 'name',
              type: 'string',
              isOptional: true,
              description: 'The name of the message.',
            },
            {
              name: 'data',
              type: 'JSONValue',
              isOptional: true,
              description: 'Additional data sent along with the message.',
            },
            {
              name: 'annotations',
              type: 'Array<JSONValue>',
              isOptional: true,
              description:
                'Additional annotations sent along with the message.',
            },
            {
              name: 'toolInvocations',
              type: 'Array<ToolInvocation>',
              isOptional: true,
              description:
                'An array of tool invocations that are associated with the (assistant) message.',
              properties: [
                {
                  type: 'ToolInvocation',
                  parameters: [
                    {
                      name: 'state',
                      type: "'partial-call'",
                      description:
                        'The state of the tool call when it was partially created.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'ID of the tool call. This ID is used to match the tool call with the tool result.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description: 'Name of the tool that is being called.',
                    },
                    {
                      name: 'args',
                      type: 'any',
                      description:
                        'Partial arguments of the tool call. This is a JSON-serializable object.',
                    },
                  ],
                },
                {
                  type: 'ToolInvocation',
                  parameters: [
                    {
                      name: 'state',
                      type: "'call'",
                      description:
                        'The state of the tool call when it was fully created.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'ID of the tool call. This ID is used to match the tool call with the tool result.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description: 'Name of the tool that is being called.',
                    },
                    {
                      name: 'args',
                      type: 'any',
                      description:
                        'Arguments of the tool call. This is a JSON-serializable object that matches the tools input schema.',
                    },
                  ],
                },
                {
                  type: 'ToolInvocation',
                  parameters: [
                    {
                      name: 'state',
                      type: "'result'",
                      description:
                        'The state of the tool call when the result is available.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'ID of the tool call. This ID is used to match the tool call with the tool result.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description: 'Name of the tool that is being called.',
                    },
                    {
                      name: 'args',
                      type: 'any',
                      description:
                        'Arguments of the tool call. This is a JSON-serializable object that matches the tools input schema.',
                    },
                    {
                      name: 'result',
                      type: 'any',
                      description: 'The result of the tool call.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'experimental_attachments',
              type: 'Array<Attachment>',
              isOptional: true,
              description:
                'Additional attachments sent along with the message.',
              properties: [
                {
                  type: 'Attachment',
                  description:
                    'An attachment object that can be used to describe the metadata of the file.',
                  parameters: [
                    {
                      name: 'name',
                      type: 'string',
                      isOptional: true,
                      description:
                        'The name of the attachment, usually the file name.',
                    },
                    {
                      name: 'contentType',
                      type: 'string',
                      isOptional: true,
                      description:
                        'A string indicating the media type of the file.',
                    },
                    {
                      name: 'url',
                      type: 'string',
                      description:
                        'The URL of the attachment. It can either be a URL to a hosted file or a Data URL.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'error',
      type: 'Error | undefined',
      description: 'An error object returned by SWR, if any.',
    },
    {
      name: 'append',
      type: '(message: Message | CreateMessage, options?: ChatRequestOptions) => Promise<string | undefined>',
      description:
        'Function to append a message to the chat, triggering an API call for the AI response. It returns a promise that resolves to full response message content when the API call is successfully finished, or throws an error when the API call fails.',
      properties: [
        {
          type: 'ChatRequestOptions',
          parameters: [
            {
              name: 'headers',
              type: 'Record<string, string> | Headers',
              description:
                'Additional headers that should be to be passed to the API endpoint.',
            },
            {
              name: 'body',
              type: 'object',
              description:
                'Additional body JSON properties that should be sent to the API endpoint.',
            },
            {
              name: 'data',
              type: 'JSONValue',
              description: 'Additional data to be sent to the API endpoint.',
            },
          ],
        },
      ],
    },
    {
      name: 'reload',
      type: '() => Promise<string | undefined>',
      description:
        "Function to reload the last AI chat response for the given chat history. If the last message isn't from the assistant, it will request the API to generate a new response.",
    },
    {
      name: 'stop',
      type: '() => void',
      description: 'Function to abort the current API request.',
    },
    {
      name: 'setMessages',
      type: '(messages: Message[] | ((messages: Message[]) => Message[]) => void',
      description:
        'Function to update the `messages` state locally without triggering an API call.',
    },
    {
      name: 'input',
      type: 'string',
      description: 'The current value of the input field.',
    },
    {
      name: 'setInput',
      type: 'React.Dispatch<React.SetStateAction<string>>',
      description: 'Function to update the `input` value.',
    },
    {
      name: 'handleInputChange',
      type: '(event: any) => void',
      description:
        "Handler for the `onChange` event of the input field to control the input's value.",
    },
    {
      name: 'handleSubmit',
      type: '(event?: { preventDefault?: () => void }, options?: ChatRequestOptions) => void',
      description:
        'Form submission handler that automatically resets the input field and appends a user message. You can use the `options` parameter to send additional data, headers and more to the server.',
      properties: [
        {
          type: 'ChatRequestOptions',
          parameters: [
            {
              name: 'headers',
              type: 'Record<string, string> | Headers',
              description:
                'Additional headers that should be to be passed to the API endpoint.',
            },
            {
              name: 'body',
              type: 'object',
              description:
                'Additional body JSON properties that should be sent to the API endpoint.',
            },
            {
              name: 'data',
              type: 'JSONValue',
              description: 'Additional data to be sent to the API endpoint.',
            },
            {
              name: 'allowEmptySubmit',
              type: 'boolean',
              isOptional: true,
              description:
                'A boolean that determines whether to allow submitting an empty input that triggers a generation. Defaults to `false`.',
            },
            {
              name: 'experimental_attachments',
              type: 'FileList | Array<Attachment>',
              isOptional: true,
              description:
                'An array of attachments to be sent to the API endpoint.',
              properties: [
                {
                  type: 'FileList',
                  parameters: [
                    {
                      name: '',
                      type: '',
                      description:
                        "A list of files that have been selected by the user using an <input type='file'> element. It's also used for a list of files dropped into web content when using the drag and drop API.",
                    },
                  ],
                },
                {
                  type: 'Attachment',
                  description:
                    'An attachment object that can be used to describe the metadata of the file.',
                  parameters: [
                    {
                      name: 'name',
                      type: 'string',
                      isOptional: true,
                      description:
                        'The name of the attachment, usually the file name.',
                    },
                    {
                      name: 'contentType',
                      type: 'string',
                      isOptional: true,
                      description:
                        'A string indicating the media type of the file.',
                    },
                    {
                      name: 'url',
                      type: 'string',
                      description:
                        'The URL of the attachment. It can either be a URL to a hosted file or a Data URL.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'isLoading',
      type: 'boolean',
      description:
        'Boolean flag indicating whether a request is currently in progress.',
    },
    {
      name: 'data',
      type: 'JSONValue[]',
      description: 'Data returned from StreamData.',
    },
    {
      name: 'setData',
      type: '(data: JSONValue[] | undefined | ((data: JSONValue[] | undefined) => JSONValue[] | undefined)) => void',
      description:
        'Function to update the `data` state which contains data from StreamData.',
    },
    {
      name: 'addToolResult',
      type: '({toolCallId: string; result: any;}) => void',
      description:
        'React and SolidJS only. Function to add a tool result to the chat. This will update the chat messages with the tool result and call the API route if all tool results for the last message are available.',
    },
  ]}
/>

## Learn more

- [Chatbot](/docs/ai-sdk-ui/chatbot)
- [Chatbot with Tools](/docs/ai-sdk-ui/chatbot-with-tool-calling)

---
title: useCompletion
description: API reference for the useCompletion hook.
---

# `useCompletion()`

Allows you to create text completion based capabilities for your application. It enables the streaming of text completions from your AI provider, manages the state for chat input, and updates the UI automatically as new messages are received.

## Import

<Tabs items={['React', 'Svelte', 'Vue', 'Solid']}>
  <Tab>
    <Snippet
      text="import { useCompletion } from 'ai/react'"
      dark
      prompt={false}
    />
  </Tab>
  <Tab>
    <Snippet
      text="import { useCompletion } from '@ai-sdk/svelte'"
      dark
      prompt={false}
    />
  </Tab>
  <Tab>
    <Snippet
      text="import { useCompletion } from '@ai-sdk/vue'"
      dark
      prompt={false}
    />
  </Tab>
  <Tab>
    <Snippet
      text="import { useCompletion } from '@ai-sdk/solid'"
      dark
      prompt={false}
    />
  </Tab>
</Tabs>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'api',
      type: "string = '/api/completion'",
      description:
        'The API endpoint that is called to generate text. It can be a relative path (starting with `/`) or an absolute URL.',
    },
    {
      name: 'id',
      type: 'string',
      description:
        'An unique identifier for the completion. If not provided, a random one will be generated. When provided, the `useCompletion` hook with the same `id` will have shared states across components. This is useful when you have multiple components showing the same chat stream',
    },
    {
      name: 'initialInput',
      type: 'string',
      description: 'An optional string for the initial prompt input.',
    },
    {
      name: 'initialCompletion',
      type: 'string',
      description: 'An optional string for the initial completion result.',
    },
    {
      name: 'onResponse',
      type: '(response: Response) => void',
      description:
        'An optional callback function that is called with the response from the API endpoint. Useful for throwing customized errors or logging.',
    },
    {
      name: 'onFinish',
      type: '(prompt: string, completion: string) => void',
      description:
        'An optional callback function that is called when the completion stream ends.',
    },
    {
      name: 'onError',
      type: '(error: Error) => void',
      description:
        'An optional callback that will be called when the chat stream encounters an error.',
    },
    {
      name: 'headers',
      type: 'Record<string, string> | Headers',
      description:
        'An optional object of headers to be passed to the API endpoint.',
    },
    {
      name: 'body',
      type: 'any',
      description:
        'An optional, additional body object to be passed to the API endpoint.',
    },
    {
      name: 'credentials',
      type: "'omit' | 'same-origin' | 'include'",
      description:
        'An optional literal that sets the mode of credentials to be used on the request. Defaults to same-origin.',
    },
    {
      name: 'sendExtraMessageFields',
      type: 'boolean',
      description:
        "An optional boolean that determines whether to send extra fields you've added to `messages`. Defaults to `false` and only the `content` and `role` fields will be sent to the API endpoint.",
    },
    {
      name: 'streamProtocol',
      type: "'text' | 'data'",
      isOptional: true,
      description:
        'An optional literal that sets the type of stream to be used. Defaults to `data`. If set to `text`, the stream will be treated as a text stream.',
    },
    {
      name: 'fetch',
      type: 'FetchFunction',
      isOptional: true,
      description:
        'Optional. A custom fetch function to be used for the API call. Defaults to the global fetch function.',
    },
    {
      name: 'experimental_throttle',
      type: 'number',
      isOptional: true,
      description:
        'React only. Custom throttle wait time in milliseconds for the completion and data updates. When specified, throttles how often the UI updates during streaming. Default is undefined, which disables throttling.',
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'completion',
      type: 'string',
      description: 'The current text completion.',
    },
    {
      name: 'complete',
      type: '(prompt: string, options: { headers, body }) => void',
      description:
        'Function to execute text completion based on the provided prompt.',
    },
    {
      name: 'error',
      type: 'undefined | Error',
      description: 'The error thrown during the completion process, if any.',
    },
    {
      name: 'setCompletion',
      type: '(completion: string) => void',
      description: 'Function to update the `completion` state.',
    },
    {
      name: 'stop',
      type: '() => void',
      description: 'Function to abort the current API request.',
    },
    {
      name: 'input',
      type: 'string',
      description: 'The current value of the input field.',
    },
    {
      name: 'setInput',
      type: 'React.Dispatch<React.SetStateAction<string>>',
      description: 'The current value of the input field.',
    },
    {
      name: 'handleInputChange',
      type: '(event: any) => void',
      description:
        "Handler for the `onChange` event of the input field to control the input's value.",
    },
    {
      name: 'handleSubmit',
      type: '(event?: { preventDefault?: () => void }) => void',
      description:
        'Form submission handler that automatically resets the input field and appends a user message.',
    },
    {
      name: 'isLoading',
      type: 'boolean',
      description:
        'Boolean flag indicating whether a fetch operation is currently in progress.',
    },
  ]}
/>

---
title: useObject
description: API reference for the useObject hook.
---

# `experimental_useObject()`

<Note>`useObject` is an experimental feature and only available in React.</Note>

Allows you to consume text streams that represent a JSON object and parse them into a complete object based on a schema.
You can use it together with [`streamObject`](/docs/reference/ai-sdk-core/stream-object) in the backend.

```tsx
'use client';

import { experimental_useObject as useObject } from 'ai/react';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/use-object',
    schema: z.object({ content: z.string() }),
  });

  return (
    <div>
      <button onClick={() => submit('example input')}>Generate</button>
      {object?.content && <p>{object.content}</p>}
    </div>
  );
}
```

## Import

<Snippet
  text="import { experimental_useObject as useObject } from 'ai/react'"
  dark
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'api',
      type: 'string',
      description:
        'The API endpoint that is called to generate objects. It should stream JSON that matches the schema as chunked text. It can be a relative path (starting with `/`) or an absolute URL.',
    },
    {
      name: 'schema',
      type: 'Zod Schema | JSON Schema',
      description:
        'A schema that defines the shape of the complete object. You can either pass in a Zod schema or a JSON schema (using the `jsonSchema` function).',
    },
    {
      name: 'id?',
      type: 'string',
      description:
        'A unique identifier. If not provided, a random one will be generated. When provided, the `useObject` hook with the same `id` will have shared states across components.',
    },
    {
      name: 'initialValue',
      type: 'DeepPartial<RESULT> | undefined',
      isOptional: true,
      description: 'An value for the initial object. Optional.',
    },
    {
      name: 'fetch',
      type: 'FetchFunction',
      isOptional: true,
      description:
        'A custom fetch function to be used for the API call. Defaults to the global fetch function. Optional.',
    },
    {
      name: 'headers',
      type: 'Record<string, string> | Headers',
      isOptional: true,
      description:
        'A headers object to be passed to the API endpoint. Optional.',
    },
    {
      name: 'onError',
      type: '(error: Error) => void',
      isOptional: true,
      description:
        'Callback function to be called when an error is encountered. Optional.',
    },
    {
      name: 'onFinish',
      type: '(result: OnFinishResult) => void',
      isOptional: true,
      description: 'Called when the streaming response has finished.',
      properties: [
        {
          type: 'OnFinishResult',
          parameters: [
            {
              name: 'object',
              type: 'T | undefined',
              description:
                'The generated object (typed according to the schema). Can be undefined if the final object does not match the schema.',
            },
            {
              name: 'error',
              type: 'unknown | undefined',
              description:
                'Optional error object. This is e.g. a TypeValidationError when the final object does not match the schema.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'submit',
      type: '(input: INPUT) => void',
      description: 'Calls the API with the provided input as JSON body.',
    },
    {
      name: 'object',
      type: 'DeepPartial<RESULT> | undefined',
      description:
        'The current value for the generated object. Updated as the API streams JSON chunks.',
    },
    {
      name: 'error',
      type: 'Error | unknown',
      description: 'The error object if the API call fails.',
    },
    {
      name: 'isLoading',
      type: 'boolean',
      description:
        'Boolean flag indicating whether a request is currently in progress.',
    },
    {
      name: 'stop',
      type: '() => void',
      description: 'Function to abort the current API request.',
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Streaming Object Generation with useObject',
      link: '/examples/next-pages/basics/streaming-object-generation',
    },
  ]}
/>

---
title: useAssistant
description: API reference for the useAssistant hook.
---

# `useAssistant()`

Allows you to handle the client state when interacting with an OpenAI compatible assistant API.
This hook is useful when you want to integrate assistant capibilities into your application,
with the UI updated automatically as the assistant is streaming its execution.

This works in conjunction with [`AssistantResponse`](./assistant-response) in the backend.

<Note>
  `useAssistant` is supported in `ai/react`, `ai/svelte`, and `ai/vue`.
</Note>

## Import

<Tabs items={['React', 'Svelte']}>
  <Tab>
    <Snippet
      text="import { useAssistant } from 'ai/react'"
      dark
      prompt={false}
    />
  </Tab>
  <Tab>
    <Snippet
      text="import { useAssistant } from '@ai-sdk/svelte'"
      dark
      prompt={false}
    />
  </Tab>
</Tabs>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'api',
      type: 'string',
      description:
        'The API endpoint that accepts a threadId and message object and returns an AssistantResponse stream. It can be a relative path (starting with `/`) or an absolute URL.',
    },
    {
      name: 'threadId',
      type: 'string | undefined',
      isOptional: true,
      description:
        'Represents the ID of an existing thread. If not provided, a new thread will be created.',
    },
    {
      name: 'credentials',
      type: "'omit' | 'same-origin' | 'include' = 'same-origin'",
      isOptional: true,
      description: 'Sets the mode of credentials to be used on the request.',
    },
    {
      name: 'headers',
      type: 'Record<string, string> | Headers',
      isOptional: true,
      description: 'Headers to be passed to the API endpoint.',
    },
    {
      name: 'body',
      type: 'any',
      isOptional: true,
      description: 'Additional body to be passed to the API endpoint.',
    },
    {
      name: 'onError',
      type: '(error: Error) => void',
      isOptional: true,
      description:
        'Callback that will be called when the assistant encounters an error',
    },
    {
      name: 'fetch',
      type: 'FetchFunction',
      isOptional: true,
      description:
        'Optional. A custom fetch function to be used for the API call. Defaults to the global fetch function.',
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'messages',
      type: 'Message[]',
      description: 'The current array of chat messages.',
    },
    {
      name: 'setMessages',
      type: 'React.Dispatch<React.SetStateAction<Message>>',
      description: 'Function to update the `messages` array.',
    },
    {
      name: 'threadId',
      type: 'string | undefined',
      description: 'The current thread ID.',
    },
    {
      name: 'setThreadId',
      type: '(threadId: string | undefined) => void',
      description:
        "Set the current thread ID. Specifying a thread ID will switch to that thread, if it exists. If set to 'undefined', a new thread will be created. For both cases, `threadId` will be updated with the new value and `messages` will be cleared.",
    },
    {
      name: 'input',
      type: 'string',
      description: 'The current value of the input field.',
    },
    {
      name: 'setInput',
      type: 'React.Dispatch<React.SetStateAction<string>>',
      description: 'Function to update the `input` value.',
    },
    {
      name: 'handleInputChange',
      type: '(event: any) => void',
      description:
        "Handler for the `onChange` event of the input field to control the input's value.",
    },
    {
      name: 'submitMessage',
      type: '(event?: { preventDefault?: () => void }) => void',
      description:
        'Form submission handler that automatically resets the input field and appends a user message.',
    },
    {
      name: 'status',
      type: "'awaiting_message' | 'in_progress'",
      description:
        'The current status of the assistant. This can be used to show a loading indicator.',
    },
    {
      name: 'append',
      type: '(message: Message | CreateMessage, chatRequestOptions: { options: { headers, body } }) => Promise<string | undefined>',
      description:
        "Function to append a user message to the current thread. This triggers the API call to fetch the assistant's response.",
    },
    {
      name: 'stop',
      type: '() => void',
      description:
        'Function to abort the current request from streaming the assistant response. Note that the run will still be in progress.',
    },
    {
      name: 'error',
      type: 'undefined | Error',
      description:
        'The error thrown during the assistant message processing, if any.',
    },
  ]}
/>

---
title: AssistantResponse
description: API reference for the AssistantResponse streaming helper.
---

# `AssistantResponse`

The AssistantResponse class is designed to facilitate streaming assistant responses to the [`useAssistant`](/docs/reference/ai-sdk-ui/use-assistant) hook.
It receives an assistant thread and a current message, and can send messages and data messages to the client.

## Import

<Snippet text={`import { AssistantResponse } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'settings',
      type: 'Settings',
      description:
        'You can pass the id of thread and the latest message which helps establish the context for the response.',
      properties: [
        {
          type: 'Settings',
          parameters: [
            {
              name: 'threadId',
              type: 'string',
              description:
                'The thread ID that the response is associated with.',
            },
            {
              name: 'messageId',
              type: 'string',
              description:
                'The id of the latest message the response is associated with.',
            },
          ],
        },
      ],
    },
    {
      name: 'process',
      type: 'AssistantResponseCallback',
      description:
        'A callback in which you can run the assistant on threads, and send messages and data messages to the client.',
      properties: [
        {
          type: 'AssistantResponseCallback',
          parameters: [
            {
              name: 'forwardStream',
              type: '(stream: AssistantStream) => Run | undefined',
              description:
                'Forwards the assistant response stream to the client. Returns the Run object after it completes, or when it requires an action.',
            },
            {
              name: 'sendDataMessage',
              type: '(message: DataMessage) => void',
              description:
                'Send a data message to the client. You can use this to provide information for rendering custom UIs while the assistant is processing the thread.',
            },
          ],
        },
      ],
    },
  ]}
/>

---
title: convertToCoreMessages
description: Convert useChat messages to CoreMessages for AI core functions (API Reference)
---

# `convertToCoreMessages()`

<Note title="warning">
  The `convertToCoreMessages` function is no longer required. The AI SDK now
  automatically converts the incoming messages to the `CoreMessage` format.
</Note>

The `convertToCoreMessages` function is used to transform an array of UI messages from the `useChat` hook into an array of `CoreMessage` objects. These `CoreMessage` objects are compatible with AI core functions like `streamText`.

```ts filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { convertToCoreMessages, streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages: convertToCoreMessages(messages),
  });

  return result.toDataStreamResponse();
}
```

## Import

<Snippet text={`import { convertToCoreMessages } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'messages',
      type: 'Message[]',
      description:
        'An array of UI messages from the useChat hook to be converted',
    },
    {
      name: 'options',
      type: '{ tools?: Record<string, Tool> }',
      description:
        'Optional configuration object. Provide tools to enable multi-modal tool responses.',
    },
  ]}
/>

### Returns

An array of [`CoreMessage`](/docs/reference/ai-sdk-core/core-message) objects.

<PropertiesTable
  content={[
    {
      name: 'CoreMessage[]',
      type: 'Array',
      description: 'An array of CoreMessage objects',
    },
  ]}
/>

## Multi-modal Tool Responses

The `convertToCoreMessages` function supports tools that can return multi-modal content. This is useful when tools need to return non-text content like images.

```ts
import { tool } from 'ai';
import { z } from 'zod';

const screenshotTool = tool({
  parameters: z.object({}),
  execute: async () => 'imgbase64',
  experimental_toToolResultContent: result => [{ type: 'image', data: result }],
});

const result = streamText({
  model: openai('gpt-4'),
  messages: convertToCoreMessages(messages, {
    tools: {
      screenshot: screenshotTool,
    },
  }),
});
```

Tools can implement the optional `experimental_toToolResultContent` method to transform their results into multi-modal content. The content is an array of content parts, where each part has a `type` (e.g., 'text', 'image') and corresponding data.

---
title: StreamData
description: Learn to use streamData helper function in your application.
---

# `StreamData`

The `StreamData` class allows you to stream additional data to the client (see [Streaming Data](/docs/ai-sdk-ui/streaming-data)).

## Import

### React

<Snippet text={`import { StreamData } from "ai"`} prompt={false} />

## API Signature

### Constructor

```ts
const data = new StreamData();
```

### Methods

#### `append`

Appends a value to the stream data.

```ts
data.append(value: JSONValue)
```

#### `appendMessageAnnotation`

Appends a message annotation to the stream data.

```ts
data.appendMessageAnnotation(annotation: JSONValue)
```

#### `close`

Closes the stream data.

```ts
data.close();
```

---
title: AI SDK UI
description: Reference documentation for the AI SDK UI
collapsed: true
---

# AI SDK UI

[AI SDK UI](/docs/ai-sdk-ui) is designed to help you build interactive chat, completion, and assistant applications with ease.
It is framework-agnostic toolkit, streamlining the integration of advanced AI functionalities into your applications.

AI SDK UI contains the following hooks:

<IndexCards
  cards={[
    {
      title: 'useChat',
      description:
        'Use a hook to interact with language models in a chat interface.',
      href: '/docs/reference/ai-sdk-ui/use-chat',
    },
    {
      title: 'useCompletion',
      description:
        'Use a hook to interact with language models in a completion interface.',
      href: '/docs/reference/ai-sdk-ui/use-completion',
    },
    {
      title: 'useObject',
      description: 'Use a hook for consuming a streamed JSON objects.',
      href: '/docs/reference/ai-sdk-ui/use-object',
    },
    {
      title: 'useAssistant',
      description: 'Use a hook to interact with OpenAI assistants.',
      href: '/docs/reference/ai-sdk-ui/use-assistant',
    },
    {
      title: 'streamData',
      description:
        'Stream additional data to the client along with generations.',
      href: '/docs/reference/ai-sdk-ui/stream-data',
    },
  ]}
/>

It also contains the following helper functions:

<IndexCards
  cards={[
    {
      title: 'AssistantResponse',
      description: 'Streaming helper for assistant responses.',
      href: '/docs/reference/ai-sdk-ui/assistant-response',
    },
  ]}
/>

## UI Framework Support

AI SDK UI supports the following frameworks: [React](https://react.dev/), [Svelte](https://svelte.dev/), [Vue.js](https://vuejs.org/), and [SolidJS](https://www.solidjs.com/).
Here is a comparison of the supported functions across these frameworks:

| Function                                                  | React               | Svelte              | Vue.js              | SolidJS             |
| --------------------------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| [useChat](/docs/reference/ai-sdk-ui/use-chat)             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useChat](/docs/reference/ai-sdk-ui/use-chat) attachments | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useCompletion](/docs/reference/ai-sdk-ui/use-completion) | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [useObject](/docs/reference/ai-sdk-ui/use-object)         | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [useAssistant](/docs/reference/ai-sdk-ui/use-assistant)   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |

<Note>
  [Contributions](https://github.com/vercel/ai/blob/main/CONTRIBUTING.md) are
  welcome to implement missing features for non-React frameworks.
</Note>

---
title: streamUI
description: Reference for the streamUI function from the AI SDK RSC
---

# `streamUI`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

A helper function to create a streamable UI from LLM providers. This function is similar to AI SDK Core APIs and supports the same model interfaces.

To see `streamUI` in action, check out [these examples](#examples).

## Import

<Snippet text={`import { streamUI } from "ai/rsc"`} prompt={false} />

## Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'LanguageModel',
      description: 'The language model to use. Example: openai("gpt-4-turbo")',
    },
    {
      name: 'initial',
      isOptional: true,
      type: 'ReactNode',
      description: 'The initial UI to render.',
    },
    {
      name: 'system',
      type: 'string',
      description:
        'The system prompt to use that specifies the behavior of the model.',
    },
    {
      name: 'prompt',
      type: 'string',
      description: 'The input prompt to generate the text from.',
    },
    {
      name: 'messages',
      type: 'Array<CoreSystemMessage | CoreUserMessage | CoreAssistantMessage | CoreToolMessage> | Array<UIMessage>',
      description:
        'A list of messages that represent a conversation. Automatically converts UI messages from the useChat hook.',
      properties: [
        {
          type: 'CoreSystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'CoreUserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ImagePart | FilePart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ImagePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'image'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'image',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The image content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      isOptional: true,
                      description: 'The mime type of the image. Optional.',
                    },
                  ],
                },
                {
                  type: 'FilePart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'file'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'data',
                      type: 'string | Uint8Array | Buffer | ArrayBuffer | URL',
                      description:
                        'The file content of the message part. String are either base64 encoded content, base64 data URLs, or http(s) URLs.',
                    },
                    {
                      name: 'mimeType',
                      type: 'string',
                      description: 'The mime type of the file.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreAssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string | Array<TextPart | ToolCallPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'TextPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'text'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'text',
                      type: 'string',
                      description: 'The text content of the message part.',
                    },
                  ],
                },
                {
                  type: 'ToolCallPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-call'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool, which typically would be the name of the function.',
                    },
                    {
                      name: 'args',
                      type: 'object based on zod schema',
                      description:
                        'Parameters generated by the model to be used by the tool.',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'CoreToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'Array<ToolResultPart>',
              description: 'The content of the message.',
              properties: [
                {
                  type: 'ToolResultPart',
                  parameters: [
                    {
                      name: 'type',
                      type: "'tool-result'",
                      description: 'The type of the message part.',
                    },
                    {
                      name: 'toolCallId',
                      type: 'string',
                      description:
                        'The id of the tool call the result corresponds to.',
                    },
                    {
                      name: 'toolName',
                      type: 'string',
                      description:
                        'The name of the tool the result corresponds to.',
                    },
                    {
                      name: 'result',
                      type: 'unknown',
                      description:
                        'The result returned by the tool after execution.',
                    },
                    {
                      name: 'isError',
                      type: 'boolean',
                      isOptional: true,
                      description:
                        'Whether the result is an error or an error message.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
    {
      name: 'maxTokens',
      type: 'number',
      isOptional: true,
      description: 'Maximum number of tokens to generate.',
    },
    {
      name: 'temperature',
      type: 'number',
      isOptional: true,
      description:
        'Temperature setting. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topP',
      type: 'number',
      isOptional: true,
      description:
        'Nucleus sampling. The value is passed through to the provider. The range depends on the provider and model. It is recommended to set either `temperature` or `topP`, but not both.',
    },
    {
      name: 'topK',
      type: 'number',
      isOptional: true,
      description:
        'Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Recommended for advanced use cases only. You usually only need to use temperature.',
    },
    {
      name: 'presencePenalty',
      type: 'number',
      isOptional: true,
      description:
        'Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'frequencyPenalty',
      type: 'number',
      isOptional: true,
      description:
        'Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The value is passed through to the provider. The range depends on the provider and model.',
    },
    {
      name: 'stopSequences',
      type: 'string[]',
      isOptional: true,
      description:
        'Sequences that will stop the generation of the text. If the model generates any of these sequences, it will stop generating further text.',
    },
    {
      name: 'seed',
      type: 'number',
      isOptional: true,
      description:
        'The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.',
    },
    {
      name: 'maxRetries',
      type: 'number',
      isOptional: true,
      description:
        'Maximum number of retries. Set to 0 to disable retries. Default: 2.',
    },
    {
      name: 'abortSignal',
      type: 'AbortSignal',
      isOptional: true,
      description:
        'An optional abort signal that can be used to cancel the call.',
    },
    {
      name: 'headers',
      type: 'Record<string, string>',
      isOptional: true,
      description:
        'Additional HTTP headers to be sent with the request. Only applicable for HTTP-based providers.',
    },
    {
      name: 'tools',
      type: 'Record<string, Tool>',
      description:
        'Tools that are accessible to and can be called by the model.',
      properties: [
        {
          type: 'Tool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.',
            },
            {
              name: 'parameters',
              type: 'zod schema',
              description:
                'The typed schema that describes the parameters of the tool that can also be used to validation and error handling.',
            },
            {
              name: 'generate',
              isOptional: true,
              type: '(async (parameters) => ReactNode) | AsyncGenerator<ReactNode, ReactNode, void>',
              description:
                'A function or a generator function that is called with the arguments from the tool call and yields React nodes as the UI.',
            },
          ],
        },
      ],
    },
    {
      name: 'toolChoice',
      isOptional: true,
      type: '"auto" | "none" | "required" | { "type": "tool", "toolName": string }',
      description:
        'The tool choice setting. It specifies how tools are selected for execution. The default is "auto". "none" disables tool execution. "required" requires tools to be executed. { "type": "tool", "toolName": string } specifies a specific tool to execute.',
    },
    {
      name: 'text',
      isOptional: true,
      type: '(Text) => ReactNode',
      description: 'Callback to handle the generated tokens from the model.',
      properties: [
        {
          type: 'Text',
          parameters: [
            {
              name: 'content',
              type: 'string',
              description: 'The full content of the completion.',
            },
            { name: 'delta', type: 'string', description: 'The delta.' },
            { name: 'done', type: 'boolean', description: 'Is it done?' },
          ],
        },
      ],
    },
    {
      name: 'experimental_providerMetadata',
      type: 'Record<string,Record<string,JSONValue>> | undefined',
      isOptional: true,
      description:
        'Optional metadata from the provider. The outer key is the provider name. The inner values are the metadata. Details depend on the provider.',
    },
    {
      name: 'onFinish',
      type: '(result: OnFinishResult) => void',
      isOptional: true,
      description:
        'Callback that is called when the LLM response and all request tool executions (for tools that have a `generate` function) are finished.',
      properties: [
        {
          type: 'OnFinishResult',
          parameters: [
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
            {
              name: 'value',
              type: 'ReactNode',
              description: 'The final ui node that was generated.',
            },
            {
              name: 'warnings',
              type: 'Warning[] | undefined',
              description:
                'Warnings from the model provider (e.g. unsupported settings).',
            },
            {
              name: 'rawResponse',
              type: 'RawResponse',
              description: 'Optional raw response data.',
              properties: [
                {
                  type: 'RawResponse',
                  parameters: [
                    {
                      name: 'headers',
                      isOptional: true,
                      type: 'Record<string, string>',
                      description: 'Response headers.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
  ]}
/>

## Returns

<PropertiesTable
  content={[
    {
      name: 'value',
      type: 'ReactNode',
      description: 'The user interface based on the stream output.',
    },
    {
      name: 'rawResponse',
      type: 'RawResponse',
      isOptional: true,
      description: 'Optional raw response data.',
      properties: [
        {
          type: 'RawResponse',
          parameters: [
            {
              name: 'headers',
              isOptional: true,
              type: 'Record<string, string>',
              description: 'Response headers.',
            },
          ],
        },
      ],
    },
    {
      name: 'warnings',
      type: 'Warning[] | undefined',
      description:
        'Warnings from the model provider (e.g. unsupported settings).',
    },
    {
      name: 'stream',
      type: 'AsyncIterable<StreamPart> & ReadableStream<StreamPart>',
      description:
        'A stream with all events, including text deltas, tool calls, tool results, and errors. You can use it as either an AsyncIterable or a ReadableStream. When an error occurs, the stream will throw the error.',
      properties: [
        {
          type: 'StreamPart',
          parameters: [
            {
              name: 'type',
              type: "'text-delta'",
              description: 'The type to identify the object as text delta.',
            },
            {
              name: 'textDelta',
              type: 'string',
              description: 'The text delta.',
            },
          ],
        },
        {
          type: 'StreamPart',
          parameters: [
            {
              name: 'type',
              type: "'tool-call'",
              description: 'The type to identify the object as tool call.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
            {
              name: 'toolName',
              type: 'string',
              description:
                'The name of the tool, which typically would be the name of the function.',
            },
            {
              name: 'args',
              type: 'object based on zod schema',
              description:
                'Parameters generated by the model to be used by the tool.',
            },
          ],
        },
        {
          type: 'StreamPart',
          parameters: [
            {
              name: 'type',
              type: "'error'",
              description: 'The type to identify the object as error.',
            },
            {
              name: 'error',
              type: 'Error',
              description:
                'Describes the error that may have occurred during execution.',
            },
          ],
        },
        {
          type: 'StreamPart',
          parameters: [
            {
              name: 'type',
              type: "'finish'",
              description: 'The type to identify the object as finish.',
            },
            {
              name: 'finishReason',
              type: "'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown'",
              description: 'The reason the model finished generating the text.',
            },
            {
              name: 'usage',
              type: 'TokenUsage',
              description: 'The token usage of the generated text.',
              properties: [
                {
                  type: 'TokenUsage',
                  parameters: [
                    {
                      name: 'promptTokens',
                      type: 'number',
                      description: 'The total number of tokens in the prompt.',
                    },
                    {
                      name: 'completionTokens',
                      type: 'number',
                      description:
                        'The total number of tokens in the completion.',
                    },
                    {
                      name: 'totalTokens',
                      type: 'number',
                      description: 'The total number of tokens generated.',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title:
        'Learn to render a React component as a function call using a language model in Next.js',
      link: '/examples/next-app/state-management/ai-ui-states',
    },
    {
      title: 'Learn to persist and restore states UI/AI states in Next.js',
      link: '/examples/next-app/state-management/save-and-restore-states',
    },
    {
      title:
        'Learn to route React components using a language model in Next.js',
      link: '/examples/next-app/interface/route-components',
    },
    {
      title: 'Learn to stream component updates to the client in Next.js',
      link: '/examples/next-app/interface/stream-component-updates',
    },
  ]}
/>

---
title: createAI
description: Reference for the createAI function from the AI SDK RSC
---

# `createAI`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Creates a client-server context provider that can be used to wrap parts of your application tree to easily manage both UI and AI states of your application.

## Import

<Snippet text={`import { createAI } from "ai/rsc"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'actions',
      type: 'Record<string, Action>',
      description: 'Server side actions that can be called from the client.',
    },
    {
      name: 'initialAIState',
      type: 'any',
      description: 'Initial AI state to be used in the client.',
    },
    {
      name: 'initialUIState',
      type: 'any',
      description: 'Initial UI state to be used in the client.',
    },
    {
      name: 'onGetUIState',
      type: '() => UIState',
      description: 'is called during SSR to compare and update UI state.',
    },
    {
      name: 'onSetAIState',
      type: '(Event) => void',
      description:
        'is triggered whenever an update() or done() is called by the mutable AI state in your action, so you can safely store your AI state in the database.',
      properties: [
        {
          type: 'Event',
          parameters: [
            {
              name: 'state',
              type: 'AIState',
              description: 'The resulting AI state after the update.',
            },
            {
              name: 'done',
              type: 'boolean',
              description:
                'Whether the AI state updates have been finalized or not.',
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

It returns an `<AI/>` context provider.

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to manage AI and UI states in Next.js',
      link: '/examples/next-app/state-management/ai-ui-states',
    },
    {
      title: 'Learn to persist and restore states UI/AI states in Next.js',
      link: '/examples/next-app/state-management/save-and-restore-states',
    },
  ]}
/>

---
title: createStreamableUI
description: Reference for the createStreamableUI function from the AI SDK RSC
---

# `createStreamableUI`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Create a stream that sends UI from the server to the client. On the client side, it can be rendered as a normal React node.

## Import

<Snippet text={`import { createStreamableUI } from "ai/rsc"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'initialValue',
      type: 'ReactNode',
      isOptional: true,
      description: 'The initial value of the streamable UI.',
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'value',
      type: 'ReactNode',
      description:
        'The value of the streamable UI. This can be returned from a Server Action and received by the client.',
    },
  ]}
/>

### Methods

<PropertiesTable
  content={[
    {
      name: 'update',
      type: '(ReactNode) => void',
      description:
        'Updates the current UI node. It takes a new UI node and replaces the old one.',
    },
    {
      name: 'append',
      type: '(ReactNode) => void',
      description:
        'Appends a new UI node to the end of the old one. Once appended a new UI node, the previous UI node cannot be updated anymore.',
    },
    {
      name: 'done',
      type: '(ReactNode | null) => void',
      description:
        'Marks the UI node as finalized and closes the stream. Once called, the UI node cannot be updated or appended anymore. This method is always required to be called, otherwise the response will be stuck in a loading state.',
    },
    {
      name: 'error',
      type: '(Error) => void',
      description:
        'Signals that there is an error in the UI stream. It will be thrown on the client side and caught by the nearest error boundary component.',
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Render a React component during a tool call',
      link: '/examples/next-app/tools/render-interface-during-tool-call',
    },
  ]}
/>

---
title: createStreamableValue
description: Reference for the createStreamableValue function from the AI SDK RSC
---

# `createStreamableValue`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Create a stream that sends values from the server to the client. The value can be any serializable data.

## Import

<Snippet
  text={`import { createStreamableValue } from "ai/rsc"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'value',
      type: 'any',
      description: 'Any data that RSC supports. Example, JSON.',
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'value',
      type: 'streamable',
      description:
        'This creates a special value that can be returned from Actions to the client. It holds the data inside and can be updated via the update method.',
    },
  ]}
/>

---
title: readStreamableValue
description: Reference for the readStreamableValue function from the AI SDK RSC
---

# `readStreamableValue`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

It is a function that helps you read the streamable value from the client that was originally created using [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value) on the server.

## Import

<Snippet text={`import { readStreamableValue } from "ai/rsc"`} prompt={false} />

## Example

```ts filename="app/actions.ts"
async function generate() {
  'use server';
  const streamable = createStreamableValue();

  streamable.update(1);
  streamable.update(2);
  streamable.done(3);

  return streamable.value;
}
```

```tsx filename="app/page.tsx" highlight="12"
import { readStreamableValue } from 'ai/rsc';

export default function Page() {
  const [generation, setGeneration] = useState('');

  return (
    <div>
      <button
        onClick={async () => {
          const stream = await generate();

          for await (const delta of readStreamableValue(stream)) {
            setGeneration(generation => generation + delta);
          }
        }}
      >
        Generate
      </button>
    </div>
  );
}
```

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'stream',
      type: 'StreamableValue',
      description: 'The streamable value to read from.',
    },
  ]}
/>

### Returns

It returns an async iterator that contains the values emitted by the streamable value.

---
title: getAIState
description: Reference for the getAIState function from the AI SDK RSC
---

# `getAIState`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Get the current AI state.

## Import

<Snippet text={`import { getAIState } from "ai/rsc"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'key',
      type: 'string',
      isOptional: true,
      description:
        "Returns the value of the specified key in the AI state, if it's an object.",
    },
  ]}
/>

### Returns

The AI state.

## Examples

<ExampleLinks
  examples={[
    {
      title:
        'Learn to render a React component during a tool call made by a language model in Next.js',
      link: '/examples/next-app/tools/render-interface-during-tool-call',
    },
  ]}
/>

---
title: getMutableAIState
description: Reference for the getMutableAIState function from the AI SDK RSC
---

# `getMutableAIState`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

Get a mutable copy of the AI state. You can use this to update the state in the server.

## Import

<Snippet text={`import { getMutableAIState } from "ai/rsc"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'key',
      isOptional: true,
      type: 'string',
      description:
        "Returns the value of the specified key in the AI state, if it's an object.",
    },
  ]}
/>

### Returns

The mutable AI state.

### Methods

<PropertiesTable
  content={[
    {
      name: 'update',
      type: '(newState: any) => void',
      description: 'Updates the AI state with the new state.',
    },
    {
      name: 'done',
      type: '(newState: any) => void',
      description:
        'Updates the AI state with the new state, marks it as finalized and closes the stream.',
    },
  ]}
/>

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to persist and restore states AI and UI states in Next.js',
      link: '/examples/next-app/state-management/save-and-restore-states',
    },
  ]}
/>

---
title: useAIState
description: Reference for the useAIState function from the AI SDK RSC
---

# `useAIState`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

It is a hook that enables you to read and update the AI state. The AI state is shared globally between all `useAIState` hooks under the same `<AI/>` provider.

The AI state is intended to contain context and information shared with the AI model, such as system messages, function responses, and other relevant data.

## Import

<Snippet text={`import { useAIState } from "ai/rsc"`} prompt={false} />

## API Signature

### Returns

A single element array where the first element is the current AI state.

---
title: useActions
description: Reference for the useActions function from the AI SDK RSC
---

# `useActions`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

It is a hook to help you access your Server Actions from the client. This is particularly useful for building interfaces that require user interactions with the server.

It is required to access these server actions via this hook because they are patched when passed through the context. Accessing them directly may result in a [Cannot find Client Component error](/docs/troubleshooting/common-issues/server-actions-in-client-components).

## Import

<Snippet text={`import { useActions } from "ai/rsc"`} prompt={false} />

## API Signature

### Returns

`Record<string, Action>`, a dictionary of server actions.

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to manage AI and UI states in Next.js',
      link: '/examples/next-app/state-management/ai-ui-states',
    },
    {
      title:
        'Learn to route React components using a language model in Next.js',
      link: '/examples/next-app/interface/route-components',
    },
  ]}
/>

---
title: useUIState
description: Reference for the useUIState function from the AI SDK RSC
---

# `useUIState`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

It is a hook that enables you to read and update the UI State. The state is client-side and can contain functions, React nodes, and other data. UIState is the visual representation of the AI state.

## Import

<Snippet text={`import { useUIState } from "ai/rsc"`} prompt={false} />

## API Signature

### Returns

Similar to useState, it is an array, where the first element is the current UI state and the second element is the function that updates the state.

## Examples

<ExampleLinks
  examples={[
    {
      title: 'Learn to manage AI and UI states in Next.js',
      link: '/examples/next-app/state-management/ai-ui-states',
    },
  ]}
/>

---
title: useStreamableValue
description: Reference for the useStreamableValue function from the AI SDK RSC
---

# `useStreamableValue`

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

It is a React hook that takes a streamable value created using [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value) and returns the current value, error, and pending state.

## Import

<Snippet text={`import { useStreamableValue } from "ai/rsc"`} prompt={false} />

## Example

This is useful for consuming streamable values received from a component's props.

```tsx
function MyComponent({ streamableValue }) {
  const [data, error, pending] = useStreamableValue(streamableValue);

  if (pending) return <div>Loading...</div>;
  if (error) return <div>Error: {error.message}</div>;

  return <div>Data: {data}</div>;
}
```

## API Signature

### Parameters

It accepts a streamable value created using `createStreamableValue`.

### Returns

It is an array, where the first element contains the data, the second element contains an error if it is thrown anytime during the stream, and the third is a boolean indicating if the value is pending.

---
title: render (Removed)
description: Reference for the render function from the AI SDK RSC
---

# `render` (Removed)

<Note type="warning">"render" has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

A helper function to create a streamable UI from LLM providers. This function is similar to AI SDK Core APIs and supports the same model interfaces.

> **Note**: `render` has been deprecated in favor of [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui). During migration, please ensure that the `messages` parameter follows the updated [specification](/docs/reference/ai-sdk-rsc/stream-ui#messages).

## Import

<Snippet text={`import { render } from "ai/rsc"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'model',
      type: 'string',
      description: 'Model identifier, must be OpenAI SDK compatible.',
    },
    {
      name: 'provider',
      type: 'provider client',
      description:
        'Currently the only provider available is OpenAI. This needs to match the model name.',
    },
    {
      name: 'initial',
      isOptional: true,
      type: 'ReactNode',
      description: 'The initial UI to render.',
    },
    {
      name: 'messages',
      type: 'Array<SystemMessage | UserMessage | AssistantMessage | ToolMessage>',
      description: 'A list of messages that represent a conversation.',
      properties: [
        {
          type: 'SystemMessage',
          parameters: [
            {
              name: 'role',
              type: "'system'",
              description: 'The role for the system message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'UserMessage',
          parameters: [
            {
              name: 'role',
              type: "'user'",
              description: 'The role for the user message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
          ],
        },
        {
          type: 'AssistantMessage',
          parameters: [
            {
              name: 'role',
              type: "'assistant'",
              description: 'The role for the assistant message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
            {
              name: 'tool_calls',
              type: 'ToolCall[]',
              description: 'A list of tool calls made by the model.',
              properties: [
                {
                  type: 'ToolCall',
                  parameters: [
                    {
                      name: 'id',
                      type: 'string',
                      description: 'The id of the tool call.',
                    },
                    {
                      name: 'type',
                      type: "'function'",
                      description: 'The type of the tool call.',
                    },
                    {
                      name: 'function',
                      type: 'Function',
                      description: 'The function to call.',
                      properties: [
                        {
                          type: 'Function',
                          parameters: [
                            {
                              name: 'name',
                              type: 'string',
                              description: 'The name of the function.',
                            },
                            {
                              name: 'arguments',
                              type: 'string',
                              description: 'The arguments of the function.',
                            },
                          ],
                        },
                      ],
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          type: 'ToolMessage',
          parameters: [
            {
              name: 'role',
              type: "'tool'",
              description: 'The role for the tool message.',
            },
            {
              name: 'content',
              type: 'string',
              description: 'The content of the message.',
            },
            {
              name: 'toolCallId',
              type: 'string',
              description: 'The id of the tool call.',
            },
          ],
        },
      ],
    },
    {
      name: 'functions',
      type: 'Record<string, Tool>',
      isOptional: true,
      description:
        'Tools that are accessible to and can be called by the model.',
      properties: [
        {
          type: 'Tool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.',
            },
            {
              name: 'parameters',
              type: 'zod schema',
              description:
                'The typed schema that describes the parameters of the tool that can also be used to validation and error handling.',
            },
            {
              name: 'render',
              isOptional: true,
              type: 'async (parameters) => any',
              description:
                'An async function that is called with the arguments from the tool call and produces a result.',
            },
          ],
        },
      ],
    },
    {
      name: 'tools',
      type: 'Record<string, Tool>',
      isOptional: true,
      description:
        'Tools that are accessible to and can be called by the model.',
      properties: [
        {
          type: 'Tool',
          parameters: [
            {
              name: 'description',
              isOptional: true,
              type: 'string',
              description:
                'Information about the purpose of the tool including details on how and when it can be used by the model.',
            },
            {
              name: 'parameters',
              type: 'zod schema',
              description:
                'The typed schema that describes the parameters of the tool that can also be used to validation and error handling.',
            },
            {
              name: 'render',
              isOptional: true,
              type: 'async (parameters) => any',
              description:
                'An async function that is called with the arguments from the tool call and produces a result.',
            },
          ],
        },
      ],
    },
    {
      name: 'text',
      isOptional: true,
      type: '(Text) => ReactNode',
      description: 'Callback to handle the generated tokens from the model.',
      properties: [
        {
          type: 'Text',
          parameters: [
            {
              name: 'content',
              type: 'string',
              description: 'The full content of the completion.',
            },
            { name: 'delta', type: 'string', description: 'The delta.' },
            { name: 'done', type: 'boolean', description: 'Is it done?' },
          ],
        },
      ],
    },
    {
      name: 'temperature',
      isOptional: true,
      type: 'number',
      description: 'The temperature to use for the model.',
    },
  ]}
/>

### Returns

It can return any valid ReactNode.

---
title: AI SDK RSC
description: Reference documentation for the AI SDK UI
collapsed: true
---

# AI SDK RSC

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

<IndexCards
  cards={[
    {
      title: 'streamUI',
      description:
        'Use a helper function that streams React Server Components on tool execution.',
      href: '/docs/reference/ai-sdk-rsc/stream-ui',
    },
    {
      title: 'createAI',
      description:
        'Create a context provider that wraps your application and shares state between the client and language model on the server.',
      href: '/docs/reference/ai-sdk-rsc/create-ai',
    },
    {
      title: 'createStreamableUI',
      description:
        'Create a streamable UI component that can be rendered on the server and streamed to the client.',
      href: '/docs/reference/ai-sdk-rsc/create-streamable-ui',
    },
    {
      title: 'createStreamableValue',
      description:
        'Create a streamable value that can be rendered on the server and streamed to the client.',
      href: '/docs/reference/ai-sdk-rsc/create-streamable-value',
    },
    {
      title: 'getAIState',
      description: 'Read the AI state on the server.',
      href: '/docs/reference/ai-sdk-rsc/get-ai-state',
    },
    {
      title: 'getMutableAIState',
      description: 'Read and update the AI state on the server.',
      href: '/docs/reference/ai-sdk-rsc/get-mutable-ai-state',
    },
    {
      title: 'useAIState',
      description: 'Get the AI state on the client from the context provider.',
      href: '/docs/reference/ai-sdk-rsc/use-ai-state',
    },
    {
      title: 'useUIState',
      description: 'Get the UI state on the client from the context provider.',
      href: '/docs/reference/ai-sdk-rsc/use-ui-state',
    },
    {
      title: 'useActions',
      description: 'Call server actions from the client.',
      href: '/docs/reference/ai-sdk-rsc/use-actions',
    },
  ]}
/>

---
title: AIStream
description: Learn to use AIStream helper function in your application.
---

# `AIStream`

<Note type="warning">
  AIStream has been removed in AI SDK 4.0. Use
  `streamText.toDataStreamResponse()` instead.
</Note>

Creates a readable stream for AI responses. This is based on the responses returned
by fetch and serves as the basis for the OpenAIStream and AnthropicStream. It allows
you to handle AI response streams in a controlled and customized manner that will
work with useChat and useCompletion.

AIStream will throw an error if response doesn't have a 2xx status code. This is to ensure that the stream is only created for successful responses.

## Import

### React

<Snippet text={`import { AIStream } from "ai"`} prompt={false} />

## API Signature

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        "This is the response object returned by fetch. It's used as the source of the readable stream.",
    },
    {
      name: 'customParser',
      type: '(AIStreamParser) => void',
      description:
        'This is a function that is used to parse the events in the stream. It should return a function that receives a stringified chunk from the LLM and extracts the message content. The function is expected to return nothing (void) or a string.',
      properties: [
        {
          type: 'AIStreamParser',
          parameters: [
            {
              name: '',
              type: '(data: string) => string | void',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

---
title: StreamingTextResponse
description: Learn to use StreamingTextResponse helper function in your application.
---

# `StreamingTextResponse`

<Note type="warning">
  `StreamingTextResponse` has been removed in AI SDK 4.0. Use
  [`streamText.toDataStreamResponse()`](/docs/reference/ai-sdk-core/stream-text)
  instead.
</Note>

It is a utility class that simplifies the process of returning a ReadableStream of text in HTTP responses.
It is a lightweight wrapper around the native Response class, automatically setting the status code to 200 and the Content-Type header to 'text/plain; charset=utf-8'.

## Import

<Snippet text={`import { StreamingTextResponse } from "ai"`} prompt={false} />

## API Signature

## Parameters

<PropertiesTable
  content={[
    {
      name: 'stream',
      type: 'ReadableStream',
      description: 'The stream of content which represents the HTTP response.',
    },
    {
      name: 'init',
      isOptional: true,
      type: 'ResponseInit',
      description:
        'It can be used to customize the properties of the HTTP response. It is an object that corresponds to the ResponseInit object used in the Response constructor.',
      properties: [
        {
          type: 'ResponseInit',
          parameters: [
            {
              name: 'status',
              type: 'number',
              isOptional: true,
              description:
                'The status code for the response. StreamingTextResponse will overwrite this value with 200.',
            },
            {
              name: 'statusText',
              type: 'string',
              isOptional: true,
              description:
                'The status message associated with the status code.',
            },
            {
              name: 'headers',
              type: 'HeadersInit',
              isOptional: true,
              description:
                "Any headers you want to add to your response. StreamingTextResponse will add 'Content-Type': 'text/plain; charset=utf-8' to these headers.",
            },
          ],
        },
      ],
    },
    {
      name: 'data',
      isOptional: true,
      type: 'StreamData',
      description:
        'StreamData object that you are using to generate additional data for the response.',
    },
  ]}
/>

### Returns

An instance of Response with the provided ReadableStream as the body, the status set to 200, and the Content-Type header set to 'text/plain; charset=utf-8'. Additional headers and properties can be added using the init parameter

---
title: streamToResponse
description: Learn to use streamToResponse helper function in your application.
---

# `streamToResponse`

<Note type="warning">
  `streamToResponse` has been removed in AI SDK 4.0. Use
  `pipeDataStreamToResponse` from
  [streamText](/docs/reference/ai-sdk-core/stream-text) instead.
</Note>

`streamToResponse` pipes a data stream to a Node.js `ServerResponse` object and sets the status code and headers.

This is useful to create data stream responses in environments that use `ServerResponse` objects, such as Node.js HTTP servers.

The status code and headers can be configured using the `options` parameter.
By default, the status code is set to 200 and the Content-Type header is set to `text/plain; charset=utf-8`.

## Import

<Snippet text={`import { streamToResponse } from "ai"`} prompt={false} />

## Example

You can e.g. use `streamToResponse` to pipe a data stream to a Node.js HTTP server response:

```ts
import { openai } from '@ai-sdk/openai';
import { StreamData, streamText, streamToResponse } from 'ai';
import { createServer } from 'http';

createServer(async (req, res) => {
  const result = streamText({
    model: openai('gpt-4-turbo'),
    prompt: 'What is the weather in San Francisco?',
  });

  // use stream data
  const data = new StreamData();

  data.append('initialized call');

  streamToResponse(
    result.toAIStream({
      onFinal() {
        data.append('call completed');
        data.close();
      },
    }),
    res,
    {},
    data,
  );
}).listen(8080);
```

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'stream',
      type: 'ReadableStream',
      description:
        'The Web Stream to pipe to the response. It can be the return value of OpenAIStream, HuggingFaceStream, AnthropicStream, or an AIStream instance.',
    },
    {
      name: 'response',
      type: 'ServerResponse',
      description:
        'The Node.js ServerResponse object to pipe the stream to. This is usually the second argument of a Node.js HTTP request handler.',
    },
    {
      name: 'options',
      type: 'Options',
      description: 'Configure the response',
      properties: [
        {
          type: 'Options',
          parameters: [
            {
              name: 'status',
              type: 'number',
              description:
                'The status code to set on the response. Defaults to `200`.',
            },
            {
              name: 'headers',
              type: 'Record<string, string>',
              description:
                "Additional headers to set on the response. Defaults to `{ 'Content-Type': 'text/plain; charset=utf-8' }`.",
            },
          ],
        },
      ],
    },
    {
      name: 'data',
      type: 'StreamData',
      description:
        'StreamData object for forwarding additional data to the client.',
    },
  ]}
/>

---
title: OpenAIStream
description: Learn to use OpenAIStream helper function in your application.
---

# `OpenAIStream`

<Note type="warning">OpenAIStream has been removed in AI SDK 4.0</Note>

<Note type="warning">
  OpenAIStream is part of the legacy OpenAI integration. It is not compatible
  with the AI SDK 3.1 functions. It is recommended to use the [AI SDK OpenAI
  Provider](/providers/ai-sdk-providers/openai) instead.
</Note>

Transforms the response from OpenAI's language models into a ReadableStream.

Note: Prior to v4, the official OpenAI API SDK does not support the Edge Runtime and only works in serverless environments. The openai-edge package is based on fetch instead of axios (and thus works in the Edge Runtime) so we recommend using openai v4+ or openai-edge.

## Import

### React

<Snippet text={`import { OpenAIStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        'The response object returned by a call made by the Provider SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

---
title: AnthropicStream
description: Learn to use AnthropicStream helper function in your application.
---

# `AnthropicStream`

<Note type="warning">AnthropicStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  AnthropicStream is part of the legacy Anthropic integration. It is not
  compatible with the AI SDK 3.1 functions. It is recommended to use the [AI SDK
  Anthropic Provider](/providers/ai-sdk-providers/anthropic) instead.
</Note>

It is a utility function that transforms the output from Anthropic's SDK into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Anthropic's response data structure.

## Import

### React

<Snippet text={`import { AnthropicStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        'The response object returned by a call made by the Provider SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: AWSBedrockStream
description: Learn to use AWSBedrockStream helper function in your application.
---

# `AWSBedrockStream`

<Note type="warning">AWSBedrockStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  AWSBedrockStream is part of the legacy AWS Bedrock integration. It is not
  compatible with the AI SDK 3.1 functions.
</Note>

The AWS Bedrock stream functions are utilties that transform the outputs from the AWS Bedrock API into a ReadableStream. It uses AIStream under the hood and handle parsing Bedrock's response.

## Import

### React

<Snippet text={`import { AWSBedrockStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'AWSBedrockResponse',
      description: 'The response object returned from AWS Bedrock.',
      properties: [
        {
          type: 'AWSBedrockResponse',
          parameters: [
            {
              name: 'body',
              isOptional: true,
              type: 'AsyncIterable<{ chunk?: { bytes?: Uint8Array } }>',
              description:
                'An optional async iterable of objects containing optional binary data chunks.',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: AWSBedrockAnthropicStream
description: Learn to use AWSBedrockAnthropicStream helper function in your application.
---

# `AWSBedrockAnthropicStream`

<Note type="warning">
  AWSBedrockAnthropicStream has been removed in AI SDK 4.0.
</Note>

<Note type="warning">
  AWSBedrockAnthropicStream is part of the legacy AWS Bedrock integration. It is
  not compatible with the AI SDK 3.1 functions.
</Note>

The AWS Bedrock stream functions are utilties that transform the outputs from the AWS Bedrock API into a ReadableStream. It uses AIStream under the hood and handle parsing Bedrock's response.

## Import

### React

<Snippet
  text={`import { AWSBedrockAnthropicStream } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'AWSBedrockResponse',
      description: 'The response object returned from AWS Bedrock.',
      properties: [
        {
          type: 'AWSBedrockResponse',
          parameters: [
            {
              name: 'body',
              isOptional: true,
              type: 'AsyncIterable<{ chunk?: { bytes?: Uint8Array } }>',
              description:
                'An optional async iterable of objects containing optional binary data chunks.',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: AWSBedrockAnthropicMessagesStream
description: Learn to use AWSBedrockAnthropicMessagesStream helper function in your application.
---

# `AWSBedrockAnthropicMessagesStream`

<Note type="warning">
  AWSBedrockAnthropicMessagesStream has been removed in AI SDK 4.0.
</Note>

<Note type="warning">
  AWSBedrockAnthropicMessagesStream is part of the legacy AWS Bedrock
  integration. It is not compatible with the AI SDK 3.1 functions.
</Note>

The AWS Bedrock stream functions are utilties that transform the outputs from the AWS Bedrock API into a ReadableStream. It uses AIStream under the hood and handle parsing Bedrock's response.

## Import

### React

<Snippet
  text={`import { AWSBedrockAnthropicMessagesStream } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'AWSBedrockResponse',
      description: 'The response object returned from AWS Bedrock.',
      properties: [
        {
          type: 'AWSBedrockResponse',
          parameters: [
            {
              name: 'body',
              isOptional: true,
              type: 'AsyncIterable<{ chunk?: { bytes?: Uint8Array } }>',
              description:
                'An optional async iterable of objects containing optional binary data chunks.',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: AWSBedrockCohereStream
description: Learn to use AWSBedrockCohereStream helper function in your application.
---

# `AWSBedrockCohereStream`

<Note type="warning">
  AWSBedrockCohereStream has been removed in AI SDK 4.0.
</Note>

<Note type="warning">
  AWSBedrockCohereStream is part of the legacy AWS Bedrock integration. It is
  not compatible with the AI SDK 3.1 functions.
</Note>

## Import

The AWS Bedrock stream functions are utilties that transform the outputs from the AWS Bedrock API into a ReadableStream. It uses AIStream under the hood and handles parsing Bedrock's response.

### React

<Snippet text={`import { AWSBedrockCohereStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'AWSBedrockResponse',
      description: 'The response object returned from AWS Bedrock.',
      properties: [
        {
          type: 'AWSBedrockResponse',
          parameters: [
            {
              name: 'body',
              isOptional: true,
              type: 'AsyncIterable<{ chunk?: { bytes?: Uint8Array } }>',
              description:
                'An optional async iterable of objects containing optional binary data chunks.',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: AWSBedrockLlama2Stream
description: Learn to use AWSBedrockLlama2Stream helper function in your application.
---

# `AWSBedrockLlama2Stream`

<Note type="warning">
  AWSBedrockLlama2Stream has been removed in AI SDK 4.0.
</Note>

<Note type="warning">
  AWSBedrockLlama2Stream is part of the legacy AWS Bedrock integration. It is
  not compatible with the AI SDK 3.1 functions.
</Note>

The AWS Bedrock stream functions are utilties that transform the outputs from the AWS Bedrock API into a ReadableStream. It uses AIStream under the hood and handle parsing Bedrock's response.

## Import

### React

<Snippet text={`import { AWSBedrockLlama2Stream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'AWSBedrockResponse',
      description: 'The response object returned from AWS Bedrock.',
      properties: [
        {
          type: 'AWSBedrockResponse',
          parameters: [
            {
              name: 'body',
              isOptional: true,
              type: 'AsyncIterable<{ chunk?: { bytes?: Uint8Array } }>',
              description:
                'An optional async iterable of objects containing optional binary data chunks.',
            },
          ],
        },
      ],
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: CohereStream
description: Learn to use CohereStream helper function in your application.
---

# `CohereStream`

<Note type="warning">CohereStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  CohereStream is part of the legacy Cohere integration. It is not compatible
  with the AI SDK 3.1 functions.
</Note>

The CohereStream function is a utility that transforms the output from Cohere's API into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Cohere's response data structure. This works with the official Cohere API, and it's supported in both Node.js, the Edge Runtime, and browser environments.

## Import

### React

<Snippet text={`import { CohereStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        'The response object returned by a call made by the Provider SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: GoogleGenerativeAIStream
description: Learn to use GoogleGenerativeAIStream helper function in your application.
---

# `GoogleGenerativeAIStream`

<Note type="warning">
  GoogleGenerativeAIStream has been removed in AI SDK 4.0.
</Note>

<Note type="warning">
  GoogleGenerativeAIStream is part of the legacy Google Generative AI
  integration. It is not compatible with the AI SDK 3.1 functions. It is
  recommended to use the [AI SDK Google Generative AI
  Provider](/providers/ai-sdk-providers/google-generative-ai) instead.
</Note>

The GoogleGenerativeAIStream function is a utility that transforms the output from Google's Generative AI SDK into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Google's response data structure. This works with the official Generative AI SDK, and it's supported in both Node.js, Edge Runtime, and browser environments.

## Import

### React

<Snippet
  text={`import { GoogleGenerativeAIStream } from "ai"`}
  prompt={false}
/>

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: '{ stream: AsyncIterable<GenerateContentResponse> }',
      description:
        'The response object returned by the Google Generative AI API.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: HuggingFaceStream
description: Learn to use HuggingFaceStream helper function in your application.
---

# `HuggingFaceStream`

<Note type="warning">HuggingFaceStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  HuggingFaceStream is part of the legacy Hugging Face integration. It is not
  compatible with the AI SDK 3.1 functions.
</Note>

Converts the output from language models hosted on Hugging Face into a ReadableStream.

While HuggingFaceStream is compatible with most Hugging Face language models, the rapidly evolving landscape of models may result in certain new or niche models not being supported. If you encounter a model that isn't supported, we encourage you to open an issue.

To ensure that AI responses are comprised purely of text without any delimiters that could pose issues when rendering in chat or completion modes, we standardize and remove special end-of-response tokens. If your use case requires a different handling of responses, you can fork and modify this stream to meet your specific needs.

Currently, `
` and `
` are recognized as end-of-stream tokens.

## Import

### React

<Snippet text={`import { HuggingFaceStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'iter',
      type: 'AsyncGenerator<any>',
      description:
        'This parameter should be the generator function returned by the hf.textGenerationStream method in the Hugging Face Inference SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: LangChainAdapter
description: API Reference for LangChainAdapter.
---

# `LangChainAdapter`

The `LangChainAdapter` module provides helper functions to transform LangChain output streams into data streams and data stream responses.
See the [LangChain Adapter documentation](/providers/adapters/langchain) for more information.

It supports:

- LangChain StringOutputParser streams
- LangChain AIMessageChunk streams
- LangChain StreamEvents v2 streams

## Import

<Snippet text={`import { LangChainAdapter } from "ai"`} prompt={false} />

## API Signature

### Methods

<PropertiesTable
  content={[
    {
      name: 'toDataStream',
      type: '(stream: ReadableStream<LangChainAIMessageChunk> | ReadableStream<string>, AIStreamCallbacksAndOptions) => AIStream',
      description: 'Converts LangChain output streams to data stream.',
    },
    {
      name: 'toDataStreamResponse',
      type: '(stream: ReadableStream<LangChainAIMessageChunk> | ReadableStream<string>, options?: {init?: ResponseInit, data?: StreamData, callbacks?: AIStreamCallbacksAndOptions}) => Response',
      description: 'Converts LangChain output streams to data stream response.',
    },
  ]}
/>

## Examples

### Convert LangChain Expression Language Stream

```tsx filename="app/api/completion/route.ts" highlight={"14"}
import { ChatOpenAI } from '@langchain/openai';
import { LangChainAdapter } from 'ai';

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const model = new ChatOpenAI({
    model: 'gpt-3.5-turbo-0125',
    temperature: 0,
  });

  const stream = await model.stream(prompt);

  return LangChainAdapter.toDataStreamResponse(stream);
}
```

### Convert StringOutputParser Stream

```tsx filename="app/api/completion/route.ts" highlight={"16"}
import { ChatOpenAI } from '@langchain/openai';
import { LangChainAdapter } from 'ai';
import { StringOutputParser } from '@langchain/core/output_parsers';

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const model = new ChatOpenAI({
    model: 'gpt-3.5-turbo-0125',
    temperature: 0,
  });

  const parser = new StringOutputParser();
  const stream = await model.pipe(parser).stream(prompt);

  return LangChainAdapter.toDataStreamResponse(stream);
}
```

---
title: LangChainStream
description: API Reference for LangChainStream.
---

# `LangChainStream`

<Note type="warning">LangChainStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  LangChainStream is part of the legacy LangChain integration. It is recommended
  to use the [LangChain Adapter](/providers/adapters/langchain) instead.
</Note>

Helps with the integration of LangChain. It is compatible with useChat and useCompletion.

## Import

<Snippet text={`import { LangChainStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

<PropertiesTable
  content={[
    {
      name: 'stream',
      type: 'ReadableStream',
      description:
        'This is the readable stream that can be piped into another stream. This stream contains the results of the LangChain process.',
    },
    {
      name: 'handlers',
      type: 'LangChainCallbacks',
      description:
        'This object contains handlers that can be used to handle certain callbacks provided by LangChain.',
    },
  ]}
/>

---
title: LlamaIndexAdapter
description: API Reference for LlamaIndexAdapter.
---

# `LlamaIndexAdapter`

The `LlamaIndexAdapter` module provides helper functions to transform LlamaIndex output streams into data streams and data stream responses.
See the [LlamaIndex Adapter documentation](/providers/adapters/llamaindex) for more information.

It supports:

- LlamaIndex ChatEngine streams
- LlamaIndex QueryEngine streams

## Import

<Snippet text={`import { LlamaIndexAdapter } from "ai"`} prompt={false} />

## API Signature

### Methods

<PropertiesTable
  content={[
    {
      name: 'toDataStream',
      type: '(stream: AsyncIterable<EngineResponse>, AIStreamCallbacksAndOptions) => AIStream',
      description: 'Converts LlamaIndex output streams to data stream.',
    },
    {
      name: 'toDataStreamResponse',
      type: '(stream: AsyncIterable<EngineResponse>, options?: {init?: ResponseInit, data?: StreamData, callbacks?: AIStreamCallbacksAndOptions}) => Response',
      description:
        'Converts LlamaIndex output streams to data stream response.',
    },
  ]}
/>

## Examples

### Convert LlamaIndex ChatEngine Stream

```tsx filename="app/api/completion/route.ts" highlight="15"
import { OpenAI, SimpleChatEngine } from 'llamaindex';
import { LlamaIndexAdapter } from 'ai';

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const llm = new OpenAI({ model: 'gpt-4o' });
  const chatEngine = new SimpleChatEngine({ llm });

  const stream = await chatEngine.chat({
    message: prompt,
    stream: true,
  });

  return LlamaIndexAdapter.toDataStreamResponse(stream);
}
```

---
title: MistralStream
description: Learn to use MistralStream helper function in your application.
---

# `MistralStream`

<Note type="warning">MistralStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  MistralStream is part of the legacy Mistral integration. It is not compatible
  with the AI SDK 3.1 functions. It is recommended to use the [AI SDK Mistral
  Provider](/providers/ai-sdk-providers/mistral) instead.
</Note>

Transforms the output from Mistral's language models into a ReadableStream.

This works with the official Mistral API, and it's supported in both Node.js, the Edge Runtime, and browser environments.

## Import

### React

<Snippet text={`import { MistralStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        'The response object returned by a call made by the Provider SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: ReplicateStream
description: Learn to use ReplicateStream helper function in your application.
---

# `ReplicateStream`

<Note type="warning">ReplicateStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  ReplicateStream is part of the legacy Replicate integration. It is not
  compatible with the AI SDK 3.1 functions.
</Note>

The ReplicateStream function is a utility that handles extracting the stream from the output of [Replicate](https://replicate.com)'s API. It expects a Prediction object as returned by the [Replicate JavaScript SDK](https://github.com/replicate/replicate-javascript), and returns a ReadableStream. Unlike other wrappers, ReplicateStream returns a Promise because it makes a fetch call to the [Replicate streaming API](https://github.com/replicate/replicate-javascript#streaming) under the hood.

## Import

### React

<Snippet text={`import { ReplicateStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'pre',
      type: 'Prediction',
      description: 'Object returned by the Replicate JavaScript SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
    {
      name: 'options',
      type: '{ headers?: Record<string, string> }',
      isOptiona: true,
      description: 'An optional parameter for passing additional headers.',
    },
  ]}
/>

### Returns

A `ReadableStream` wrapped in a promise.

---
title: InkeepStream
description: Learn to use InkeepStream helper function in your application.
---

# `InkeepStream`

<Note type="warning">InkeepStream has been removed in AI SDK 4.0.</Note>

<Note type="warning">
  InkeepStream is part of the legacy Inkeep integration. It is not compatible
  with the AI SDK 3.1 functions.
</Note>

The InkeepStream function is a utility that transforms the output from [Inkeep](https://inkeep.com)'s API into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Inkeep's response data structure.

This works with the official Inkeep API, and it's supported in both Node.js, the Edge Runtime, and browser environments.

## Import

### React

<Snippet text={`import { InkeepStream } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'response',
      type: 'Response',
      description:
        'The response object returned by a call made by the Provider SDK.',
    },
    {
      name: 'callbacks',
      type: 'AIStreamCallbacksAndOptions',
      isOptional: true,
      description:
        'An object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.',
      properties: [
        {
          type: 'AIStreamCallbacksAndOptions',
          parameters: [
            {
              name: 'onStart',
              type: '() => Promise<void>',
              description:
                'An optional function that is called at the start of the stream processing.',
            },
            {
              name: 'onCompletion',
              type: '(completion: string) => Promise<void>',
              description:
                "An optional function that is called for every completion. It's passed the completion as a string.",
            },
            {
              name: 'onFinal',
              type: '(completion: string) => Promise<void>',
              description:
                'An optional function that is called once when the stream is closed with the final completion message.',
            },
            {
              name: 'onToken',
              type: '(token: string) => Promise<void>',
              description:
                "An optional function that is called for each token in the stream. It's passed the token as a string.",
            },
          ],
        },
      ],
    },
  ]}
/>

### Returns

A `ReadableStream`.

---
title: generateId
description: Generate a unique identifier (API Reference)
---

# `generateId()`

Generates a unique identifier. You can optionally provide the length of the ID.

This is the same id generator used by the AI SDK.

```ts
import { generateId } from 'ai';

const id = generateId();
```

## Import

<Snippet text={`import { generateId } from "ai"`} prompt={false} />

## API Signature

### Parameters

<PropertiesTable
  content={[
    {
      name: 'size',
      type: 'number',
      description: 'The length of the generated ID. It defaults to 16.',
    },
  ]}
/>

### Returns

A string representing the generated ID.

---
title: Stream Helpers
description: Learn to use help functions that help stream generations from different providers.
collapsed: true
---

<IndexCards
  cards={[
    {
      title: 'AIStream',
      description: 'Create a readable stream for AI responses.',
      href: '/docs/reference/stream-helpers/ai-stream',
    },
    {
      title: 'StreamingTextResponse',
      description: 'Create a streaming response for text generations.',
      href: '/docs/reference/stream-helpers/streaming-text-response',
    },
    {
      title: 'streamtoResponse',
      description: 'Pipe a ReadableStream to a Node.js ServerResponse object.',
      href: '/docs/reference/stream-helpers/stream-to-response',
    },
    {
      title: 'OpenAIStream',
      description:
        "Transforms the response from OpenAI's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/openai-stream',
    },
    {
      title: 'AnthropicStream',
      description:
        "Transforms the response from Anthropic's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/anthropic-stream',
    },
    {
      title: 'AWSBedrockStream',
      description:
        "Transforms the response from AWS Bedrock's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/aws-bedrock-stream',
    },
    {
      title: 'AWSBedrockMessagesStream',
      description:
        "Transforms the response from AWS Bedrock Message's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/aws-bedrock-messages-stream',
    },
    {
      title: 'AWSBedrockCohereStream',
      description:
        "Transforms the response from AWS Bedrock Cohere's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/aws-bedrock-cohere-stream',
    },
    {
      title: 'AWSBedrockLlama-2Stream',
      description:
        "Transforms the response from AWS Bedrock Llama-2's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/aws-bedrock-llama-2-stream',
    },
    {
      title: 'CohereStream',
      description:
        "Transforms the response from Cohere's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/cohere-stream',
    },
    {
      title: 'GoogleGenerativeAIStream',
      description:
        "Transforms the response from Google's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/google-generative-ai-stream',
    },
    {
      title: 'HuggingFaceStream',
      description:
        "Transforms the response from Hugging Face's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/hugging-face-stream',
    },
    {
      title: 'LangChainStream',
      description:
        "Transforms the response from LangChain's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/langchain-stream',
    },
    {
      title: 'LangChainAdapter',
      description:
        "Transforms the response from LangChain's stream into data streams.",
      href: '/docs/reference/stream-helpers/langchain-adapter',
    },
     {
      title: 'LlamaIndexAdapter',
      description:
        "Transforms the response from LlamaIndex's streams into data streams.",
      href: '/docs/reference/stream-helpers/llamaindex-adapter',
    },
    {
      title: 'MistralStream',
      description:
        "Transforms the response from Mistral's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/mistral-stream',
    },
    {
      title: 'ReplicateStream',
      description:
        "Transforms the response from Replicate's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/replicate-stream',
    },
    {
      title: 'InkeepsStream',
      description:
        "Transforms the response from Inkeeps's language models into a readable stream.",
      href: '/docs/reference/stream-helpers/inkeep-stream',
    },
    {
      title: 'generateId()',
      description:
        'Helper function for generating unique IDs',
      href: '/docs/reference/stream-helpers/generate-id',
    }

]}
/>

---
title: AI_APICallError
description: Learn how to fix AI_APICallError
---

# AI_APICallError

This error occurs when an API call fails.

## Properties

- `url`: The URL of the API request that failed
- `requestBodyValues`: The request body values sent to the API
- `statusCode`: The HTTP status code returned by the API
- `responseHeaders`: The response headers returned by the API
- `responseBody`: The response body returned by the API
- `isRetryable`: Whether the request can be retried based on the status code
- `data`: Any additional data associated with the error

## Checking for this Error

You can check if an error is an instance of `AI_APICallError` using:

```typescript
import { APICallError } from 'ai';

if (APICallError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_DownloadError
description: Learn how to fix AI_DownloadError
---

# AI_DownloadError

This error occurs when a download fails.

## Properties

- `url`: The URL that failed to download
- `statusCode`: The HTTP status code returned by the server
- `statusText`: The HTTP status text returned by the server
- `message`: The error message containing details about the download failure

## Checking for this Error

You can check if an error is an instance of `AI_DownloadError` using:

```typescript
import { DownloadError } from 'ai';

if (DownloadError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_EmptyResponseBodyError
description: Learn how to fix AI_EmptyResponseBodyError
---

# AI_EmptyResponseBodyError

This error occurs when the server returns an empty response body.

## Properties

- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_EmptyResponseBodyError` using:

```typescript
import { EmptyResponseBodyError } from 'ai';

if (EmptyResponseBodyError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidArgumentError
description: Learn how to fix AI_InvalidArgumentError
---

# AI_InvalidArgumentError

This error occurs when an invalid argument was provided.

## Properties

- `parameter`: The name of the parameter that is invalid
- `value`: The invalid value
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_InvalidArgumentError` using:

```typescript
import { InvalidArgumentError } from 'ai';

if (InvalidArgumentError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidDataContentError
description: How to fix AI_InvalidDataContentError
---

# AI_InvalidDataContentError

This error occurs when the data content provided in a multi-modal message part is invalid. Check out the [ prompt examples for multi-modal messages ](/docs/foundations/prompts#message-prompts).

## Properties

- `content`: The invalid content value
- `message`: The error message describing the expected and received content types

## Checking for this Error

You can check if an error is an instance of `AI_InvalidDataContentError` using:

```typescript
import { InvalidDataContentError } from 'ai';

if (InvalidDataContentError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidDataContent
description: Learn how to fix AI_InvalidDataContent
---

# AI_InvalidDataContent

This error occurs when invalid data content is provided.

## Properties

- `content`: The invalid content value
- `message`: The error message
- `cause`: The cause of the error

## Checking for this Error

You can check if an error is an instance of `AI_InvalidDataContent` using:

```typescript
import { InvalidDataContent } from 'ai';

if (InvalidDataContent.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidMessageRoleError
description: Learn how to fix AI_InvalidMessageRoleError
---

# AI_InvalidMessageRoleError

This error occurs when an invalid message role is provided.

## Properties

- `role`: The invalid role value
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_InvalidMessageRoleError` using:

```typescript
import { InvalidMessageRoleError } from 'ai';

if (InvalidMessageRoleError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidPromptError
description: Learn how to fix AI_InvalidPromptError
---

# AI_InvalidPromptError

This error occurs when the prompt provided is invalid.

## Properties

- `prompt`: The invalid prompt value
- `message`: The error message
- `cause`: The cause of the error

## Checking for this Error

You can check if an error is an instance of `AI_InvalidPromptError` using:

```typescript
import { InvalidPromptError } from 'ai';

if (InvalidPromptError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidResponseDataError
description: Learn how to fix AI_InvalidResponseDataError
---

# AI_InvalidResponseDataError

This error occurs when the server returns a response with invalid data content.

## Properties

- `data`: The invalid response data value
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_InvalidResponseDataError` using:

```typescript
import { InvalidResponseDataError } from 'ai';

if (InvalidResponseDataError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_InvalidToolArgumentsError
description: Learn how to fix AI_InvalidToolArgumentsError
---

# AI_InvalidToolArgumentsError

This error occurs when invalid tool argument was provided.

## Properties

- `toolName`: The name of the tool with invalid arguments
- `toolArgs`: The invalid tool arguments
- `message`: The error message
- `cause`: The cause of the error

## Checking for this Error

You can check if an error is an instance of `AI_InvalidToolArgumentsError` using:

```typescript
import { InvalidToolArgumentsError } from 'ai';

if (InvalidToolArgumentsError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_JSONParseError
description: Learn how to fix AI_JSONParseError
---

# AI_JSONParseError

This error occurs when JSON fails to parse.

## Properties

- `text`: The text value that could not be parsed
- `message`: The error message including parse error details

## Checking for this Error

You can check if an error is an instance of `AI_JSONParseError` using:

```typescript
import { JSONParseError } from 'ai';

if (JSONParseError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_LoadAPIKeyError
description: Learn how to fix AI_LoadAPIKeyError
---

# AI_LoadAPIKeyError

This error occurs when API key is not loaded successfully.

## Properties

- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_LoadAPIKeyError` using:

```typescript
import { LoadAPIKeyError } from 'ai';

if (LoadAPIKeyError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_LoadSettingError
description: Learn how to fix AI_LoadSettingError
---

# AI_LoadSettingError

This error occurs when a setting is not loaded successfully.

## Properties

- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_LoadSettingError` using:

```typescript
import { LoadSettingError } from 'ai';

if (LoadSettingError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_MessageConversionError
description: Learn how to fix AI_MessageConversionError
---

# AI_MessageConversionError

This error occurs when message conversion fails.

## Properties

- `originalMessage`: The original message that failed conversion
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_MessageConversionError` using:

```typescript
import { MessageConversionError } from 'ai';

if (MessageConversionError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_NoContentGeneratedError
description: Learn how to fix AI_NoContentGeneratedError
---

# AI_NoContentGeneratedError

This error occurs when the AI provider fails to generate content.

## Properties

- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_NoContentGeneratedError` using:

```typescript
import { NoContentGeneratedError } from 'ai';

if (NoContentGeneratedError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_NoObjectGeneratedError
description: Learn how to fix AI_NoObjectGeneratedError
---

# AI_NoObjectGeneratedError

This error occurs when the AI provider fails to generate a parsable object.

## Properties

- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_NoObjectGeneratedError` using:

```typescript
import { NoObjectGeneratedError } from 'ai';

if (NoObjectGeneratedError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_NoSuchModelError
description: Learn how to fix AI_NoSuchModelError
---

# AI_NoSuchModelError

This error occurs when a model ID is not found.

## Properties

- `modelId`: The ID of the model that was not found
- `modelType`: The type of model
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_NoSuchModelError` using:

```typescript
import { NoSuchModelError } from 'ai';

if (NoSuchModelError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_NoSuchProviderError
description: Learn how to fix AI_NoSuchProviderError
---

# AI_NoSuchProviderError

This error occurs when a provider ID is not found.

## Properties

- `providerId`: The ID of the provider that was not found
- `availableProviders`: Array of available provider IDs
- `modelId`: The ID of the model
- `modelType`: The type of model
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_NoSuchProviderError` using:

```typescript
import { NoSuchProviderError } from 'ai';

if (NoSuchProviderError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_NoSuchToolError
description: Learn how to fix AI_NoSuchToolError
---

# AI_NoSuchToolError

This error occurs when a model tries to call an unavailable tool.

## Properties

- `toolName`: The name of the tool that was not found
- `availableTools`: Array of available tool names
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_NoSuchToolError` using:

```typescript
import { NoSuchToolError } from 'ai';

if (NoSuchToolError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_RetryError
description: Learn how to fix AI_RetryError
---

# AI_RetryError

This error occurs when a retry operation fails.

## Properties

- `reason`: The reason for the retry failure
- `lastError`: The most recent error that occurred during retries
- `errors`: Array of all errors that occurred during retry attempts
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_RetryError` using:

```typescript
import { RetryError } from 'ai';

if (RetryError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_TooManyEmbeddingValuesForCallError
description: Learn how to fix AI_TooManyEmbeddingValuesForCallError
---

# AI_TooManyEmbeddingValuesForCallError

This error occurs when too many values are provided in a single embedding call.

## Properties

- `provider`: The AI provider name
- `modelId`: The ID of the embedding model
- `maxEmbeddingsPerCall`: The maximum number of embeddings allowed per call
- `values`: The array of values that was provided

## Checking for this Error

You can check if an error is an instance of `AI_TooManyEmbeddingValuesForCallError` using:

```typescript
import { TooManyEmbeddingValuesForCallError } from 'ai';

if (TooManyEmbeddingValuesForCallError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_TypeValidationError
description: Learn how to fix AI_TypeValidationError
---

# AI_TypeValidationError

This error occurs when type validation fails.

## Properties

- `value`: The value that failed validation
- `message`: The error message including validation details

## Checking for this Error

You can check if an error is an instance of `AI_TypeValidationError` using:

```typescript
import { TypeValidationError } from 'ai';

if (TypeValidationError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI_UnsupportedFunctionalityError
description: Learn how to fix AI_UnsupportedFunctionalityError
---

# AI_UnsupportedFunctionalityError

This error occurs when functionality is not unsupported.

## Properties

- `functionality`: The name of the unsupported functionality
- `message`: The error message

## Checking for this Error

You can check if an error is an instance of `AI_UnsupportedFunctionalityError` using:

```typescript
import { UnsupportedFunctionalityError } from 'ai';

if (UnsupportedFunctionalityError.isInstance(error)) {
  // Handle the error
}
```

---
title: AI SDK Errors
description: Troubleshooting information for common AI SDK errors.
collapsed: true
---

# AI SDK Errors

- [AI_InvalidDataContentError](/docs/reference/ai-sdk-errors/ai-invalid-data-content-error)
- [AI_APICallError](/docs/reference/ai-sdk-errors/ai-api-call-error)
- [AI_EmptyResponseBodyError](/docs/reference/ai-sdk-errors/ai-empty-response-body-error)
- [AI_InvalidPromptError](/docs/reference/ai-sdk-errors/ai-invalid-prompt-error)
- [AI_InvalidResponseDataError](/docs/reference/ai-sdk-errors/ai-invalid-response-data-error)
- [AI_JSONParseError](/docs/reference/ai-sdk-errors/ai-json-parse-error)
- [AI_LoadAPIKeyError](/docs/reference/ai-sdk-errors/ai-load-api-key-error)
- [AI_LoadSettingError](/docs/reference/ai-sdk-errors/ai-load-setting-error)
- [AI_NoContentGeneratedError](/docs/reference/ai-sdk-errors/ai-no-content-generated-error)
- [AI_NoSuchModelError](/docs/reference/ai-sdk-errors/ai-no-such-model-error)
- [AI_NoSuchProviderError](/docs/reference/ai-sdk-errors/ai-no-such-provider-error)
- [AI_NoSuchToolError](/docs/reference/ai-sdk-errors/ai-no-such-tool-error)
- [AI_NoObjectGeneratedError](/docs/reference/ai-sdk-errors/ai-no-object-generated-error)
- [AI_TooManyEmbeddingValuesForCallError](/docs/reference/ai-sdk-errors/ai-too-many-embedding-values-for-call-error)
- [AI_TypeValidationError](/docs/reference/ai-sdk-errors/ai-type-validation-error)
- [AI_UnsupportedFunctionalityError](/docs/reference/ai-sdk-errors/ai-unsupported-functionality-error)
- [AI_InvalidArgumentError](/docs/reference/ai-sdk-errors/ai-invalid-argument-error)
- [AI_InvalidToolArgumentsError](/docs/reference/ai-sdk-errors/ai-invalid-tool-arguments-error)
- [AI_RetryError](/docs/reference/ai-sdk-errors/ai-retry-error)
- [AI_DownloadError](/docs/reference/ai-sdk-errors/ai-download-error)
- [AI_InvalidDataContent](/docs/reference/ai-sdk-errors/ai-invalid-data-content)
- [AI_InvalidMessageRoleError](/docs/reference/ai-sdk-errors/ai-invalid-message-role-error)
- [AI_MessageConversionError](/docs/reference/ai-sdk-errors/ai-message-conversion-error)

---
title: Reference
description: Reference documentation for the AI SDK
---

# API Reference

<IndexCards
  cards={[
    {
      title: 'AI SDK Core',
      description: 'Switch between model providers without changing your code.',
      href: '/docs/reference/ai-sdk-core',
    },
    {
      title: 'AI SDK RSC',
      description:
        'Use React Server Components to stream user interfaces to the client.',
      href: '/docs/reference/ai-sdk-rsc',
    },
    {
      title: 'AI SDK UI',
      description:
        'Use hooks to integrate user interfaces that interact with language models.',
      href: '/docs/reference/ai-sdk-ui',
    },
    {
      title: 'Stream Helpers',
      description:
        'Use special functions that help stream model generations from various providers.',
      href: '/docs/reference/stream-helpers',
    },
  ]}
/>

---
title: Migrate AI SDK 3.4 to 4.0
description: Learn how to upgrade AI SDK 3.4 to 4.0.
---

# Migrate AI SDK 3.4 to 4.0

## Recommended Migration Process

1. Backup your project. If you use a versioning control system, make sure all previous versions are committed.
1. [Migrate to AI SDK 3.4](/docs/troubleshooting/migration-guide/migration-guide-3-4).
1. Upgrade to AI SDK 4.0.
1. Automatically migrate your code using [codemods](#codemods).
   > If you don't want to use codemods, we recommend resolving all deprecation warnings before upgrading to AI SDK 4.0.
1. Follow the breaking changes guide below.
1. Verify your project is working as expected.
1. Commit your changes.

## AI SDK 4.0 package versions

You need to update the following packages to the following versions in your `package.json` file(s):

- `ai` package: `4.0.*`
- `ai-sdk@provider-utils` package: `2.0.*`
- `ai-sdk/*` packages: `1.0.*` (other `@ai-sdk` packages)

## Codemods

The AI SDK provides Codemod transformations to help upgrade your codebase when a
feature is deprecated, removed, or otherwise changed.

Codemods are transformations that run on your codebase programmatically. They
allow you to easily apply many changes without having to manually go through
every file.

<Note>
  Codemods are intended as a tool to help you with the upgrade process. They may
  not cover all of the changes you need to make. You may need to make additional
  changes manually.
</Note>

You can run all codemods provided as part of the 4.0 upgrade process by running
the following command from the root of your project:

```sh
npx @ai-sdk/codemod upgrade
```

Individual codemods can be run by specifying the name of the codemod:

```sh
npx @ai-sdk/codemod <codemod-name> <path>
```

See also the [table of codemods](#codemod-table). In addition, the latest set of
codemods can be found in the
[`@ai-sdk/codemod`](https://github.com/vercel/ai/tree/main/packages/codemod/src/codemods)
repository.

## Provider Changes

### Removed `baseUrl` option

The `baseUrl` option has been removed from all providers. Please use the `baseURL` option instead.

```ts filename="AI SDK 3.4"
const perplexity = createOpenAI({
  // ...
  baseUrl: 'https://api.perplexity.ai/',
});
```

```ts filename="AI SDK 4.0"
const perplexity = createOpenAI({
  // ...
  baseURL: 'https://api.perplexity.ai/',
});
```

### Anthropic Provider

#### Removed `Anthropic` facade

The `Anthropic` facade has been removed from the Anthropic provider.
Please use the `anthropic` object or the `createAnthropic` function instead.

```ts filename="AI SDK 3.4"
const anthropic = new Anthropic({
  // ...
});
```

```ts filename="AI SDK 4.0"
const anthropic = createAnthropic({
  // ...
});
```

#### Removed `topK` setting

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The model specific `topK` setting has been removed from the Anthropic provider.
You can use the standard `topK` setting instead.

```ts filename="AI SDK 3.4"
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-latest', {
    topK: 0.5,
  }),
});
```

```ts filename="AI SDK 4.0"
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-latest'),
  topK: 0.5,
});
```

### Google Generative AI Provider

#### Removed `Google` facade

The `Google` facade has been removed from the Google Generative AI provider.
Please use the `google` object or the `createGoogleGenerativeAI` function instead.

```ts filename="AI SDK 3.4"
const google = new Google({
  // ...
});
```

```ts filename="AI SDK 4.0"
const google = createGoogleGenerativeAI({
  // ...
});
```

#### Removed `topK` setting

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The model-specific `topK` setting has been removed from the Google Generative AI provider.
You can use the standard `topK` setting instead.

```ts filename="AI SDK 3.4"
const result = await generateText({
  model: google('gemini-1.5-flash', {
    topK: 0.5,
  }),
});
```

```ts filename="AI SDK 4.0"
const result = await generateText({
  model: google('gemini-1.5-flash'),
  topK: 0.5,
});
```

### Google Vertex Provider

#### Removed `topK` setting

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The model-specific `topK` setting has been removed from the Google Vertex provider.
You can use the standard `topK` setting instead.

```ts filename="AI SDK 3.4"
const result = await generateText({
  model: vertex('gemini-1.5-flash', {
    topK: 0.5,
  }),
});
```

```ts filename="AI SDK 4.0"
const result = await generateText({
  model: vertex('gemini-1.5-flash'),
  topK: 0.5,
});
```

### Mistral Provider

#### Removed `Mistral` facade

The `Mistral` facade has been removed from the Mistral provider.
Please use the `mistral` object or the `createMistral` function instead.

```ts filename="AI SDK 3.4"
const mistral = new Mistral({
  // ...
});
```

```ts filename="AI SDK 4.0"
const mistral = createMistral({
  // ...
});
```

### OpenAI Provider

#### Removed `OpenAI` facade

The `OpenAI` facade has been removed from the OpenAI provider.
Please use the `openai` object or the `createOpenAI` function instead.

```ts filename="AI SDK 3.4"
const openai = new OpenAI({
  // ...
});
```

```ts filename="AI SDK 4.0"
const openai = createOpenAI({
  // ...
});
```

### LangChain Adapter

#### Removed `toAIStream`

The `toAIStream` function has been removed from the LangChain adapter.
Please use the `toDataStream` function instead.

```ts filename="AI SDK 3.4"
LangChainAdapter.toAIStream(stream);
```

```ts filename="AI SDK 4.0"
LangChainAdapter.toDataStream(stream);
```

## AI SDK Core Changes

### `streamText` returns immediately

Instead of returning a Promise, the `streamText` function now returns immediately.
It is not necessary to await the result of `streamText`.

```ts filename="AI SDK 3.4"
const result = await streamText({
  // ...
});
```

```ts filename="AI SDK 4.0"
const result = streamText({
  // ...
});
```

### `streamObject` returns immediately

Instead of returning a Promise, the `streamObject` function now returns immediately.
It is not necessary to await the result of `streamObject`.

```ts filename="AI SDK 3.4"
const result = await streamObject({
  // ...
});
```

```ts filename="AI SDK 4.0"
const result = streamObject({
  // ...
});
```

### Remove roundtrips

The `maxToolRoundtrips` and `maxAutomaticRoundtrips` options have been removed from the `generateText` and `streamText` functions.
Please use the `maxSteps` option instead.

The `roundtrips` property has been removed from the `GenerateTextResult` type.
Please use the `steps` property instead.

```ts filename="AI SDK 3.4"
const { text, roundtrips } = await generateText({
  maxToolRoundtrips: 1, // or maxAutomaticRoundtrips
  // ...
});
```

```ts filename="AI SDK 4.0"
const { text, steps } = await generateText({
  maxSteps: 2,
  // ...
});
```

### Removed `nanoid` export

The `nanoid` export has been removed. Please use [`generateId`](/docs/reference/stream-helpers/generate-id) instead.

```ts filename="AI SDK 3.4"
import { nanoid } from 'ai';
```

```ts filename="AI SDK 4.0"
import { generateId } from 'ai';
```

### Increased default size of generated IDs

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The [`generateId`](/docs/reference/stream-helpers/generate-id) function now
generates 16-character IDs. The previous default was 7 characters.

This might e.g. require updating your database schema if you limit the length of
IDs.

```ts filename="AI SDK 4.0"
import { generateId } from 'ai';

const id = generateId(); // now 16 characters
```

### Removed `ExperimentalMessage` types

The following types have been removed:

- `ExperimentalMessage` (use `CoreMessage` instead)
- `ExperimentalUserMessage` (use `CoreUserMessage` instead)
- `ExperimentalAssistantMessage` (use `CoreAssistantMessage` instead)
- `ExperimentalToolMessage` (use `CoreToolMessage` instead)

```ts filename="AI SDK 3.4"
import {
  ExperimentalMessage,
  ExperimentalUserMessage,
  ExperimentalAssistantMessage,
  ExperimentalToolMessage,
} from 'ai';
```

```ts filename="AI SDK 4.0"
import {
  CoreMessage,
  CoreUserMessage,
  CoreAssistantMessage,
  CoreToolMessage,
} from 'ai';
```

### Removed `ExperimentalTool` type

The `ExperimentalTool` type has been removed. Please use the `CoreTool` type instead.

```ts filename="AI SDK 3.4"
import { ExperimentalTool } from 'ai';
```

```ts filename="AI SDK 4.0"
import { CoreTool } from 'ai';
```

### Removed experimental AI function exports

The following exports have been removed:

- `experimental_generateText` (use `generateText` instead)
- `experimental_streamText` (use `streamText` instead)
- `experimental_generateObject` (use `generateObject` instead)
- `experimental_streamObject` (use `streamObject` instead)

```ts filename="AI SDK 3.4"
import {
  experimental_generateText,
  experimental_streamText,
  experimental_generateObject,
  experimental_streamObject,
} from 'ai';
```

```ts filename="AI SDK 4.0"
import { generateText, streamText, generateObject, streamObject } from 'ai';
```

### Removed AI-stream related methods from `streamText`

The following methods have been removed from the `streamText` result:

- `toAIStream`
- `pipeAIStreamToResponse`
- `toAIStreamResponse`

Use the `toDataStream`, `pipeDataStreamToResponse`, and `toDataStreamResponse` functions instead.

```ts filename="AI SDK 3.4"
const result = await streamText({
  // ...
});

result.toAIStream();
result.pipeAIStreamToResponse(response);
result.toAIStreamResponse();
```

```ts filename="AI SDK 4.0"
const result = streamText({
  // ...
});

result.toDataStream();
result.pipeDataStreamToResponse(response);
result.toDataStreamResponse();
```

### Renamed "formatStreamPart" to "formatDataStreamPart"

The `formatStreamPart` function has been renamed to `formatDataStreamPart`.

```ts filename="AI SDK 3.4"
formatStreamPart('text', 'Hello, world!');
```

```ts filename="AI SDK 4.0"
formatDataStreamPart('text', 'Hello, world!');
```

### Renamed "parseStreamPart" to "parseDataStreamPart"

The `parseStreamPart` function has been renamed to `parseDataStreamPart`.

```ts filename="AI SDK 3.4"
const part = parseStreamPart(line);
```

```ts filename="AI SDK 4.0"
const part = parseDataStreamPart(line);
```

### Renamed `TokenUsage`, `CompletionTokenUsage` and `EmbeddingTokenUsage` types

The `TokenUsage`, `CompletionTokenUsage` and `EmbeddingTokenUsage` types have been
to `LanguageModelUsage` and `EmbeddingModelUsage`.

```ts filename="AI SDK 3.4"
import { TokenUsage, CompletionTokenUsage, EmbeddingTokenUsage } from 'ai';
```

```ts filename="AI SDK 4.0"
import { LanguageModelUsage, EmbeddingModelUsage } from 'ai';
```

### Removed deprecated telemetry data

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The following telemetry data values have been removed:

- `ai.finishReason` (now in `ai.response.finishReason`)
- `ai.result.object` (now in `ai.response.object`)
- `ai.result.text` (now in `ai.response.text`)
- `ai.result.toolCalls` (now in `ai.response.toolCalls`)
- `ai.stream.msToFirstChunk` (now in `ai.response.msToFirstChunk`)

This change will apply to observability providers and any scripts or automation that you use for processing telemetry data.

### Provider Registry

#### Removed experimental_Provider, experimental_ProviderRegistry, and experimental_ModelRegistry

The `experimental_Provider` interface, `experimental_ProviderRegistry` interface, and `experimental_ModelRegistry` interface have been removed.
Please use the `Provider` interface instead.

```ts filename="AI SDK 3.4"
import { experimental_Provider, experimental_ProviderRegistry } from 'ai';
```

```ts filename="AI SDK 4.0"
import { Provider } from 'ai';
```

<Note>
  The model registry is not available any more. Please [register
  providers](/docs/reference/ai-sdk-core/provider-registry#setup) instead.
</Note>

#### Removed `experimental_​createModelRegistry` function

The `experimental_createModelRegistry` function has been removed.
Please use the `experimental_createProviderRegistry` function instead.

```ts filename="AI SDK 3.4"
import { experimental_createModelRegistry } from 'ai';
```

```ts filename="AI SDK 4.0"
import { experimental_createProviderRegistry } from 'ai';
```

<Note>
  The model registry is not available any more. Please [register
  providers](/docs/reference/ai-sdk-core/provider-registry#setup) instead.
</Note>

### Removed `rawResponse` from results

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `rawResponse` property has been removed from the `generateText`, `streamText`, `generateObject`, and `streamObject` results.
You can use the `response` property instead.

```ts filename="AI SDK 3.4"
const { text, rawResponse } = await generateText({
  // ...
});
```

```ts filename="AI SDK 4.0"
const { text, response } = await generateText({
  // ...
});
```

### Removed `init` option from `pipeDataStreamToResponse` and `toDataStreamResponse`

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `init` option has been removed from the `pipeDataStreamToResponse` and `toDataStreamResponse` functions.
You can set the values from `init` directly into the `options` object.

```ts filename="AI SDK 3.4"
const result = await streamText({
  // ...
});

result.toDataStreamResponse(response, {
  init: {
    headers: {
      'X-Custom-Header': 'value',
    },
  },
  // ...
});
```

```ts filename="AI SDK 4.0"
const result = streamText({
  // ...
});

result.toDataStreamResponse(response, {
  headers: {
    'X-Custom-Header': 'value',
  },
  // ...
});
```

### Removed `responseMessages` from `generateText` and `streamText`

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `responseMessages` property has been removed from the `generateText` and `streamText` results.
This includes the `onFinish` callback.
Please use the `response.messages` property instead.

```ts filename="AI SDK 3.4"
const { text, responseMessages } = await generateText({
  // ...
});
```

```ts filename="AI SDK 4.0"
const { text, response } = await generateText({
  // ...
});

const responseMessages = response.messages;
```

### Removed `experimental_​continuationSteps` option

The `experimental_continuationSteps` option has been removed from the `generateText` function.
Please use the `experimental_continueSteps` option instead.

```ts filename="AI SDK 3.4"
const result = await generateText({
  experimental_continuationSteps: true,
  // ...
});
```

```ts filename="AI SDK 4.0"
const result = await generateText({
  experimental_continueSteps: true,
  // ...
});
```

### Removed `LanguageModelResponseMetadataWithHeaders` type

The `LanguageModelResponseMetadataWithHeaders` type has been removed.
Please use the `LanguageModelResponseMetadata` type instead.

```ts filename="AI SDK 3.4"
import { LanguageModelResponseMetadataWithHeaders } from 'ai';
```

```ts filename="AI SDK 4.0"
import { LanguageModelResponseMetadata } from 'ai';
```

#### Changed `streamText` warnings result to Promise

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `warnings` property of the `StreamTextResult` type is now a Promise.

```ts filename="AI SDK 3.4"
const result = await streamText({
  // ...
});

const warnings = result.warnings;
```

```ts filename="AI SDK 4.0"
const result = streamText({
  // ...
});

const warnings = await result.warnings;
```

#### Changed `streamObject` warnings result to Promise

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `warnings` property of the `StreamObjectResult` type is now a Promise.

```ts filename="AI SDK 3.4"
const result = await streamObject({
  // ...
});

const warnings = result.warnings;
```

```ts filename="AI SDK 4.0"
const result = streamObject({
  // ...
});

const warnings = await result.warnings;
```

#### Renamed `simulateReadableStream` `values` to `chunks`

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `simulateReadableStream` function from `ai/test` has been renamed to `chunks`.

```ts filename="AI SDK 3.4"
import { simulateReadableStream } from 'ai/test';

const stream = simulateReadableStream({
  values: [1, 2, 3],
  chunkDelayInMs: 100,
});
```

```ts filename="AI SDK 4.0"
import { simulateReadableStream } from 'ai/test';

const stream = simulateReadableStream({
  chunks: [1, 2, 3],
  chunkDelayInMs: 100,
});
```

## AI SDK RSC Changes

<Note type="warning">
  There are no codemods available for the changes in this section. Please review
  and update your code manually.
</Note>

### Removed `render` function

The AI SDK RSC 3.0 `render` function has been removed.
Please use the `streamUI` function instead or [switch to AI SDK UI](/docs/ai-sdk-rsc/migrating-to-ui).

```ts filename="AI SDK 3.0"
import { render } from 'ai/rsc';
```

```ts filename="AI SDK 4.0"
import { streamUI } from 'ai/rsc';
```

## AI SDK UI Changes

### Removed Svelte, Vue, and SolidJS exports

<Note type="warning">
  This codemod only operates on `.ts` and `.tsx` files. If you have code in
  files with other suffixes, please review and update your code manually.
</Note>

The `ai` package no longer exports Svelte, Vue, and SolidJS UI integrations.
You need to install the `@ai-sdk/svelte`, `@ai-sdk/vue`, and `@ai-sdk/solid` packages directly.

```ts filename="AI SDK 3.4"
import { useChat } from 'ai/svelte';
```

```ts filename="AI SDK 4.0"
import { useChat } from '@ai-sdk/svelte';
```

### Removed `experimental_StreamData`

The `experimental_StreamData` export has been removed.
Please use the `StreamData` export instead.

```ts filename="AI SDK 3.4"
import { experimental_StreamData } from 'ai';
```

```ts filename="AI SDK 4.0"
import { StreamData } from 'ai';
```

### `useChat` hook

<Note type="warning">
  There are no codemods available for the changes in this section. Please review
  and update your code manually.
</Note>

#### Removed `streamMode` setting

The `streamMode` options has been removed from the `useChat` hook.
Please use the `streamProtocol` parameter instead.

```ts filename="AI SDK 3.4"
const { messages } = useChat({
  streamMode: 'text',
  // ...
});
```

```ts filename="AI SDK 4.0"
const { messages } = useChat({
  streamProtocol: 'text',
  // ...
});
```

#### Replaced roundtrip setting with `maxSteps`

The following options have been removed from the `useChat` hook:

- `experimental_maxAutomaticRoundtrips`
- `maxAutomaticRoundtrips`
- `maxToolRoundtrips`

Please use the [`maxSteps`](/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls) option instead.
The value of `maxSteps` is equal to roundtrips + 1.

```ts filename="AI SDK 3.4"
const { messages } = useChat({
  experimental_maxAutomaticRoundtrips: 2,
  // or maxAutomaticRoundtrips
  // or maxToolRoundtrips
  // ...
});
```

```ts filename="AI SDK 4.0"
const { messages } = useChat({
  maxSteps: 3, // 2 roundtrips + 1
  // ...
});
```

#### Removed `options` setting

The `options` parameter in the `useChat` hook has been removed.
Please use the `headers` and `body` parameters instead.

```ts filename="AI SDK 3.4"
const { messages } = useChat({
  options: {
    headers: {
      'X-Custom-Header': 'value',
    },
  },
  // ...
});
```

```ts filename="AI SDK 4.0"
const { messages } = useChat({
  headers: {
    'X-Custom-Header': 'value',
  },
  // ...
});
```

#### Removed `experimental_addToolResult` method

The `experimental_addToolResult` method has been removed from the `useChat` hook.
Please use the `addToolResult` method instead.

```ts filename="AI SDK 3.4"
const { messages, experimental_addToolResult } = useChat({
  // ...
});
```

```ts filename="AI SDK 4.0"
const { messages, addToolResult } = useChat({
  // ...
});
```

#### Changed default value of `keepLastMessageOnError` to true and deprecated the option

The `keepLastMessageOnError` option has been changed to default to `true`.
The option will be removed in the next major release.

```ts filename="AI SDK 3.4"
const { messages } = useChat({
  keepLastMessageOnError: true,
  // ...
});
```

```ts filename="AI SDK 4.0"
const { messages } = useChat({
  // ...
});
```

### `useCompletion` hook

<Note type="warning">
  There are no codemods available for the changes in this section. Please review
  and update your code manually.
</Note>

#### Removed `streamMode` setting

The `streamMode` options has been removed from the `useCompletion` hook.
Please use the `streamProtocol` parameter instead.

```ts filename="AI SDK 3.4"
const { text } = useCompletion({
  streamMode: 'text',
  // ...
});
```

```ts filename="AI SDK 4.0"
const { text } = useCompletion({
  streamProtocol: 'text',
  // ...
});
```

### `useAssistant` hook

#### Removed `experimental_useAssistant` export

The `experimental_useAssistant` export has been removed from the `useAssistant` hook.
Please use the `useAssistant` hook directly instead.

```ts filename="AI SDK 3.4"
import { experimental_useAssistant } from 'ai/react';
```

```ts filename="AI SDK 4.0"
import { useAssistant } from 'ai/react';
```

#### Removed `threadId` and `messageId` from `AssistantResponse`

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `threadId` and `messageId` parameters have been removed from the `AssistantResponse` function.
Please use the `threadId` and `messageId` variables from the outer scope instead.

```ts filename="AI SDK 3.4"
return AssistantResponse(
  { threadId: myThreadId, messageId: myMessageId },
  async ({ forwardStream, sendDataMessage, threadId, messageId }) => {
    // use threadId and messageId here
  },
);
```

```ts filename="AI SDK 4.0"
return AssistantResponse(
  { threadId: myThreadId, messageId: myMessageId },
  async ({ forwardStream, sendDataMessage }) => {
    // use myThreadId and myMessageId here
  },
);
```

#### Removed `experimental_​AssistantResponse` export

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `experimental_AssistantResponse` export has been removed.
Please use the `AssistantResponse` function directly instead.

```ts filename="AI SDK 3.4"
import { experimental_AssistantResponse } from 'ai';
```

```ts filename="AI SDK 4.0"
import { AssistantResponse } from 'ai';
```

### `experimental_useObject` hook

<Note type="warning">
  There are no codemods available for the changes in this section. Please review
  and update your code manually.
</Note>

The `setInput` helper has been removed from the `experimental_useObject` hook.
Please use the `submit` helper instead.

```ts filename="AI SDK 3.4"
const { object, setInput } = useObject({
  // ...
});
```

```ts filename="AI SDK 4.0"
const { object, submit } = useObject({
  // ...
});
```

## AI SDK Errors

### Removed `isXXXError` static methods

The `isXXXError` static methods have been removed from AI SDK errors.
Please use the `isInstance` method of the corresponding error class instead.

```ts filename="AI SDK 3.4"
import { APICallError } from 'ai';

APICallError.isAPICallError(error);
```

```ts filename="AI SDK 4.0"
import { APICallError } from 'ai';

APICallError.isInstance(error);
```

### Removed `toJSON` method

<Note type="warning">
  There is no codemod available for this change. Please review and update your
  code manually.
</Note>

The `toJSON` method has been removed from AI SDK errors.

## AI SDK 2.x Legacy Changes

<Note type="warning">
  There are no codemods available for the changes in this section. Please review
  and update your code manually.
</Note>

### Removed 2.x legacy providers

Legacy providers from AI SDK 2.x have been removed. Please use the new [AI SDK provider architecture](/docs/foundations/providers-and-models) instead.

#### Removed 2.x legacy function and tool calling

The legacy `function_call` and `tools` options have been removed from `useChat` and `Message`.
The `name` property from the `Message` type has been removed.
Please use the [AI SDK Core tool calling](/docs/ai-sdk-core/tools-and-tool-calling) instead.

### Removed 2.x prompt helpers

Prompt helpers for constructing message prompts are no longer needed with the AI SDK provider architecture and have been removed.

### Removed 2.x `AIStream`

The `AIStream` function and related exports have been removed.
Please use the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function and its `toDataStream()` method instead.

### Removed 2.x `StreamingTextResponse`

The `StreamingTextResponse` function has been removed.
Please use the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function and its `toDataStreamResponse()` method instead.

### Removed 2.x `streamToResponse`

The `streamToResponse` function has been removed.
Please use the [`streamText`](/docs/reference/ai-sdk-core/stream-text) function and its `pipeDataStreamToResponse()` method instead.

### Removed 2.x RSC `Tokens` streaming

The legacy `Tokens` RSC streaming from 2.x has been removed.
`Tokens` were implemented prior to AI SDK RSC and are no longer needed.

## Codemod Table

The following table lists codemod availability for the AI SDK 4.0 upgrade
process. Note the codemod `upgrade` command will run all of them for you. This
list is provided to give visibility into which migrations have some automation.
It can also be helpful to find the codemod names if you'd like to run a subset
of codemods. For more, see the [Codemods](#codemods) section.

| Change                                                                                              | Codemod                                            |
| --------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| **Provider Changes**                                                                                |                                                    |
| Removed baseUrl option                                                                              | `replace-baseurl`                                  |
| **Anthropic Provider**                                                                              |                                                    |
| Removed Anthropic facade                                                                            | `remove-anthropic-facade`                          |
| Removed topK setting                                                                                | _N/A_                                              |
| **Google Generative AI Provider**                                                                   |                                                    |
| Removed Google facade                                                                               | `remove-google-facade`                             |
| Removed topK setting                                                                                | _N/A_                                              |
| **Google Vertex Provider**                                                                          |                                                    |
| Removed topK setting                                                                                | _N/A_                                              |
| **Mistral Provider**                                                                                |                                                    |
| Removed Mistral facade                                                                              | `remove-mistral-facade`                            |
| **OpenAI Provider**                                                                                 |                                                    |
| Removed OpenAI facade                                                                               | `remove-openai-facade`                             |
| **LangChain Adapter**                                                                               |                                                    |
| Removed toAIStream                                                                                  | `replace-langchain-toaistream`                     |
| **AI SDK Core Changes**                                                                             |                                                    |
| streamText returns immediately                                                                      | `remove-await-streamtext`                          |
| streamObject returns immediately                                                                    | `remove-await-streamobject`                        |
| Remove roundtrips                                                                                   | `replace-roundtrips-with-maxsteps`                 |
| Removed nanoid export                                                                               | `replace-nanoid`                                   |
| Increased default size of generated IDs                                                             | _N/A_                                              |
| Removed ExperimentalMessage types                                                                   | `remove-experimental-message-types`                |
| Removed ExperimentalTool type                                                                       | `remove-experimental-tool`                         |
| Removed experimental AI function exports                                                            | `remove-experimental-ai-fn-exports`                |
| Removed AI-stream related methods from streamText                                                   | `remove-ai-stream-methods-from-stream-text-result` |
| Renamed "formatStreamPart" to "formatDataStreamPart"                                                | `rename-format-stream-part`                        |
| Renamed "parseStreamPart" to "parseDataStreamPart"                                                  | `rename-parse-stream-part`                         |
| Renamed TokenUsage, CompletionTokenUsage and EmbeddingTokenUsage types                              | `replace-token-usage-types`                        |
| Removed deprecated telemetry data                                                                   | _N/A_                                              |
| **Provider Registry**                                                                               |                                                    |
| &rarr; Removed experimental_Provider, experimental_ProviderRegistry, and experimental_ModelRegistry | `remove-deprecated-provider-registry-exports`      |
| &rarr; Removed experimental_createModelRegistry function                                            | _N/A_                                              |
| Removed rawResponse from results                                                                    | _N/A_                                              |
| Removed init option from pipeDataStreamToResponse and toDataStreamResponse                          | _N/A_                                              |
| Removed responseMessages from generateText and streamText                                           | _N/A_                                              |
| Removed experimental_continuationSteps option                                                       | `replace-continuation-steps`                       |
| Removed LanguageModelResponseMetadataWithHeaders type                                               | `remove-metadata-with-headers`                     |
| Changed streamText warnings result to Promise                                                       | _N/A_                                              |
| Changed streamObject warnings result to Promise                                                     | _N/A_                                              |
| Renamed simulateReadableStream values to chunks                                                     | _N/A_                                              |
| **AI SDK RSC Changes**                                                                              |                                                    |
| Removed render function                                                                             | _N/A_                                              |
| **AI SDK UI Changes**                                                                               |                                                    |
| Removed Svelte, Vue, and SolidJS exports                                                            | `rewrite-framework-imports`                        |
| Removed experimental_StreamData                                                                     | `remove-experimental-streamdata`                   |
| **useChat hook**                                                                                    |                                                    |
| Removed streamMode setting                                                                          | _N/A_                                              |
| Replaced roundtrip setting with maxSteps                                                            | `replace-roundtrips-with-maxsteps`                 |
| Removed options setting                                                                             | _N/A_                                              |
| Removed experimental_addToolResult method                                                           | _N/A_                                              |
| Changed default value of keepLastMessageOnError to true and deprecated the option                   | _N/A_                                              |
| **useCompletion hook**                                                                              |                                                    |
| Removed streamMode setting                                                                          | _N/A_                                              |
| **useAssistant hook**                                                                               |                                                    |
| Removed experimental_useAssistant export                                                            | `remove-experimental-useassistant`                 |
| Removed threadId and messageId from AssistantResponse                                               | _N/A_                                              |
| Removed experimental_AssistantResponse export                                                       | _N/A_                                              |
| **experimental_useObject hook**                                                                     |                                                    |
| Removed setInput helper                                                                             | _N/A_                                              |
| **AI SDK Errors**                                                                                   |                                                    |
| Removed isXXXError static methods                                                                   | `remove-isxxxerror`                                |
| Removed toJSON method                                                                               | _N/A_                                              |
| **AI SDK 2.x Legacy Changes**                                                                       |                                                    |
| Removed 2.x legacy providers                                                                        | _N/A_                                              |
| Removed 2.x legacy function and tool calling                                                        | _N/A_                                              |
| Removed 2.x prompt helpers                                                                          | _N/A_                                              |
| Removed 2.x AIStream                                                                                | _N/A_                                              |
| Removed 2.x StreamingTextResponse                                                                   | _N/A_                                              |
| Removed 2.x streamToResponse                                                                        | _N/A_                                              |
| Removed 2.x RSC Tokens streaming                                                                    | _N/A_                                              |

---
title: Migrate AI SDK 3.3 to 3.4
description: Learn how to upgrade AI SDK 3.3 to 3.4.
---

# Migrate AI SDK 3.3 to 3.4

<Note>
  Check out the [AI SDK 3.4 release blog
  post](https://vercel.com/blog/ai-sdk-3-4) for more information about the
  release.
</Note>

No breaking changes in this release.

---
title: Migrate AI SDK 3.2 to 3.3
description: Learn how to upgrade AI SDK 3.2 to 3.3.
---

# Migrate AI SDK 3.2 to 3.3

<Note>
  Check out the [AI SDK 3.3 release blog
  post](https://vercel.com/blog/vercel-ai-sdk-3-3) for more information about
  the release.
</Note>

No breaking changes in this release.

The following changelog encompasses all changes made in the 3.2.x series,
introducing significant improvements and new features across the AI SDK and its associated libraries:

## New Features

### Open Telemetry Support

- Added experimental [OpenTelemetry support](https://sdk.vercel.ai/docs/ai-sdk-core/telemetry#telemetry) for all [AI SDK Core functions](https://sdk.vercel.ai/docs/ai-sdk-core/overview#ai-sdk-core-functions), enabling better observability and tracing capabilities.

### AI SDK UI Improvements

- Introduced the experimental **`useObject`** hook (for React) that can be used in conjunction with **`streamObject`** on the backend to enable seamless streaming of structured data.
- Enhanced **`useChat`** with experimental support for attachments and streaming tool calls, providing more versatile chat functionalities.
- Patched **`useChat`** to prevent empty submissions, improving the quality of user interactions by ensuring that only intended inputs are processed.
- Fix **`useChat`**'s **`reload`** function, now correctly sending data, body, and headers.
- Implemented **`setThreadId`** helper for **`useAssistant`**, simplifying thread management.
- Documented the stream data protocol for **`useChat`** and **`useCompletion`**, allowing developers to use these functions with any backend. The stream data protocol also enables the use of custom frontends with **`streamText`**.
- Added support for custom fetch functions and request body customization, offering greater control over API interactions.
- Added **`onFinish`** to **`useChat`** hook for access to token usage and finish reason.

### Core Enhancements

- Implemented support for sending custom request headers, enabling more tailored API requests.
- Added raw JSON schema support alongside existing Zod support, providing more options for schema and data validation.
- Introduced usage information for **`embed`** and **`embedMany`** functions, offering insights into token usage.
- Added support for additional settings including **`stopSequences`** and **`topK`**, allowing for finer control over text generation.
- Provided access to information for all steps on **`generateText`**, providing access to intermediate tool calls and results.

### New Providers

- [AWS Bedrock provider](https://sdk.vercel.ai/providers/ai-sdk-providers/amazon-bedrock).
- [Chrome AI provider](https://sdk.vercel.ai/providers/community-providers/chrome-ai) (community-maintained).

### Provider Improvements

- Enhanced existing providers including Anthropic, Google, Azure, and OpenAI with various improvements and bug fixes.
- Upgraded the LangChain adapter with StreamEvent v2 support and introduced the **`toDataStreamResponse`** function, enabling conversion of LangChain output streams to data stream responses.
- Added legacy function calling support to the OpenAI provider.
- Updated Mistral AI provider with fixes and improvements for tool calling support.

### UI Framework Support Expansion

- SolidJS: Updated **`useChat`** and **`useCompletion`** to achieve feature parity with React implementations.
- Vue.js: Introduced **`useAssistant`** hook.
- Vue.js / Nuxt: [Updated examples](https://github.com/vercel/ai/tree/main/examples/nuxt-openai) to showcase latest features and best practices.
- Svelte: Added tool calling support to **`useChat`.**

## Fixes and Improvements

- Resolved various issues across different components of the SDK, including race conditions, error handling, and state management.

---
title: Migrate AI SDK 3.1 to 3.2
description: Learn how to upgrade AI SDK 3.1 to 3.2.
---

# Migrate AI SDK 3.1 to 3.2

<Note>
  Check out the [AI SDK 3.2 release blog
  post](https://vercel.com/blog/introducing-vercel-ai-sdk-3-2) for more
  information about the release.
</Note>

This guide will help you upgrade to AI SDK 3.2:

- Experimental `StreamingReactResponse` functionality has been removed
- Several features have been deprecated
- UI framework integrations have moved to their own Node modules

## Upgrading

### AI SDK

To update to AI SDK version 3.2, run the following command using your preferred package manager:

<Snippet text="pnpm add ai@latest" />

## Removed Functionality

The experimental `StreamingReactResponse` has been removed. You can use [AI SDK RSC](/docs/ai-sdk-rsc/overview) to build streaming UIs.

## Deprecated Functionality

The `nanoid` export has been deprecated. Please use [`generateId`](/docs/reference/stream-helpers/generate-id) instead.

## UI Package Separation

AI SDK UI supports several frameworks: [React](https://react.dev/), [Svelte](https://svelte.dev/), [Vue.js](https://vuejs.org/), and [SolidJS](https://www.solidjs.com/).

The integrations (other than React and RSC) have moved to separate Node modules. You need to update the import and require statements as follows:

- Change `ai/svelte` to `@ai-sdk/svelte`
- Change `ai/vue` to `@ai-sdk/vue`
- Change `ai/solid` to `@ai-sdk/solid`

The old exports are still available but will be removed in a future release.

---
title: Migrate AI SDK 3.0 to 3.1
description: Learn how to upgrade AI SDK 3.0 to 3.1.
---

# Migrate AI SDK 3.0 to 3.1

<Note>
  Check out the [AI SDK 3.1 release blog
  post](https://vercel.com/blog/vercel-ai-sdk-3-1-modelfusion-joins-the-team)
  for more information about the release.
</Note>

This guide will help you:

- Upgrade to AI SDK 3.1
- Migrate from Legacy Providers to AI SDK Core
- Migrate from [`render`](/docs/reference/ai-sdk-rsc/render) to [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui)

Upgrading to AI SDK 3.1 does not require using the newly released AI SDK Core API or [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function.

## Upgrading

### AI SDK

To update to AI SDK version 3.1, run the following command using your preferred package manager:

<Snippet text="pnpm add ai@3.1" />

## Next Steps

The release of AI SDK 3.1 introduces several new features that improve the way you build AI applications with the SDK:

- AI SDK Core, a brand new unified API for interacting with large language models (LLMs).
- [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui), a new abstraction, built upon AI SDK Core functions that simplifies building streaming UIs.

## Migrating from Legacy Providers to AI SDK Core

Prior to AI SDK Core, you had to use a model provider's SDK to query their models.

In the following Route Handler, you use the OpenAI SDK to query their model. You then pipe that response into the [`OpenAIStream`](/docs/reference/stream-helpers/openai-stream) function which returns a [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream) that you can pass to the client using a new [`StreamingTextResponse`](/docs/reference/stream-helpers/streaming-text-response).

```tsx
import OpenAI from 'openai';
import { OpenAIStream, StreamingTextResponse } from 'ai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const response = await openai.chat.completions.create({
    model: 'gpt-4-turbo',
    stream: true,
    messages,
  });

  const stream = OpenAIStream(response);

  return new StreamingTextResponse(stream);
}
```

With AI SDK Core you have a unified API for any provider that implements the [AI SDK Language Model Specification](https://sdk.vercel.ai/providers/community-providers/custom-providers).

Let’s take a look at the example above, but refactored to utilize the AI SDK Core API alongside the AI SDK OpenAI provider. In this example, you import the LLM function you want to use from the `ai` package, import the OpenAI provider from `@ai-sdk/openai`, and then you call the model and return the response using the `toDataStreamResponse()` helper function.

```tsx
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

## Migrating from `render` to `streamUI`

The AI SDK RSC API was launched as part of version 3.0. This API introduced the [`render`](/docs/reference/ai-sdk-rsc/render) function, a helper function to create streamable UIs with OpenAI models. With the new AI SDK Core API, it became possible to make streamable UIs possible with any compatible provider.

The following example Server Action uses the `render` function using the model provider directly from OpenAI. You first create an OpenAI provider instance with the OpenAI SDK. Then, you pass it to the provider key of the render function alongside a tool that returns a React Server Component, defined in the `render` key of the tool.

```tsx
import { render } from 'ai/rsc';
import OpenAI from 'openai';
import { z } from 'zod';
import { Spinner, Weather } from '@/components';
import { getWeather } from '@/utils';

const openai = new OpenAI();

async function submitMessage(userInput = 'What is the weather in SF?') {
  'use server';

  return render({
    provider: openai,
    model: 'gpt-4-turbo',
    messages: [
      { role: 'system', content: 'You are a helpful assistant' },
      { role: 'user', content: userInput },
    ],
    text: ({ content }) => <p>{content}</p>,
    tools: {
      get_city_weather: {
        description: 'Get the current weather for a city',
        parameters: z
          .object({
            city: z.string().describe('the city'),
          })
          .required(),
        render: async function* ({ city }) {
          yield <Spinner />;
          const weather = await getWeather(city);
          return <Weather info={weather} />;
        },
      },
    },
  });
}
```

With the new [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function, you can now use any compatible AI SDK provider. In this example, you import the AI SDK OpenAI provider. Then, you pass it to the [`model`](/docs/reference/ai-sdk-rsc/stream-ui#model) key of the new [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function. Finally, you declare a tool and return a React Server Component, defined in the [`generate`](/docs/reference/ai-sdk-rsc/stream-ui#tools-generate) key of the tool.

```tsx
import { streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';
import { Spinner, Weather } from '@/components';
import { getWeather } from '@/utils';

async function submitMessage(userInput = 'What is the weather in SF?') {
  'use server';

  const result = await streamUI({
    model: openai('gpt-4-turbo'),
    system: 'You are a helpful assistant',
    messages: [{ role: 'user', content: userInput }],
    text: ({ content }) => <p>{content}</p>,
    tools: {
      get_city_weather: {
        description: 'Get the current weather for a city',
        parameters: z
          .object({
            city: z.string().describe('Name of the city'),
          })
          .required(),
        generate: async function* ({ city }) {
          yield <Spinner />;
          const weather = await getWeather(city);
          return <Weather info={weather} />;
        },
      },
    },
  });

  return result.value;
}
```

---
title: Migration Guides
description: Learn how to upgrade between Vercel AI versions.
collapsed: true
---

# Migration Guides

- [ Migrate AI SDK 3.4 to 4.0 ](/docs/migration-guides/migration-guide-4-0)
- [ Migrate AI SDK 3.3 to 3.4 ](/docs/migration-guides/migration-guide-3-4)
- [ Migrate AI SDK 3.2 to 3.3 ](/docs/migration-guides/migration-guide-3-3)
- [ Migrate AI SDK 3.1 to 3.2 ](/docs/migration-guides/migration-guide-3-2)
- [ Migrate AI SDK 3.0 to 3.1 ](/docs/migration-guides/migration-guide-3-1)

---
title: Azure OpenAI Slow to Stream
description: Learn to troubleshoot Azure OpenAI slow to stream issues.
---

# Azure OpenAI Slow To Stream

## Issue

When using OpenAI hosted on Azure, streaming is slow and in big chunks.

## Solution

This is a Microsoft Azure issue. Some users have reported the following solutions:

- **Update Content Filtering Settings**:
  Inside [Azure AI Studio](https://ai.azure.com/), within "Shared resources" > "Content filters", create a new
  content filter and set the "Streaming mode (Preview)" under "Output filter" from "Default"
  to "Asynchronous Filter".

---
title: Client-Side Function Calls Not Invoked
description: Troubleshooting client-side function calls not being invoked.
---

# Client-Side Function Calls Not Invoked

## Issue

I upgraded the AI SDK to v3.0.20 or newer. I am using [`OpenAIStream`](/docs/reference/stream-helpers/openai-stream). Client-side function calls are no longer invoked.

## Solution

You will need to add a stub for `experimental_onFunctionCall` to [`OpenAIStream`](/docs/reference/stream-helpers/openai-stream) to enable the correct forwarding of the function calls to the client.

```tsx
const stream = OpenAIStream(response, {
  async experimental_onFunctionCall() {
    return;
  },
});
```

---
title: Server Actions in Client Components
description: Troubleshooting errors related to server actions in client components.
---

# Server Actions in Client Components

You may use Server Actions in client components, but sometimes you may encounter the following issues.

## Issue

It is not allowed to define inline `"use server"` annotated Server Actions in Client Components.

## Solution

To use Server Actions in a Client Component, you can either:

- Export them from a separate file with `"use server"` at the top.
- Pass them down through props from a Server Component.
- Implement a combination of [`createAI`](/docs/reference/ai-sdk-rsc/create-ai) and [`useActions`](/docs/reference/ai-sdk-rsc/use-actions) hooks to access them.

Learn more about [Server Actions and Mutations](https://nextjs.org/docs/app/api-reference/functions/server-actions#with-client-components).

```ts file='actions.ts'
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function getAnswer(question: string) {
  'use server';

  const { text } = await generateText({
    model: openai.chat('gpt-3.5-turbo'),
    prompt: question,
  });

  return { answer: text };
}
```

---
title: useChat/useCompletion stream output contains 0:... instead of text
description: How to fix strange stream output in the UI
---

# useChat/useCompletion stream output contains 0:... instead of text

## Issue

I am using custom client code to process a server response that is sent using [`StreamingTextResponse`](/docs/reference/stream-helpers/streaming-text-response). I am using version `3.0.20` or newer of the AI SDK. When I send a query, the UI streams text such as `0: "Je"`, `0: " suis"`, `0: "des"...` instead of the text that I’m looking for.

## Background

The AI SDK has switched to the stream data protocol in version `3.0.20`. It sends different stream parts to support data, tool calls, etc. What you see is the raw stream data protocol response.

## Solution

You have several options:

1. Use the AI Core [`streamText`](/docs/reference/ai-sdk-core/stream-text) function to send a raw text stream:

   ```tsx
   export async function POST(req: Request) {
     const { prompt } = await req.json();

     const result = streamText({
       model: openai.completion('gpt-3.5-turbo-instruct'),
       maxTokens: 2000,
       prompt,
     });

     return result.toTextStreamResponse();
   }
   ```

2. Pin the AI SDK version to `3.0.19` . This will keep the raw text stream.
3. Process the stream data stream using `readDataStream`.

   ```tsx
   for await (const { type, value } of readDataStream(reader, {
     isAborted: () => abortControllerRef?.current === null,
   })) {
     if (type === 'text') {
       doSomething(value);
     }
   }
   ```

---
title: Streamable UI Errors
description: Troubleshooting errors related to streamable UI.
---

# Streamable UI Component Error

## Issue

- Variable Not Found
- Cannot find `div`
- `Component` refers to a value, but is being used as a type

## Solution

If you encounter these errors when working with streamable UIs within server actions, it is likely because the file ends in `.ts` instead of `.tsx`.

---
title: Streaming Not Working When Deploying on Vercel (Next.js Pages Router)
description: Troubleshooting streaming issues when deploying to Vercel with the Next.js Pages Router.
---

# Streaming Not Working When Deploying on Vercel (Next.js Pages Router)

## Issue

I'm using the Next.js Pages Router. Streaming with the AI SDK works in my local development environment.
However, when deploying to Vercel, streaming does not work in the deployed app.
Instead of streaming, only the full response is returned after a while.

## Cause

The Next.js Pages Router currently does not support streaming with its own routes.

## Solution

With Next.js 13+, you can mix and match App Router and Pages Router routes in the same project.
You need to use App Router routes for streaming with the AI SDK.

Example App Router route:

```tsx filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { StreamingTextResponse, streamText } from 'ai';

// Force the route to be dynamic and allow streaming responses up to 30 seconds
export const dynamic = 'force-dynamic';
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
  });

  return result.toDataStreamResponse();
}
```

---
title: Streaming Not Working When Deploying on Vercel (Next.js App Router)
description: Troubleshooting streaming issues when deploying to Vercel with the Next.js App Router.
---

# Streaming Not Working When Deploying on Vercel (Next.js App Router)

## Issue

I'm using the Next.js App Router. Streaming with the AI SDK works in my local development environment.
However, when deploying to Vercel, streaming does not work in the deployed app.
Instead of streaming, only the full response is returned after a while.

## Cause

The route was inferred as a static route and has been compiled as such.

## Solution

You need to explicitly enforce dynamic behavior for the route.

Add the following to your route file:

```tsx
export const dynamic = 'force-dynamic';
```

This will enforce dynamic behavior, which is required for streaming.

> **Note:** When deploying on Vercel you may also need the following workaround:

```tsx
import { unstable_noStore as noStore } from 'next/cache';

export default async function Component() {
  noStore();
  const result = await generateText({...})
  ...
}
```

---
title: Getting Timeouts When Deploying on Vercel
description: Learn how to fix timeouts and cut off responses when deploying to Vercel.
---

# Getting Timeouts When Deploying on Vercel

## Issue

Streaming with the AI SDK works in my local development environment.
However, when I'm deploying to Vercel, longer responses get chopped off in the UI and I'm seeing timeouts in the Vercel logs or I'm seeing the error: `Uncaught (in promise) Error: Connection closed`.

## Solution

If you are using Next.js with the App Router, you can add the following to your route file or the page you are calling your Server Action from:

```tsx
export const maxDuration = 30;
```

This increases the maximum duration of the function to 30 seconds.

For other frameworks such as Svelte, you can set timeouts in your `vercel.json` file:

```json
{
  "functions": {
    "api/chat/route.ts": {
      "maxDuration": 30
    }
  }
}
```

## Learn more

- [Configuring Maximum Duration for Vercel Functions](https://vercel.com/docs/functions/configuring-functions/duration)
- [Maximum Duration Limits](https://vercel.com/docs/functions/runtimes#max-duration)

---
title: Unclosed Streams
description: Troubleshooting errors related to unclosed streams.
---

# Unclosed Streams

Sometimes streams are not closed properly, which can lead to unexpected behavior. The following are some common issues that can occur when streams are not closed properly.

## Issue

The streamable UI has been slow to update.

## Solution

This happens when you create a streamable UI using [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) and fail to close the stream.
In order to fix this, you must ensure you close the stream by calling the [`.done()`](/docs/reference/ai-sdk-rsc/create-streamable-ui#done) method.
This will ensure the stream is closed.

```tsx file='app/actions.tsx'
import { createStreamableUI } from 'ai/rsc';

const submitMessage = async () => {
  'use server';

  const stream = createStreamableUI('1');

  stream.update('2');
  stream.append('3');
  stream.done('4'); // [!code ++]

  return stream.value;
};
```

---
title: useChat Failed to Parse Stream
description: Troubleshooting errors related to the Use Chat Failed to Parse Stream error.
---

# `useChat` "Failed to Parse Stream String" Error

## Issue

I am using [`useChat`](/docs/reference/ai-sdk-ui/use-chat) or [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion), and I am getting a `"Failed to parse stream string. Invalid code"` error. I am using version `3.0.20` or newer of the AI SDK.

## Background

The AI SDK has switched to the stream data protocol in version `3.0.20`.
[`useChat`](/docs/reference/ai-sdk-ui/use-chat) and [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion) expect stream parts that support data, tool calls, etc.
What you see is a failure to parse the stream.
This can be caused by using an older version of the AI SDK in the backend, by providing a text stream using a custom provider, or by using a raw LangChain stream result.

## Solution

You can switch [`useChat`](/docs/reference/ai-sdk-ui/use-chat) and [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion) to raw text stream processing with the [`streamProtocol`](/docs/reference/ai-sdk-ui/use-completion#stream-protocol) parameter.
Set it to `text` as follows:

```tsx
const { messages, append } = useChat({ streamProtocol: 'text' });
```

---
title: Server Action Plain Objects Error
description: Troubleshooting errors related to using AI SDK Core functions with Server Actions.
---

# "Only plain objects can be passed from client components" Server Action Error

## Issue

I am using [`streamText`](/docs/reference/ai-sdk-core/stream-text) or [`streamObject`](/docs/reference/ai-sdk-core/stream-object) with Server Actions, and I am getting a `"only plain objects and a few built ins can be passed from client components"` error.

## Background

This error occurs when you're trying to return a non-serializable object from a Server Action to a Client Component. The streamText function likely returns an object with methods or complex structures that can't be directly serialized and passed to the client.

## Solution

To fix this issue, you need to ensure that you're only returning serializable data from your Server Action. Here's how you can modify your approach:

1. Instead of returning the entire result object from streamText, extract only the necessary serializable data.
2. Use the [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value) function to create a streamable value that can be safely passed to the client.

Here's an example that demonstrates how to implement this solution: [Streaming Text Generation](/examples/next-app/basics/streaming-text-generation).

This approach ensures that only serializable data (the text) is passed to the client, avoiding the "only plain objects" error.

---
title: useChat No Response with maxSteps
description: Troubleshooting errors related to the Use Chat Failed to Parse Stream error.
---

# `useChat` No Response with maxSteps

## Issue

I am using [`useChat`](/docs/reference/ai-sdk-ui/use-chat) with [`maxSteps`](/docs/reference/ai-sdk-ui/use-chat#max-steps).
When I log the incoming messages on the server, I can see the tool call and the tool result, but the model does not respond with anything.

## Background

The `useChat` hook uses a message structure (`Message`) that pre-dates the AI SDK Core message structure (`CoreMessage`).

## Solution

<Note title="warning">
  This solution is outdated. The AI SDK now automatically converts the incoming
  messages to the `CoreMessage` format.
</Note>

To resolve this issue, convert the incoming messages to the `CoreMessage` format using the [`convertToCoreMessages`](/docs/reference/ai-sdk-ui/convert-to-core-messages) function.

```tsx highlight="9"
import { openai } from '@ai-sdk/openai';
import { convertToCoreMessages, streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages: convertToCoreMessages(messages),
  });

  return result.toDataStreamResponse();
}
```

---
title: NaN token counts when using streamText with OpenAI models
description: Troubleshooting errors related to NaN token counts in OpenAI streaming.
---

# `NaN` token counts when using `streamText` with OpenAI models

## Issue

I am using `streamText` with the [OpenAI provider for the AI SDK](/providers/ai-sdk-providers/openai) and OpenAI models.
I use [`createOpenAI`](/providers/ai-sdk-providers/openai#provider-instance) to create the provider instance.
When I try to get the token counts, I get `NaN` values.

## Background

OpenAI introduced `streamOptions` parameters to enable token counts in the stream.
However, this was a breaking change for OpenAI-compatible providers,
and we therefore made it opt-in.

## Solution

When you use [`createOpenAI`](/providers/ai-sdk-providers/openai#provider-instance), you can enable a `strict` compatibility model:

```tsx
import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  compatibility: 'strict',
});
```

This will enable the token counts in the stream.
When you use the default `openai` provider instance, the setting is enabled by default.

---
title: Model is not assignable to type "LanguageModelV1"
description: Troubleshooting errors related to incompatible models.
---

# Model is not assignable to type "LanguageModelV1"

## Issue

I have updated the AI SDK and now I get the following error: `Type 'SomeModel' is not assignable to type 'LanguageModelV1'.`

<Note>Similar errors can occur with `EmbeddingModelV1` as well.</Note>

## Background

Sometimes new features are being added to the model specification.
This can cause incompatibilities with older provider versions.

## Solution

Update your provider packages and the AI SDK to the latest version.

---
title: TypeScript error "Cannot find namespace 'JSX'"
description: Troubleshooting errors related to TypeScript and JSX.
---

# TypeScript error "Cannot find namespace 'JSX'"

## Issue

I am using the AI SDK in a project without React, e.g. an Hono server, and I get the following error:
`error TS2503: Cannot find namespace 'JSX'.`

## Background

The AI SDK has a dependency on `@types/react` which defines the `JSX` namespace.
It will be removed in the next major version of the AI SDK.

## Solution

You can install the `@types/react` package as a dependency to fix the error.

```bash
npm install @types/react
```

---
title: React error "Maximum update depth exceeded"
description: Troubleshooting errors related to the "Maximum update depth exceeded" error.
---

# React error "Maximum update depth exceeded"

## Issue

I am using the AI SDK in a React project with the `useChat` or `useCompletion` hooks
and I get the following error when AI responses stream in: `Maximum update depth exceeded`.

## Background

By default, the UI is re-rendered on every chunk that arrives.
This can overload the rendering, especially on slower devices or when complex components
need updating (e.g. Markdown). Throttling can mitigate this.

## Solution

Use the `experimental_throttle` option to throttle the UI updates:

### `useChat`

```tsx filename="page.tsx" highlight="2-3"
const { messages, ... } = useChat({
  // Throttle the messages and data updates to 50ms:
  experimental_throttle: 50
})
```

### `useCompletion`

```tsx filename="page.tsx" highlight="2-3"
const { completion, ... } = useCompletion({
  // Throttle the completion and data updates to 50ms:
  experimental_throttle: 50
})
```

---
title: Troubleshooting
description: Troubleshooting information for common issues encountered with the AI SDK.
collapsed: true
---

# Troubleshooting

This section is designed to help you quickly identify and resolve common issues encountered with the AI SDK, ensuring a smoother and more efficient development experience.

<Support />

---
title: Generating Text
description: Learn to generate text using the AI SDK in your Next.js Pages Router application
---

# Generate Text

A situation may arise when you need to generate text based on a prompt. For example, you may want to generate a response to a question or summarize a body of text. The `generateText` function can be used to generate text based on the input prompt.

<Browser>
  <TextGeneration />
</Browser>

## Client

Let's create a simple React component that will make a POST request to the `/api/completion` endpoint when a button is clicked. The endpoint will generate text based on the input prompt.

```tsx filename="pages/index.tsx"
import { useState } from 'react';

export default function Page() {
  const [generation, setGeneration] = useState('');
  const [isLoading, setIsLoading] = useState(false);

  return (
    <div>
      <div
        onClick={async () => {
          setIsLoading(true);

          await fetch('/api/completion', {
            method: 'POST',
            body: JSON.stringify({
              prompt: 'Why is the sky blue?',
            }),
          }).then(response => {
            response.json().then(json => {
              setGeneration(json.text);
              setIsLoading(false);
            });
          });
        }}
      >
        Generate
      </div>

      {isLoading ? 'Loading...' : generation}
    </div>
  );
}
```

## Server

Let's create a route handler for `/api/completion` that will generate text based on the input prompt. The route will call the `generateText` function from the `ai` module, which will then generate text based on the input prompt and return it.

```typescript filename='app/api/completion/route.ts'
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const { text } = await generateText({
    model: openai('gpt-4'),
    system: 'You are a helpful assistant.',
    prompt,
  });

  return Response.json({ text });
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/basics/generate-text/index.tsx" />

---
title: Stream Text Generation
description: Learn to stream text generations using the AI SDK in your Next.js Pages Router application
---

# Stream Text Generation

Text generation can sometimes take a long time to complete, especially when you're generating a couple of paragraphs. In such cases, it is useful to stream the text generation process to the client in real-time. This allows the client to display the generated text as it is being generated, rather than have users wait for it to complete before displaying the result.

<Browser>
  <TextGeneration stream />
</Browser>

## Client

Let's create a simple React component that imports the `useCompletion` hook from the `ai/react` module. The `useCompletion` hook will call the `/api/completion` endpoint when a button is clicked. The endpoint will generate text based on the input prompt and stream it to the client.

```tsx filename="pages/index.tsx"
import { useCompletion } from 'ai/react';

export default function Page() {
  const { completion, complete } = useCompletion({
    api: '/api/completion',
  });

  return (
    <div>
      <div
        onClick={async () => {
          await complete('Why is the sky blue?');
        }}
      >
        Generate
      </div>

      {completion}
    </div>
  );
}
```

## Server

Let's create a route handler for `/api/completion` that will generate text based on the input prompt. The route will call the `streamText` function from the `ai` module, which will then generate text based on the input prompt and stream it to the client.

```typescript filename='app/api/completion/route.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = streamText({
    model: openai('gpt-4'),
    system: 'You are a helpful assistant.',
    prompt,
  });

  return result.toDataStreamResponse();
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/basics/stream-text/index.tsx" />

---
title: Generating Structured Data
description: Learn to generate structured data using the AI SDK in your Next.js Pages Router application
---

# Generate Object

Earlier functions like `generateText` and `streamText` gave us the ability to generate unstructured text. However, if you want to generate structured data like JSON, you can provide a schema that describes the structure of your desired object to the `generateObject` function.

The function requires you to provide a schema using [zod](https://zod.dev), a library for defining schemas for JavaScript objects. By using zod, you can also use it to validate the generated object and ensure that it conforms to the specified structure.

<Browser>
  <ObjectGeneration
    object={{
      notifications: [
        {
          name: 'Jamie Roberts',
          message: "Hey! How's the study grind going? Need a coffee boost?",
          minutesAgo: 15,
        },
        {
          name: 'Prof. Morgan',
          message:
            'Reminder: Your term paper is due promptly at 8 AM tomorrow. Please ensure it meets the submission guidelines outlined.',
          minutesAgo: 46,
        },
        {
          name: 'Alex Chen',
          message:
            "Dude, urgent! Borrow your notes for tomorrow's exam? I swear mine got eaten by my dog!",
          minutesAgo: 30,
        },
      ],
    }}
  />
</Browser>

## Client

Let's create a simple React component that will make a POST request to the `/api/completion` endpoint when a button is clicked. The endpoint will return the generated object based on the input prompt and we'll display it.

```tsx filename='pages/index.tsx'
import { useState } from 'react';

export default function Page() {
  const [generation, setGeneration] = useState();
  const [isLoading, setIsLoading] = useState(false);

  return (
    <div>
      <div
        onClick={async () => {
          setIsLoading(true);

          await fetch('/api/completion', {
            method: 'POST',
            body: JSON.stringify({
              prompt: 'Messages during finals week.',
            }),
          }).then(response => {
            response.json().then(json => {
              setGeneration(json.object);
              setIsLoading(false);
            });
          });
        }}
      >
        Generate
      </div>

      {isLoading ? 'Loading...' : <pre>{JSON.stringify(generation)}</pre>}
    </div>
  );
}
```

## Server

Let's create a route handler for `/api/completion` that will generate an object based on the input prompt. The route will call the `generateObject` function from the `ai` module, which will then generate an object based on the input prompt and return it.

```typescript filename='app/api/completion/route.ts'
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = await generateObject({
    model: openai('gpt-4'),
    system: 'You generate three notifications for a messages app.',
    prompt,
    schema: z.object({
      notifications: z.array(
        z.object({
          name: z.string().describe('Name of a fictional person.'),
          message: z.string().describe('Do not use emojis or links.'),
          minutesAgo: z.number(),
        }),
      ),
    }),
  });

  return result.toJsonResponse();
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/basics/generate-object/index.tsx" />

---
title: Streaming Object Generation
description: Learn to stream object generations using the AI SDK in your Next.js App Router application
---

# Stream Object Generation

Object generation can sometimes take a long time to complete, especially when you're generating a large schema.
In such cases, it is useful to stream the object generation process to the client in real-time.
This allows the client to display the generated object as it is being generated,
rather than have users wait for it to complete before displaying the result.

<Browser>
  <ObjectGeneration
    stream
    object={{
      notifications: [
        {
          name: 'Jamie Roberts',
          message: "Hey! How's the study grind going? Need a coffee boost?",
          minutesAgo: 15,
        },
        {
          name: 'Prof. Morgan',
          message:
            'Reminder: Your term paper is due promptly at 8 AM tomorrow. Please ensure it meets the submission guidelines outlined.',
          minutesAgo: 46,
        },
        {
          name: 'Alex Chen',
          message:
            "Dude, urgent! Borrow your notes for tomorrow's exam? I swear mine got eaten by my dog!",
          minutesAgo: 30,
        },
      ],
    }}
  />
</Browser>

## Schema

It is helpful to set up the schema in a separate file that is imported on both the client and server.

```ts filename='app/api/use-object/schema.ts'
import { z } from 'zod';

// define a schema for the notifications
export const notificationSchema = z.object({
  notifications: z.array(
    z.object({
      name: z.string().describe('Name of a fictional person.'),
      message: z.string().describe('Message. Do not use emojis or links.'),
    }),
  ),
});
```

## Client

The client uses [`useObject`](/docs/reference/ai-sdk-ui/use-object) to stream the object generation process.

The results are partial and are displayed as they are received.
Please note the code for handling `undefined` values in the JSX.

```tsx filename='app/page.tsx'
'use client';

import { experimental_useObject as useObject } from 'ai/react';
import { notificationSchema } from './api/use-object/schema';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/use-object',
    schema: notificationSchema,
  });

  return (
    <div>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

## Server

On the server, we use [`streamObject`](/docs/reference/ai-sdk-core/stream-object) to stream the object generation process.

```typescript filename='app/api/use-object/route.ts'
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { notificationSchema } from './schema';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamObject({
    model: openai('gpt-4-turbo'),
    schema: notificationSchema,
    prompt:
      `Generate 3 notifications for a messages app in this context:` + context,
  });

  return result.toTextStreamResponse();
}
```

## Loading State and Stopping the Stream

You can use the `loading` state to display a loading indicator while the object is being generated.
You can also use the `stop` function to stop the object generation process.

```tsx filename='app/page.tsx' highlight="7,16,21,24"
'use client';

import { experimental_useObject as useObject } from 'ai/react';
import { notificationSchema } from './api/use-object/schema';

export default function Page() {
  const { object, submit, isLoading, stop } = useObject({
    api: '/api/use-object',
    schema: notificationSchema,
  });

  return (
    <div>
      <button
        onClick={() => submit('Messages during finals week.')}
        disabled={isLoading}
      >
        Generate notifications
      </button>

      {isLoading && (
        <div>
          <div>Loading...</div>
          <button type="button" onClick={() => stop()}>
            Stop
          </button>
        </div>
      )}

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

---
title: Basics
description: Learn the basics of using the AI SDK in your Next.js Pages Router application.
---

# Basics

One of the most basic things you can do with language models is to generate text. In this section, you will learn to generate text, and also stream it to the client.

Beyond text, you will also learn to generate structured data by providing a schema of your choice, and also stream it to the client.

<IndexCards
  cards={[
    {
      title: 'Generate Text',
      description: 'Learn how to generate text.',
      href: '/examples/next-pages/basics/generating-text',
    },
    {
      title: 'Stream Text Generation',
      description: 'Learn how to stream a text generation.',
      href: '/examples/next-pages/basics/streaming-text-generation',
    },
    {
      title: 'Generate Object',
      description: 'Learn how to generate structured data.',
      href: '/examples/next-pages/basics/generating-object',
    },
    {
      title: 'Stream Object Generation',
      description: 'Learn how to stream a structured data generation.',
      href: '/examples/next-pages/basics/streaming-object-generation',
    },
  ]}
/>

---
title: Generation Chat Completion
description: Learn to generate chat completions using the AI SDK in your Next.js Pages Router application
---

# Generate Chat Completion

Previously, you were able to generate text and objects using either a single message prompt, a system prompt, or a combination of both of them. However, there may be times when you want to generate text based on a series of messages.

A chat completion allows you to generate text based on a series of messages. This series of messages can be any series of interactions between any number of systems, but the most popular and relatable use case has been a series of messages that represent a conversation between a user and a model.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{ role: 'User', content: 'Why is the sky blue?' }}
    outputMessage={{
      role: 'Assistant',
      content: 'The sky is blue because of rayleigh scattering.',
    }}
  />
</Browser>

## Client

Let's start by creating a simple chat interface with an input field that sends the user's message and displays the conversation history. You will call the `/api/chat` endpoint to generate the assistant's response.

```tsx filename='pages/index.tsx'
import { CoreMessage } from 'ai';
import { useState } from 'react';

export default function Page() {
  const [input, setInput] = useState('');
  const [messages, setMessages] = useState<CoreMessage[]>([]);

  return (
    <div>
      <input
        value={input}
        onChange={event => {
          setInput(event.target.value);
        }}
        onKeyDown={async event => {
          if (event.key === 'Enter') {
            setMessages(currentMessages => [
              ...currentMessages,
              { role: 'user', content: input },
            ]);

            const response = await fetch('/api/chat', {
              method: 'POST',
              body: JSON.stringify({
                messages: [...messages, { role: 'user', content: input }],
              }),
            });

            const { messages: newMessages } = await response.json();

            setMessages(currentMessages => [
              ...currentMessages,
              ...newMessages,
            ]);
          }
        }}
      />

      {messages.map((message, index) => (
        <div key={`${message.role}-${index}`}>
          {typeof message.content === 'string'
            ? message.content
            : message.content
                .filter(part => part.type === 'text')
                .map((part, partIndex) => (
                  <div key={partIndex}>{part.text}</div>
                ))}
        </div>
      ))}
    </div>
  );
}
```

## Server

Next, let's create the `/api/chat` endpoint that generates the assistant's response based on the conversation history.

```typescript filename='app/api/chat/route.ts'
import { CoreMessage, generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { messages }: { messages: CoreMessage[] } = await req.json();

  const { responseMessages } = await generateText({
    model: openai('gpt-4'),
    system: 'You are a helpful assistant.',
    messages,
  });

  return Response.json({ messages: responseMessages });
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/chat/generate-chat/index.tsx" />

---
title: Stream Chat Completion
description: Learn to stream chat completions using the AI SDK in your Next.js Pages Router application
---

# Stream Chat Completion

Chat completion can sometimes take a long time to finish, especially when the response is big. In such cases, it is useful to stream the chat completion to the client in real-time. This allows the client to display the new message as it is being generated by the model, rather than have users wait for it to finish.

<Browser>
  <ChatGeneration
    stream
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{ role: 'User', content: 'Why is the sky blue?' }}
    outputMessage={{
      role: 'Assistant',
      content: 'The sky is blue because of rayleigh scattering.',
    }}
  />
</Browser>

## Client

Let's create a React component that imports the `useChat` hook from the `ai/react` module. The `useChat` hook will call the `/api/chat` endpoint when the user sends a message. The endpoint will generate the assistant's response based on the conversation history and stream it to the client.

```tsx filename='pages/index.tsx'
import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, setInput, append } = useChat();

  return (
    <div>
      <input
        value={input}
        onChange={event => {
          setInput(event.target.value);
        }}
        onKeyDown={async event => {
          if (event.key === 'Enter') {
            append({ content: input, role: 'user' });
          }
        }}
      />

      {messages.map((message, index) => (
        <div key={index}>{message.content}</div>
      ))}
    </div>
  );
}
```

## Server

Next, let's create the `/api/chat` endpoint that generates the assistant's response based on the conversation history.

```typescript filename='app/api/chat/route.ts'
import { CoreMessage, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { messages }: { messages: CoreMessage[] } = await req.json();

  const result = streamText({
    model: openai('gpt-4'),
    system: 'You are a helpful assistant.',
    messages,
  });

  return result.toDataStreamResponse();
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/chat/stream-chat/index.tsx" />

---
title: Add Image Inputs
description: Learn to add image inputs to chat completions using the AI SDK in your Next.js Pages Router application
---

# Add Image Inputs

Vision models such as GPT-4 can process both text and images. In this example, we will show you how to send an image URL along with the user's message to the model.

## Using Image URLs

### Server

We split the user's message into two parts: the text and the image URL. We then send both parts to the model.
The last message is the user's message, and we add the image URL to it.

```tsx filename='app/api/chat/route.ts' highlight="8,9,23"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export const maxDuration = 60;

export async function POST(req: Request) {
  // 'data' contains the additional data that you have sent:
  const { messages, data } = await req.json();

  const initialMessages = messages.slice(0, -1);
  const currentMessage = messages[messages.length - 1];

  // Call the language model
  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages: [
      ...initialMessages,
      {
        role: 'user',
        content: [
          { type: 'text', text: currentMessage.content },
          { type: 'image', image: new URL(data.imageUrl) },
        ],
      },
    ],
  });

  // Respond with the stream
  return result.toDataStreamResponse();
}
```

### Client

On the client we can send the image URL along with the user's message by adding the `data` object to the `handleSubmit` function.
You can replace the `imageUrl` with the actual URL of the image you want to send.

```typescript filename='app/page.tsx' highlight="18-20"
'use client';

import { useChat } from 'ai/react';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
        </div>
      ))}

      <form
        onSubmit={e => {
          handleSubmit(e, {
            data: { imageUrl: 'https://somewhere.com/image.png' },
          });
        }}
      >
        <input
          value={input}
          placeholder="What does the image show..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

---
title: Custom body content for useChat
description: Learn how to control the body content that useChat sends to the server
---

# useChat: custom body content

<Note type="warning">
  `experimental_prepareRequestBody` is an experimental feature and only
  available in React.
</Note>

By default, `useChat` sends all messages as well as information from the request to the server.
However, it is often desirable to control the body content that is sent to the server, e.g. to:

- only send the last message
- send additional data along with the message
- change the structure of the request body

The `experimental_prepareRequestBody` option allows you to customize the body content that is sent to the server.
The function receives the message list, the request data, and the request body from the append call.
It should return the body content that will be sent to the server.

## Example

This example shows how to only send the text of the last message to the server.
This can be useful if you want to reduce the amount of data sent to the server.

### Client

```typescript filename='app/page.tsx' highlight="7-10"
'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    experimental_prepareRequestBody: ({ messages }) => {
      // e.g. only the text of the last message:
      return messages[messages.length - 1].content;
    },
  });

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

### Server

We need to adjust the server to only receive the text of the last message.
The rest of the message history can be loaded from storage.

```tsx filename='app/api/chat/route.ts' highlight="8,9,23"
import { openai } from '@ai-sdk/openai'
import { streamText } from 'ai'

// Allow streaming responses up to 30 seconds
export const maxDuration = 30

export async function POST(req: Request) {
  // we receive only the text from the last message
  const text = await req.json()

  // e.g. load message history from storage
  const history = await loadHistory()

  // Call the language model
  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages: [...history, { role: 'user', content: text }]
    onFinish({ text }) {
      // e.g. save the message and the response to storage
    }
  })

  // Respond with the stream
  return result.toDataStreamResponse()
}
```

---
title: Chat
description: Learn how to generate chat completions using the AI SDK in your Next.js Pages Router application
---

# Chat

So far you've learned how to generate text and structured data using single prompts. In this section, you will learn to use `messages` to add a sequence of messages to the language model and generate the response based on the context of the conversation – called chat completion.

<IndexCards
  cards={[
    {
      title: 'Generate Chat Completions',
      description: 'Learn how to generate a chat completion.',
      href: '/examples/next-pages/chat/generate-chat-completion',
    },
    {
      title: 'Stream Chat Completions',
      description: 'Learn how to stream a chat completion.',
      href: '/examples/next-pages/chat/stream-chat-completion',
    },
    {
      title: 'Add Image Inputs',
      description: 'Learn how to add image inputs to a chat completion.',
      href: '/examples/next-pages/chat/use-chat-image-input',
    },
    {
      title: 'Custom body content for useChat',
      description:
        'Learn how to control the body content that useChat sends to the server.',
      href: '/examples/next-pages/chat/use-chat-custom-body',
    },
  ]}
/>

---
title: Call Tools
description: Learn to call tools using the AI SDK in your Next.js Pages Router application
---

# Call Tools

Some models allow developers to provide a list of tools that can be called at any time during a generation. This is useful for extending the capabilites of a language model to either use logic or data to interact with systems external to the model.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is the weather in Paris and New York?',
    }}
    outputMessage={{
      role: 'Assistant',
      content:
        'The weather is 24°C in New York and 25°C in Paris. It is sunny in both cities.',
    }}
  />
</Browser>

## Client

Let's create a React component that imports the `useChat` hook from the `ai/react` module. The `useChat` hook will call the `/api/chat` endpoint when the user sends a message. The endpoint will generate the assistant's response based on the conversation history and stream it to the client. If the assistant responds with a tool call, the hook will automatically display them as well.

We will use the `maxSteps` to specify the maximum number of steps (i.e., LLM calls) that can be made to prevent infinite loops. In this example, you will set it to `2` to allow for two backend calls to happen.

```tsx filename='pages/index.tsx'
'use client';

import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, setInput, append } = useChat({
    api: '/api/chat',
    maxSteps: 2,
  });

  return (
    <div>
      <input
        value={input}
        onChange={event => {
          setInput(event.target.value);
        }}
        onKeyDown={async event => {
          if (event.key === 'Enter') {
            append({ content: input, role: 'user' });
          }
        }}
      />

      {messages.map((message, index) => (
        <div key={index}>{message.content}</div>
      ))}
    </div>
  );
}
```

## Server

You will create a new route at `/api/chat` that will use the `streamText` function from the `ai` module to generate the assistant's response based on the conversation history.

You will use the [`tools`](/docs/reference/ai-sdk-core/generate-text#tools) parameter to specify a tool called `celsiusToFahrenheit` that will convert a user given value in celsius to fahrenheit.

You will also use zod to specify the schema for the `celsiusToFahrenheit` function's parameters.

```tsx filename='app/api/chat/route.ts'
import { ToolInvocation, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

interface Message {
  role: 'user' | 'assistant';
  content: string;
  toolInvocations?: ToolInvocation[];
}

export async function POST(req: Request) {
  const { messages }: { messages: Message[] } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'You are a helpful assistant.',
    messages,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        parameters: z.object({
          city: z.string().describe('The city to get the weather for'),
          unit: z
            .enum(['C', 'F'])
            .describe('The unit to display the temperature in'),
        }),
        execute: async ({ city, unit }) => {
          const weather = {
            value: 24,
            description: 'Sunny',
          };

          return `It is currently ${weather.value}°${unit} and ${weather.description} in ${city}!`;
        },
      },
    },
  });

  return result.toDataStreamResponse();
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/tools/call-tool/index.tsx" />

---
title: Call Tools in Parallel
description: Learn to call tools in parallel using the AI SDK in your Next.js Pages Router application
---

# Call Tools in Parallel

Some language models support calling tools in parallel. This is particularly useful when multiple tools are independent of each other and can be executed in parallel during the same generation step.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is the weather in Paris and New York?',
    }}
    outputMessage={{
      role: 'Assistant',
      content:
        'The weather is 24°C in New York and 25°C in Paris. It is sunny in both cities.',
    }}
  />
</Browser>

## Client

Let's create a React component that imports the `useChat` hook from the `ai/react` module. The `useChat` hook will call the `/api/chat` endpoint when the user sends a message. The endpoint will generate the assistant's response based on the conversation history and stream it to the client. If the assistant responds with a tool call, the hook will automatically display them as well.

You will use the `maxSteps` to specify the maximum number of steps that can made before the model or the user responds with a text message. In this example, you will set it to `2` to allow for another call with the tool result to happen.

```tsx filename='pages/index.tsx'
import { useChat } from 'ai/react';

export default function Page() {
  const { messages, input, setInput, append } = useChat({
    api: '/api/chat',
    maxSteps: 2,
  });

  return (
    <div>
      <input
        value={input}
        onChange={event => {
          setInput(event.target.value);
        }}
        onKeyDown={async event => {
          if (event.key === 'Enter') {
            append({ content: input, role: 'user' });
          }
        }}
      />

      {messages.map((message, index) => (
        <div key={index}>{message.content}</div>
      ))}
    </div>
  );
}
```

## Server

You will create a new route at `/api/chat` that will use the `streamText` function from the `ai` module to generate the assistant's response based on the conversation history.

You will use the [`tools`](/docs/reference/ai-sdk-core/generate-text#tools) parameter to specify a tool called `celsiusToFahrenheit` that will convert a user given value in celsius to fahrenheit.

You will add the `getWeather` function and use zod to specify the schema for its parameters.

```ts filename='app/api/chat/route.ts'
import { ToolInvocation, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

interface Message {
  role: 'user' | 'assistant';
  content: string;
  toolInvocations?: ToolInvocation[];
}

function getWeather({ city, unit }) {
  return { value: 25, description: 'Sunny' };
}

export async function POST(req: Request) {
  const { messages }: { messages: Message[] } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'You are a helpful assistant.',
    messages,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        parameters: z.object({
          city: z.string().describe('The city to get the weather for'),
          unit: z
            .enum(['C', 'F'])
            .describe('The unit to display the temperature in'),
        }),
        execute: async ({ city, unit }) => {
          const { value, description } = getWeather({ city, unit });
          return `It is currently ${value}°${unit} and ${description} in ${city}!`;
        },
      },
    },
  });

  return result.toDataStreamResponse();
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/tools/call-tools-in-parallel/index.tsx" />

---
title: Render Interface during Tool Call
description: Learn to render a React component during a tool call using the AI SDK in your Next.js Pages Router application
---

# Render Interface During Tool Call

An interesting consequence of language models that can call [tools](/docs/ai-sdk-core/tools-and-tool-calling) is that this ability can be used to render visual interfaces by streaming React components to the client.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is the weather in San Francisco?',
    }}
    outputMessage={{
      role: 'Assistant',
      content: 'The weather is 24°C and sunny in San Francisco.',
      display: (
        <div className="py-4">
          <WeatherCard
            content={{
              weather: {
                temperature: 24,
                condition: 'Sunny',
              },
            }}
          />
        </div>
      ),
    }}
  />
</Browser>

## Client

Let's build an assistant that gets the weather for any city by calling the `getWeatherInformation` tool. Instead of returning text during the tool call, you will render a React component that displays the weather information on the client.

```tsx filename='pages/index.tsx'
import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, addToolResult } =
    useChat({
      api: '/api/use-chat',
      maxSteps: 5,

      // run client-side tools that are automatically executed:
      async onToolCall({ toolCall }) {
        if (toolCall.toolName === 'getLocation') {
          const cities = [
            'New York',
            'Los Angeles',
            'Chicago',
            'San Francisco',
          ];
          return cities[Math.floor(Math.random() * cities.length)];
        }
      },
    });

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch gap-4">
      {messages?.map((m: Message) => (
        <div key={m.id} className="whitespace-pre-wrap flex flex-col gap-1">
          <strong>{`${m.role}: `}</strong>
          {m.content}
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            const toolCallId = toolInvocation.toolCallId;

            // render confirmation tool (client-side tool with user interaction)
            if (toolInvocation.toolName === 'askForConfirmation') {
              return (
                <div
                  key={toolCallId}
                  className="text-gray-500 flex flex-col gap-2"
                >
                  {toolInvocation.args.message}
                  <div className="flex gap-2">
                    {'result' in toolInvocation ? (
                      <b>{toolInvocation.result}</b>
                    ) : (
                      <>
                        <button
                          className="px-4 py-2 font-bold text-white bg-blue-500 rounded hover:bg-blue-700"
                          onClick={() =>
                            addToolResult({
                              toolCallId,
                              result: 'Yes, confirmed.',
                            })
                          }
                        >
                          Yes
                        </button>
                        <button
                          className="px-4 py-2 font-bold text-white bg-red-500 rounded hover:bg-red-700"
                          onClick={() =>
                            addToolResult({
                              toolCallId,
                              result: 'No, denied',
                            })
                          }
                        >
                          No
                        </button>
                      </>
                    )}
                  </div>
                </div>
              );
            }

            // other tools:
            return 'result' in toolInvocation ? (
              toolInvocation.toolName === 'getWeatherInformation' ? (
                <div
                  key={toolCallId}
                  className="flex flex-col gap-2 p-4 bg-blue-400 rounded-lg"
                >
                  <div className="flex flex-row justify-between items-center">
                    <div className="text-4xl text-blue-50 font-medium">
                      {toolInvocation.result.value}°
                      {toolInvocation.result.unit === 'celsius' ? 'C' : 'F'}
                    </div>

                    <div className="h-9 w-9 bg-amber-400 rounded-full flex-shrink-0" />
                  </div>
                  <div className="flex flex-row gap-2 text-blue-50 justify-between">
                    {toolInvocation.result.weeklyForecast.map(
                      (forecast: any) => (
                        <div
                          key={forecast.day}
                          className="flex flex-col items-center"
                        >
                          <div className="text-xs">{forecast.day}</div>
                          <div>{forecast.value}°</div>
                        </div>
                      ),
                    )}
                  </div>
                </div>
              ) : toolInvocation.toolName === 'getLocation' ? (
                <div
                  key={toolCallId}
                  className="text-gray-500 bg-gray-100 rounded-lg p-4"
                >
                  User is in {toolInvocation.result}.
                </div>
              ) : (
                <div key={toolCallId} className="text-gray-500">
                  Tool call {`${toolInvocation.toolName}: `}
                  {toolInvocation.result}
                </div>
              )
            ) : (
              <div key={toolCallId} className="text-gray-500">
                Calling {toolInvocation.toolName}...
              </div>
            );
          })}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}
```

## Server

```tsx filename='api/chat.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';

export default async function POST(request: Request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: {
      // server-side tool with execute function:
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        parameters: z.object({ city: z.string() }),
        execute: async ({}: { city: string }) => {
          return {
            value: 24,
            unit: 'celsius',
            weeklyForecast: [
              { day: 'Monday', value: 24 },
              { day: 'Tuesday', value: 25 },
              { day: 'Wednesday', value: 26 },
              { day: 'Thursday', value: 27 },
              { day: 'Friday', value: 28 },
              { day: 'Saturday', value: 29 },
              { day: 'Sunday', value: 30 },
            ],
          };
        },
      },
      // client-side tool that starts user interaction:
      askForConfirmation: {
        description: 'Ask the user for confirmation.',
        parameters: z.object({
          message: z.string().describe('The message to ask for confirmation.'),
        }),
      },
      // client-side tool that is automatically executed on the client:
      getLocation: {
        description:
          'Get the user location. Always ask for confirmation before using this tool.',
        parameters: z.object({}),
      },
    },
  });

  return result.toDataStreamResponse();
}
```

---
title: Tools
description: Learn to allow language models use tools using the AI SDK in your Next.js Pages Router application
---

# Tools

Certain language models have the ability to use external tools to perform tasks, like using a calculator to solve a math problem or using a browser to search for information. The most common way to share tool information with language models is to share a function definition, along with its description, for it to execute and generate a response based on the output.

In this section, you will learn how to use the `tools` parameter to allow language models to call these functions in your Next.js application.

<IndexCards
  cards={[
    {
      title: 'Call Tools',
      href: '/examples/next-pages/tools/call-tool',
      description: 'Learn how to allow language models to use external tools.',
    },
    {
      title: 'Call Tools in Parallel',
      href: '/examples/next-pages/tools/call-tools-in-parallel',
      description:
        'Learn how to allow language models to use multiple tools in parallel.',
    },
    {
      title: 'Render Interface during Tool Call',
      href: '/examples/next-pages/tools/render-interface-during-tool-call',
      description:
        'Learn how to render an interface while a tool is being called.',
    },
  ]}
/>

---
title: Save Messages
description: Learn to save messages in your Next.js application using the AI SDK
---

# Save Messages

You can save the chat to an external database using the `onFinish` callback handler present in the `streamText` function.

```ts filename="@/app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { saveChat } from '@/utils/queries';
import { streamText, convertToCoreMessages } from 'ai';

export function POST(request) {
  const { id, messages } = await request.json();

  const coreMessages = convertToCoreMessages(messages);

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages: coreMessages,
    onFinish: async ({ responseMessages }) => {
      try {
        await saveChat({
          id,
          messages: [...coreMessages, ...responseMessages],
        });
      } catch (error) {
        console.error('Failed to save chat');
      }
    },
  });

  return result.toDataStream();
}
```

The `saveChat` function is a server action that saves the chat to the database. It first checks if the chat already exists in the database and updates the chat if it does. If the chat does not exist, it inserts a new chat into the database.

<Note>
This example assumes that the database table schema has a `messages` column that stores the chat messages as a JSON string of type `Array<CoreMessage>`. You can modify the `saveChat` function to suit your database schema.
</Note>

```ts filename="@/app/utils/queries.ts"
export async function saveChat({
  id,
  messages,
  userId,
}: {
  id: string;
  messages: any;
  userId: string;
}) {
  try {
    const selectedChats = await db.select().from(chat).where(eq(chat.id, id));

    if (selectedChats.length > 0) {
      return await db
        .update(chat)
        .set({
          messages: JSON.stringify(messages),
        })
        .where(eq(chat.id, id));
    }

    return await db.insert(chat).values({
      id,
      createdAt: new Date(),
      messages: JSON.stringify(messages),
      userId,
    });
  } catch (error) {
    console.error('Failed to save chat in database');
    throw error;
  }
}
```

---
title: Restore Messages
description: Learn to restore chat in your Next.js application using the AI SDK
---

# Restore Messages

The `useChat` hook accepts `initialMessages` to load an earlier chat history of messages.

As messages are typically loaded from an external database, we can use server actions inside the Page component to fetch an already existing chat from the database during static generation and pass the messages as props to the `<Chat/>` component.

<Note>
This example assumes that the database table schema has a `messages` column that stores the chat messages as a JSON string of type `Array<CoreMessage>`. You can modify the `convertToUIMessages` function to suit your database schema, but here's an [example implementation](https://github.com/vercel/ai-chatbot/blob/00b125378c998d19ef60b73fe576df0fe5a0e9d4/lib/utils.ts).
</Note>

```tsx filename="@/app/chat/[id]/page.tsx"
import { Chat } from '@/app/components/chat';
import { getChatById } from '@/utils/queries';
import { convertToUIMessages } from '@/utils/functions';

export default async function Page({ params }: { params: any }) {
  const { id } = params;
  const chatFromDb = await getChatById({ id });

  const chat: Chat = {
    ...chatFromDb,
    messages: convertToUIMessages(chatFromDb.messages),
  };

  return <Chat key={id} id={chat.id} initialMessages={chat.messages} />;
}
```

In the above code snippet, we fetch the chat from the database using the `getChatById` server action and convert the messages to the format expected by `useChat` component using the `convertToUIMessages` function.

```tsx filename="@/app/components/chat.tsx"
'use client';

import { Message } from 'ai';
import { useChat } from 'ai/react';

export function Chat({
  id,
  initialMessages,
}: {
  id;
  initialMessages: Array<Message>;
}) {
  const { messages } = useChat({
    id,
    initialMessages,
  });

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>
        </div>
      ))}
    </div>
  );
}
```

In the `Chat` component, we pass the `initialMessages` prop to the `useChat` hook to load the chat history. The `useChat` hook will then return the messages to be rendered in the chat window.

---
title: State Management
description: Learn to manage state in your Next.js application using the AI SDK
---

# State Management

In this section you will learn to manage state in your Next.js application using the AI SDK.

---
title: Route Components
description: Learn to route components using the AI SDK in your Next.js App Router application
---

# Route Components

Language models can call external tools to enhance responses beyond plain text. For weather queries, a React weather component could replace a text-only answer. This approach enables richer, interactive outputs. By combining AI language understanding with custom UI elements, we can create more useful and engaging interfaces. The key is thoughtfully matching user needs to appropriate visual or interactive components.

Let's create a weather tool that fetches weather data for a given location and returns the result as props for our React component.

```ts filename="@/app/api/chat/route.ts"
import { z } from 'zod';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import { streamText } from 'ai';

export function POST(request) {
  const { messages } = await request.json();

  const result = streamText({
    model: openai('gpt-4o'),
    system: 'you are a friendly assistant!',
    messages,
    tools: {
      displayWeather: {
        description: 'Display the weather for a location',
        parameters: z.object({
          latitude: z.number(),
          longitude: z.number(),
        }),
        execute: async function ({ latitude, longitude }) {
          const props = await getWeather({ latitude, longitude });
          return props;
        },
      },
    },
  });

  return result.toDataStream();
}
```

Now we can render the React component that uses the weather data returned by the `displayWeather` tool.

```tsx filename="@/components/chat.tsx"
'use client';

import { useChat } from 'ai/react';
import { Weather } from '@/components/weather';

export default function Chat() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>

          <div>
            {message.toolInvocations.map(toolInvocation => {
              const { toolName, toolCallId, state } = toolInvocation;

              if (state === 'result') {
                const { result } = toolInvocation;

                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <Weather weatherAtLocation={result} />
                    ) : null}
                  </div>
                );
              } else {
                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <div>Loading weather...</div>
                    ) : null}
                  </div>
                );
              }
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

The `Weather` component can be a simple React component that displays the weather data.

```tsx filename="@/components/weather.tsx"
import React from 'react';

export function Weather({ weatherAtLocation }) {
  const { value, unit } = weatherAtLocation;
  return (
    <div>
      {value}°{unit}
    </div>
  );
}
```

---
title: Generative User Interfaces
description: Learn to build generative user interfaces using the AI SDK in your Next.js application
---

# Generative User Interfaces

In this section you will learn to use the tool calling ability of language models to integrate more than just text responses to your application.

---
title: Stream Assistant Responses
description: Learn to stream assistant responses using the AI SDK in your Next.js Pages Router application
---

# Stream Assistant Responses

## Client

Let's create a simple chat interface that allows users to send messages to the assistant and receive responses. You will integrate the `useAssistant` hook from `ai/react` to stream the messages and status.

```tsx filename='pages/index.tsx'
import { Message, useAssistant } from 'ai/react';

export default function Page() {
  const { status, messages, input, submitMessage, handleInputChange } =
    useAssistant({ api: '/api/assistant' });

  return (
    <div className="flex flex-col gap-2">
      <div className="p-2">status: {status}</div>

      <div className="flex flex-col p-2 gap-2">
        {messages.map((message: Message) => (
          <div key={message.id} className="flex flex-row gap-2">
            <div className="w-24 text-zinc-500">{`${message.role}: `}</div>
            <div className="w-full">{message.content}</div>
          </div>
        ))}
      </div>

      <form onSubmit={submitMessage} className="fixed bottom-0 p-2 w-full">
        <input
          disabled={status !== 'awaiting_message'}
          value={input}
          onChange={handleInputChange}
          className="bg-zinc-100 w-full p-2"
        />
      </form>
    </div>
  );
}
```

## Server

Next, you will create an API route for `api/assistant` to handle the assistant's messages and responses. You will use the `AssistantResponse` function from `ai` to stream the assistant's responses back to the `useAssistant` hook on the client.

```tsx filename='app/api/assistant/route.ts'
import OpenAI from 'openai';
import { AssistantResponse } from 'ai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || '',
});

export async function POST(req: Request) {
  const input: {
    threadId: string | null;
    message: string;
  } = await req.json();

  const threadId = input.threadId ?? (await openai.beta.threads.create({})).id;

  const createdMessage = await openai.beta.threads.messages.create(threadId, {
    role: 'user',
    content: input.message,
  });

  return AssistantResponse(
    { threadId, messageId: createdMessage.id },
    async ({ forwardStream }) => {
      const runStream = openai.beta.threads.runs.stream(threadId, {
        assistant_id:
          process.env.ASSISTANT_ID ??
          (() => {
            throw new Error('ASSISTANT_ID environment is not set');
          })(),
      });

      await forwardStream(runStream);
    },
  );
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/assistants/stream-assistant-response/index.tsx" />

---
title: Stream Assistant Responses with Tools
description: Learn to stream assistant responses with tools using the AI SDK in your Next.js Pages Router application
---

# Stream Assistant Responses with Tools

Let's create a simple chat interface that allows users to send messages to the assistant and receive responses and give it the ability to use tools. You will integrate the `useAssistant` hook from `ai/react` to stream the messages and status.

You will need to provide the list of tools on the OpenAI [Assistant Dashboard](https://platform.openai.com/assistants). You can use the following schema to create a tool to convert celsius to fahrenheit.

```json
{
  "name": "celsiusToFahrenheit",
  "description": "convert celsius to fahrenheit.",
  "parameters": {
    "type": "object",
    "properties": {
      "value": {
        "type": "number",
        "description": "the value in celsius."
      }
    },
    "required": ["value"]
  }
}
```

## Client

Let's create a simple chat interface that allows users to send messages to the assistant and receive responses. You will integrate the `useAssistant` hook from `ai/react` to stream the messages and status.

```tsx filename='pages/index.tsx'
import { Message, useAssistant } from 'ai/react';

export default function Page() {
  const { status, messages, input, submitMessage, handleInputChange } =
    useAssistant({ api: '/api/assistant' });

  return (
    <div className="flex flex-col gap-2">
      <div className="p-2">status: {status}</div>

      <div className="flex flex-col p-2 gap-2">
        {messages.map((message: Message) => (
          <div key={message.id} className="flex flex-row gap-2">
            <div className="w-24 text-zinc-500">{`${message.role}: `}</div>
            <div className="w-full">{message.content}</div>
          </div>
        ))}
      </div>

      <form onSubmit={submitMessage} className="fixed bottom-0 p-2 w-full">
        <input
          disabled={status !== 'awaiting_message'}
          value={input}
          onChange={handleInputChange}
          className="bg-zinc-100 w-full p-2"
        />
      </form>
    </div>
  );
}
```

## Server

Next, you will create an API route for `api/assistant` to handle the assistant's messages and responses. You will use the `AssistantResponse` function from `ai` to stream the assistant's responses back to the `useAssistant` hook on the client.

```tsx filename='app/api/assistant/route.ts'
import { AssistantResponse } from 'ai';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || '',
});

export async function POST(req: Request) {
  const input: {
    threadId: string | null;
    message: string;
  } = await req.json();

  const threadId = input.threadId ?? (await openai.beta.threads.create({})).id;

  const createdMessage = await openai.beta.threads.messages.create(threadId, {
    role: 'user',
    content: input.message,
  });

  return AssistantResponse(
    { threadId, messageId: createdMessage.id },
    async ({ forwardStream }) => {
      const runStream = openai.beta.threads.runs.stream(threadId, {
        assistant_id:
          process.env.ASSISTANT_ID ??
          (() => {
            throw new Error('ASSISTANT_ID is not set');
          })(),
      });

      let runResult = await forwardStream(runStream);

      while (
        runResult?.status === 'requires_action' &&
        runResult.required_action?.type === 'submit_tool_outputs'
      ) {
        const tool_outputs =
          runResult.required_action.submit_tool_outputs.tool_calls.map(
            (toolCall: any) => {
              const parameters = JSON.parse(toolCall.function.arguments);

              switch (toolCall.function.name) {
                case 'celsiusToFahrenheit':
                  const celsius = parseFloat(parameters.value);
                  const fahrenheit = celsius * (9 / 5) + 32;

                  return {
                    tool_call_id: toolCall.id,
                    output: `${celsius}°C is ${fahrenheit.toFixed(2)}°F`,
                  };

                default:
                  throw new Error(
                    `Unknown tool call function: ${toolCall.function.name}`,
                  );
              }
            },
          );

        runResult = await forwardStream(
          openai.beta.threads.runs.submitToolOutputsStream(
            threadId,
            runResult.id,
            { tool_outputs },
          ),
        );
      }
    },
  );
}
```

---

<GithubLink link="https://github.com/vercel/ai/blob/main/examples/next-openai-pages/pages/assistants/stream-assistant-response-with-tools/index.tsx" />

---
title: OpenAI Assistants
description: Learn to use the OpenAI Assistant API using the AI SDK in your Next.js Pages Router application
---

# Assistants

In this section, you will learn to use OpenAI's Assistant API along with the `useAssistant` hook from `ai/react`.

<IndexCards
  cards={[
    {
      title: 'Stream Assistant Response',
      description:
        'Learn to stream assistant response from OpenAI Assistant API.',
      href: '/examples/next-pages/assistants/stream-assistant-responses',
    },
    {
      title: 'Stream Assistant Response with Tools',
      description:
        'Learn to stream assistant responses with tool calls from OpenAI Assistant API.',
      href: '/examples/next-pages/assistants/stream-assistant-responses-with-tools',
    },
  ]}
/>

---
title: Next.js
description: Explore examples of how to use the AI SDK in a Next.js Application.
---

# Next.js

The following examples demonstrate how to achieve different use cases while integrating AI into your Next.js application.

---
title: Generate Text
description: Learn to generate text in a Node.js application.
---

# Generate Text

The most basic LLM use case is generating text based on a prompt.
For example, you may want to generate a response to a question or summarize a body of text.
The `generateText` function can be used to generate text based on the input prompt.

```ts file='index.ts'
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-3.5-turbo'),
  prompt: 'Why is the sky blue?',
});

console.log(result);
```

---
title: Stream Text
description: Learn to stream text in a Node.js application.
---

# Stream Text

Text generation can sometimes take a long time to complete, especially when you're generating a couple of paragraphs.
In such cases, it is useful to stream the text to the client in real-time.
This allows the client to display the generated text as it is being generated,
rather than have users wait for it to complete before displaying the result.

```txt
Introducing "Joyful Hearts Day" - a holiday dedicated to spreading love, joy, and kindness to others.

On Joyful Hearts Day, people exchange handmade cards, gifts, and acts of kindness to show appreciation and love for their friends, family, and community members. It is a day to focus on positivity and gratitude, spreading happiness and warmth to those around us.

Traditions include decorating homes and public spaces with hearts and bright colors, hosting community events such as charity drives, volunteer projects, and festive gatherings. People also participate in random acts of kindness, such as paying for someone's coffee, leaving encouraging notes for strangers, or simply offering a helping hand to those in need.

One of the main traditions of Joyful Hearts Day is the "Heart Exchange" where people write heartfelt messages to loved ones and exchange them in person or through mail. These messages can be words of encouragement, expressions of gratitude, or simply a reminder of how much they are loved.

Overall, Joyful Hearts Day is a day to celebrate love, kindness, and positivity, and to spread joy and happiness to all those around us. It is a reminder to appreciate the people in our lives and to make the world a brighter and more loving place.
```

## Without reader

```ts file='index.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-3.5-turbo'),
  maxTokens: 512,
  temperature: 0.3,
  maxRetries: 5,
  prompt: 'Invent a new holiday and describe its traditions.',
});

for await (const textPart of result.textStream) {
  console.log(textPart);
}
```

## With reader

```ts file='index.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-3.5-turbo'),
  maxTokens: 512,
  temperature: 0.3,
  maxRetries: 5,
  prompt: 'Invent a new holiday and describe its traditions.',
});

const reader = result.textStream.getReader();

while (true) {
  const { done, value } = await reader.read();
  if (done) {
    break;
  }
  console.log(value);
}
```

---
title: Generate Text with Chat Prompt
description: Learn to generate text with chat prompt in a Node.js application.
---

# Generate text with chat prompt

Previously, we were able to generate text and objects using either a single message prompt, a system prompt, or a combination of both of them. However, there may be times when you want to generate text based on a series of messages.

A chat completion allows you to generate text based on a series of messages. This series of messages can be any series of interactions between any number of systems, but the most popular and relatable use case has been a series of messages that represent a conversation between a user and a model.

```ts file='index.ts'
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-3.5-turbo'),
  maxTokens: 1024,
  system: 'You are a helpful chatbot.',
  messages: [
    {
      role: 'user',
      content: 'Hello!',
    },
    {
      role: 'assistant',
      content: 'Hello! How can I help you today?',
    },
    {
      role: 'user',
      content: 'I need help with my computer.',
    },
  ],
});

console.log(result.text);
```

---
title: Stream Text with Chat Prompt
description: Learn to stream text with chat prompt in a Node.js application.
---

# Stream text with chat prompt

Text generation can sometimes take a long time to finish, especially when the response is big.
In such cases, it is useful to stream the chat completion to the client in real-time.
This allows the client to display the new message as it is being generated by the model,
rather than have users wait for it to finish.

```ts file='index.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-3.5-turbo'),
  maxTokens: 1024,
  system: 'You are a helpful chatbot.',
  messages: [
    {
      role: 'user',
      content: 'Hello!',
    },
    {
      role: 'assistant',
      content: 'Hello! How can I help you today?',
    },
    {
      role: 'user',
      content: 'I need help with my computer.',
    },
  ],
});

for await (const textPart of result.textStream) {
  console.log(textPart);
}
```

---
title: Generate Text with Image Prompt
description: Learn to generate text with image prompt in a Node.js application.
---

# Generate text with image prompt

Some language models that support vision capabilities accept images as part of the prompt. Here are some of the different [formats](/docs/reference/ai-sdk-core/generate-text#content-image) you can use to include images as input.

## URL

```ts file='index.ts'
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  maxTokens: 512,
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'what are the red things in this image?',
        },
        {
          type: 'image',
          image: new URL(
            'https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/2024_Solar_Eclipse_Prominences.jpg/720px-2024_Solar_Eclipse_Prominences.jpg',
          ),
        },
      ],
    },
  ],
});

console.log(result);
```

## File Buffer

```ts file='index.ts'
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import fs from 'fs';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  maxTokens: 512,
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'what are the red things in this image?',
        },
        {
          type: 'image',
          image: fs.readFileSync('./node/attachments/eclipse.jpg'),
        },
      ],
    },
  ],
});

console.log(result);
```

---
title: RAG
description: Learn how to build a Retrieval Augmented Generation application with Node.js.
---

# RAG

Retrieval Augmented Generation (RAG) is a technique that enhances the capabilities of language models by providing them with relevant information from external sources during the generation process.
This approach allows the model to access and incorporate up-to-date or specific knowledge that may not be present in its original training data.

This example uses [the following essay](https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt) as an input (`essay.txt`). This example uses a simple in-memory vector database to store and retrieve relevant information. For a more in-depth guide, check out the [RAG Chatbot Guide](/docs/guides/rag-chatbot) which will show you how to build a RAG chatbot with [Next.js](https://nextjs.org), [Drizzle ORM](https://orm.drizzle.team/) and [Postgres](https://postgresql.org).

```ts
import fs from 'fs';
import path from 'path';
import dotenv from 'dotenv';
import { openai } from '@ai-sdk/openai';
import { cosineSimilarity, embed, embedMany, generateText } from 'ai';

dotenv.config();

async function main() {
  const db: { embedding: number[]; value: string }[] = [];

  const essay = fs.readFileSync(path.join(__dirname, 'essay.txt'), 'utf8');
  const chunks = essay
    .split('.')
    .map(chunk => chunk.trim())
    .filter(chunk => chunk.length > 0 && chunk !== '\n');

  const { embeddings } = await embedMany({
    model: openai.embedding('text-embedding-3-small'),
    values: chunks,
  });
  embeddings.forEach((e, i) => {
    db.push({
      embedding: e,
      value: chunks[i],
    });
  });

  const input =
    'What were the two main things the author worked on before college?';

  const { embedding } = await embed({
    model: openai.embedding('text-embedding-3-small'),
    value: input,
  });
  const context = db
    .map(item => ({
      document: item,
      similarity: cosineSimilarity(embedding, item.embedding),
    }))
    .sort((a, b) => b.similarity - a.similarity)
    .slice(0, 3)
    .map(r => r.document.value)
    .join('\n');

  const { text } = await generateText({
    model: openai('gpt-4o'),
    prompt: `Answer the following question based only on the provided context:
             ${context}

             Question: ${input}`,
  });
  console.log(text);
}

main().catch(console.error);
```

---
title: Generating Text
description: Learn how to generate text in a Node.js application.
---

# Generating Text

The following sections will guide you through generating text with Node.js and the AI SDK.

<IndexCards
  cards={[
    {
      title: 'Generate Text',
      href: '/examples/node/generating-text/generate-text',
    },
    {
      title: 'Stream Text',
      href: '/examples/node/generating-text/stream-text',
    },
    {
      title: 'Generate Text with Chat Prompt',
      href: '/examples/node/generating-text/generate-text-with-chat-prompt',
    },
    {
      title: 'Stream Text with Chat Prompt',
      href: '/examples/node/generating-text/stream-text-with-chat-prompt',
    },
    {
      title: 'Generate Text with Image Prompt',
      href: '/examples/node/generating-text/generate-text-with-image-prompt',
    },
  ]}
/>

---
title: Generating an Object
description: Learn how to generate an object in a Node.js application.
---

# Generate Object

Earlier functions like `generateText` and `streamText` gave us the ability to generate unstructured text. However, if you want to generate structured data like JSON, you can provide a schema that describes the structure of your desired object to the `generateObject` function.

The function requires you to provide a schema using [zod](https://zod.dev), a library for defining schemas for JavaScript objects. By using zod, you can also use it to validate the generated object and ensure that it conforms to the specified structure.

```ts file='index.ts'
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const result = await generateObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(
        z.object({
          name: z.string(),
          amount: z.string(),
        }),
      ),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(JSON.stringify(result.object.recipe, null, 2));
```

---
title: Generate Object with Image Prompt
description: Learn how to generate an object with an image prompt in a Node.js application.
---

# Add images to prompt

Some language models that support vision capabilities accept images as part of the prompt. Here are some of the different [formats](/docs/reference/ai-sdk-core/generate-text#content-image) you can use to include images as input.

## URL

```ts file='index.ts'
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import dotenv from 'dotenv';
import { z } from 'zod';

dotenv.config();

async function main() {
  const { object } = await generateObject({
    model: openai('gpt-4-turbo'),
    maxTokens: 512,
    schema: z.object({
      stamps: z.array(
        z.object({
          country: z.string(),
          date: z.string(),
        }),
      ),
    }),
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'list all the stamps in these passport pages?',
          },
          {
            type: 'image',
            image: new URL(
              'https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/WW2_Spanish_official_passport.jpg/1498px-WW2_Spanish_official_passport.jpg',
            ),
          },
        ],
      },
    ],
  });

  console.log(object);
}

main();
```

## File Buffer

```ts file='index.ts'
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import dotenv from 'dotenv';
import { z } from 'zod';

dotenv.config();

async function main() {
  const { object } = await generateObject({
    model: openai('gpt-4-turbo'),
    maxTokens: 512,
    schema: z.object({
      stamps: z.array(
        z.object({
          country: z.string(),
          date: z.string(),
        }),
      ),
    }),
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'list all the stamps in these passport pages?',
          },
          {
            type: 'image',
            image: fs.readFileSync('./node/attachments/eclipse.jpg'),
          },
        ],
      },
    ],
  });

  console.log(object);
}

main();
```

---
title: Generating Structured Data
description: Learn how to generate structured data in a Node.js application.
---

# Generating Structured Data

The following sections will guide you through generating structured data with Node.js and the AI SDK.

<IndexCards
  cards={[
    {
      title: 'Generating an Object',
      href: '/examples/node/generating-structured-data/generate-object',
    },
    {
      title: 'Generate Object with Image Prompt',
      href: '/examples/node/generating-structured-data/add-images-to-prompt',
    },
  ]}
/>

---
title: Streaming Partial Objects
description: Examples of how to stream partial objects.
---

# Streaming Partial Objects

Object generation can sometimes take a long time to complete,
especially when you're generating a large schema.

In Generative UI use cases, it is useful to stream the object to the client in real-time
to render UIs as the object is being generated.
You can use the [`streamObject`](/docs/reference/ai-sdk-core/stream-object) function to generate partial object streams.

```ts file='index.ts'
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const { partialObjectStream } = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

for await (const partialObject of partialObjectStream) {
  console.clear();
  console.log(partialObject);
}
```

---
title: Recording Token Usage
description: Examples of how to record token usage when streaming structured data.
---

# Recording Token Usage

When you're streaming structured data with [`streamObject`](/docs/reference/ai-sdk-core/stream-object),
you may want to record the token usage for billing purposes.

## `onFinish` Callback

You can use the `onFinish` callback to record token usage.
It is called when the stream is finished.

```ts file='index.ts' highlight={"15-17"}
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const result = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
  onFinish({ usage }) {
    console.log('Token usage:', usage);
  },
});
```

## `usage` Promise

The [`streamObject`](/docs/reference/ai-sdk-core/stream-object) result contains a `usage` promise that resolves to the total token usage.

```ts file='index.ts' highlight={"29,32"}
import { openai } from '@ai-sdk/openai';
import { streamObject, TokenUsage } from 'ai';
import { z } from 'zod';

const result = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

// your custom function to record token usage:
function recordTokenUsage({
  promptTokens,
  completionTokens,
  totalTokens,
}: TokenUsage) {
  console.log('Prompt tokens:', promptTokens);
  console.log('Completion tokens:', completionTokens);
  console.log('Total tokens:', totalTokens);
}

// use as promise:
result.usage.then(recordTokenUsage);

// use with async/await:
recordTokenUsage(await result.usage);

// note: the stream needs to be consumed because of backpressure
for await (const partialObject of result.partialObjectStream) {
}
```

---
title: Recording Final Object
description: Examples of how to record the final object when streaming structured data.
---

# Recording Final Object

When you're streaming structured data, you may want to record the final object for logging or other purposes.

## `onFinish` Callback

You can use the `onFinish` callback to record the final object.
It is called when the stream is finished.

The `object` parameter contains the final object, or `undefined` if the type validation fails.
There is also an `error` parameter that contains error when e.g. the object does not match the schema.

```ts file='index.ts' highlight={"15-23"}
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const result = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
  onFinish({ object, error }) {
    // handle type validation failure (when the object does not match the schema):
    if (object === undefined) {
      console.error('Error:', error);
      return;
    }

    console.log('Final object:', JSON.stringify(object, null, 2));
  },
});
```

## `object` Promise

The [`streamObject`](/docs/reference/ai-sdk-core/stream-object) result contains an `object` promise that resolves to the final object.
The object is fully typed. When the type validation according to the schema fails, the promise will be rejected with a `TypeValidationError`.

```ts file='index.ts' highlight={"17-26"}
import { openai } from '@ai-sdk/openai';
import { streamObject } from 'ai';
import { z } from 'zod';

const result = streamObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

result.object
  .then(({ recipe }) => {
    // do something with the fully typed, final object:
    console.log('Recipe:', JSON.stringify(recipe, null, 2));
  })
  .catch(error => {
    // handle type validation failure
    // (when the object does not match the schema):
    console.error(error);
  });

// note: the stream needs to be consumed because of backpressure
for await (const partialObject of result.partialObjectStream) {
}
```

---
title: Streaming Structured Data
description: Examples of how to stream structured data.
---

# Streaming Structured Data

The following sections will guide you through streaming structured data with Node.js and the AI SDK.

<IndexCards
  cards={[
    {
      title: 'Streaming Partial Objects',
      href: '/examples/node/streaming-structured-data/stream-object',
    },
    {
      title: 'Recording Token Usage',
      href: '/examples/node/streaming-structured-data/token-usage',
    },
    {
      title: 'Recording Final Object',
      href: '/examples/node/streaming-structured-data/object',
    },
  ]}
/>

---
title: Call Tool
description: Learn how to make a language model call a tool in a Node.js application.
---

# Call Tool

Some models allow developers to provide a list of tools that can be called at any time during a generation.
This is useful for extending the capabilites of a language model to either use logic or data to interact with systems external to the model.

```ts
import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
    cityAttractions: tool({
      parameters: z.object({ city: z.string() }),
    }),
  },
  prompt:
    'What is the weather in San Francisco and what attractions should I visit?',
});
```

## Accessing Tool Calls and Tool Results

If the model decides to call a tool, it will generate a tool call. You can access the tool call by checking the `toolCalls` property on the result.

```ts highlight="31-44"
import { openai } from '@ai-sdk/openai';
import { generateText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';

dotenv.config();

async function main() {
  const result = await generateText({
    model: openai('gpt-3.5-turbo'),
    maxTokens: 512,
    tools: {
      weather: tool({
        description: 'Get the weather in a location',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      }),
      cityAttractions: tool({
        parameters: z.object({ city: z.string() }),
      }),
    },
    prompt:
      'What is the weather in San Francisco and what attractions should I visit?',
  });

  // typed tool calls:
  for (const toolCall of result.toolCalls) {
    switch (toolCall.toolName) {
      case 'cityAttractions': {
        toolCall.args.city; // string
        break;
      }

      case 'weather': {
        toolCall.args.location; // string
        break;
      }
    }
  }

  console.log(JSON.stringify(result, null, 2));
}

main().catch(console.error);
```

## Accessing Tool Results

You can access the result of a tool call by checking the `toolResults` property on the result.

```ts highlight="31-41"
import { openai } from '@ai-sdk/openai';
import { generateText, tool } from 'ai';
import dotenv from 'dotenv';
import { z } from 'zod';

dotenv.config();

async function main() {
  const result = await generateText({
    model: openai('gpt-3.5-turbo'),
    maxTokens: 512,
    tools: {
      weather: tool({
        description: 'Get the weather in a location',
        parameters: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      }),
      cityAttractions: tool({
        parameters: z.object({ city: z.string() }),
      }),
    },
    prompt:
      'What is the weather in San Francisco and what attractions should I visit?',
  });

  // typed tool results for tools with execute method:
  for (const toolResult of result.toolResults) {
    switch (toolResult.toolName) {
      case 'weather': {
        toolResult.args.location; // string
        toolResult.result.location; // string
        toolResult.result.temperature; // number
        break;
      }
    }
  }

  console.log(JSON.stringify(result, null, 2));
}

main().catch(console.error);
```

<Note>
  `toolResults` will only be available if the tool has an `execute` function.
</Note>

## Model Response

When using tools, it's important to note that the model won't respond with the tool call results by default.
This is because the model has technically already generated its response to the prompt: the tool call.
Many use cases will require the model to summarise the results of the tool call within the context of the original prompt automatically.
You can achieve this by [using `maxSteps`](/examples/node/tools/call-tools-with-automatic-roundtrips)
which will automatically send toolResults to the model to trigger another generation.

---
title: Call Tool with Image Prompt
description: Learn how to make a language model call a tool with an image prompt in a Node.js application.
---

# Call Tool with Image Prompt

Some language models that support vision capabilities accept images as part of the prompt. Here are some of the different [formats](/docs/reference/ai-sdk-core/generate-text#content-image) you can use to include images as input.

```ts
import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'can you log this meal for me?' },
        {
          type: 'image',
          image: new URL(
            'https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Cheeseburger_%2817237580619%29.jpg/640px-Cheeseburger_%2817237580619%29.jpg',
          ),
        },
      ],
    },
  ],
  tools: {
    logFood: tool({
      description: 'Log a food item',
      parameters: z.object({
        name: z.string(),
        calories: z.number(),
      }),
      execute({ name, calories }) {
        storeInDatabase({ name, calories });
      },
    }),
  },
});
```

---
title: Call Tools in Parallel
description: Learn how to call tools in parallel in a Node.js application.
---

# Call Tools in Parallel

Some language models support calling tools in parallel.
This is particularly useful when multiple tools are independent of each other and can be executed in parallel during the same generation step.

```ts
import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }: { location: string }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
    cityAttractions: tool({
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }: { city: string }) => {
        if (city === 'San Francisco') {
          return {
            attractions: [
              'Golden Gate Bridge',
              'Alcatraz Island',
              "Fisherman's Wharf",
            ],
          };
        } else {
          return { attractions: [] };
        }
      },
    }),
  },
  prompt:
    'What is the weather in San Francisco and what attractions should I visit?',
});

console.log(result);
```

---
title: Multi-step Tool Calls
description: Learn how to execute multi-step calls in a Node.js application.
---

# Multi-step Tool Calls

Models call tools to gather information or perform actions that are not directly available to the model.
When tool results are available, the model can use them to generate another response.

You can enable multi-step tool calls in `generateText` by setting the `maxSteps` option to
a number greater than 1.
This option specifies the maximum number of steps (i.e., LLM calls) that can be made to prevent infinite loops.

```ts highlight={"7"}
import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const { text } = await generateText({
  model: openai('gpt-4-turbo'),
  maxSteps: 5,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }: { location: string }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  prompt: 'What is the weather in San Francisco?',
});
```

---
title: Tools
description: Learn how to make language models use tools in a Node.js application.
---

# Tools

The following sections will guide you through tool use with Node.js and the AI SDK.

<IndexCards
  cards={[
    {
      title: 'Call Tool',
      href: '/examples/node/tools/call-tool',
    },
    {
      title: 'Call Tool with Image Prompt',
      href: '/examples/node/tools/call-tool-with-image-prompt',
    },
    {
      title: 'Call Tools in Parallel',
      href: '/examples/node/tools/call-tools-in-parallel',
    },
    {
      title: 'Multi-step Tool Calls',
      href: '/examples/node/tools/call-tools-with-automatic-roundtrips',
    },
  ]}
/>

---
title: Node.js
description: Explore examples of using the AI SDK in your Node.js application.
---

# Node.js Examples

The following section will guide you through the various capabilities of the AI SDKs using minimal code examples.

<IndexCards
  cards={[
    {
      title: 'Generating Text',
      description: 'Generate text with Node.js.',
      href: '/examples/node/generating-text/',
    },
    {
      title: 'Generating Structured Data',
      description: 'Generate stuctured data with Node.js.',
      href: '/examples/node/generating-structured-data',
    },
    {
      title: 'Tools',
      description: 'Use tools with Node.js.',
      href: '/examples/node/tools',
    },
  ]}
/>

---
title: Generating Text
description: Learn to generate text using the AI SDK in your Next.js App Router application
---

# Generate Text

<Note>
  This example uses React Server Components (RSC). If you want to client side
  rendering and hooks instead, check out the ["generate text" example with
  useState](/examples/next-pages/basics/generating-text).
</Note>

A situation may arise when you need to generate text based on a prompt. For example, you may want to generate a response to a question or summarize a body of text. The `generateText` function can be used to generate text based on the input prompt.

<Browser>
  <TextGeneration />
</Browser>

## Client

Let's create a simple React component that will call the `getAnswer` function when a button is clicked. The `getAnswer` function will call the `generateText` function from the `ai` module, which will then generate text based on the input prompt.

```tsx filename="app/page.tsx"
'use client';

import { useState } from 'react';
import { getAnswer } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [generation, setGeneration] = useState<string>('');

  return (
    <div>
      <button
        onClick={async () => {
          const { text } = await getAnswer('Why is the sky blue?');
          setGeneration(text);
        }}
      >
        Answer
      </button>
      <div>{generation}</div>
    </div>
  );
}
```

## Server

On the server side, we need to implement the `getAnswer` function, which will call the `generateText` function from the `ai` module. The `generateText` function will generate text based on the input prompt.

```typescript filename='app/actions.ts'
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function getAnswer(question: string) {
  const { text, finishReason, usage } = await generateText({
    model: openai('gpt-3.5-turbo'),
    prompt: question,
  });

  return { text, finishReason, usage };
}
```

---
title: Stream Text Generation
description: Learn to stream text generations using the AI SDK in your Next.js App Router application
---

# Stream Text Generation

<Note>
  This example uses React Server Components (RSC). If you want to client side
  rendering and hooks instead, check out the ["stream text" example with
  useCompletion](/examples/next-pages/basics/streaming-text-generation).
</Note>

Text generation can sometimes take a long time to complete, especially when you're generating a couple of paragraphs. In such cases, it is useful to stream the text generation process to the client in real-time. This allows the client to display the generated text as it is being generated, rather than have users wait for it to complete before displaying the result.

<Browser>
  <TextGeneration stream />
</Browser>

## Client

Let's create a simple React component that will call the `generate` function when a button is clicked. The `generate` function will call the `streamText` function, which will then generate text based on the input prompt. To consume the stream of text in the client, we will use the `readStreamableValue` function from the `ai/rsc` module.

```tsx filename="app/page.tsx"
'use client';

import { useState } from 'react';
import { generate } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [generation, setGeneration] = useState<string>('');

  return (
    <div>
      <button
        onClick={async () => {
          const { output } = await generate('Why is the sky blue?');

          for await (const delta of readStreamableValue(output)) {
            setGeneration(currentGeneration => `${currentGeneration}${delta}`);
          }
        }}
      >
        Ask
      </button>

      <div>{generation}</div>
    </div>
  );
}
```

## Server

On the server side, we need to implement the `generate` function, which will call the `streamText` function. The `streamText` function will generate text based on the input prompt. In order to stream the text generation to the client, we will use `createStreamableValue` that can wrap any changable value and stream it to the client.

Using DevTools, we can see the text generation being streamed to the client in real-time.

```typescript filename='app/actions.ts'
'use server';

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';

export async function generate(input: string) {
  const stream = createStreamableValue('');

  (async () => {
    const { textStream } = streamText({
      model: openai('gpt-3.5-turbo'),
      prompt: input,
    });

    for await (const delta of textStream) {
      stream.update(delta);
    }

    stream.done();
  })();

  return { output: stream.value };
}
```

---
title: Generating Structured Data
description: Learn to generate structured data using the AI SDK in your Next.js App Router application
---

# Generate Object

<Note>
  This example uses React Server Components (RSC). If you want to client side
  rendering and hooks instead, check out the ["generate object" example with
  useState](/examples/next-pages/basics/generating-object).
</Note>

Earlier functions like `generateText` and `streamText` gave us the ability to generate unstructured text. However, if you want to generate structured data like JSON, you can provide a schema that describes the structure of your desired object to the `generateObject` function.

The function requires you to provide a schema using [zod](https://zod.dev), a library for defining schemas for JavaScript objects. By using zod, you can also use it to validate the generated object and ensure that it conforms to the specified structure.

<Browser>
  <ObjectGeneration
    object={{
      notifications: [
        {
          name: 'Jamie Roberts',
          message: "Hey! How's the study grind going? Need a coffee boost?",
          minutesAgo: 15,
        },
        {
          name: 'Prof. Morgan',
          message:
            'Reminder: Your term paper is due promptly at 8 AM tomorrow. Please ensure it meets the submission guidelines outlined.',
          minutesAgo: 46,
        },
        {
          name: 'Alex Chen',
          message:
            "Dude, urgent! Borrow your notes for tomorrow's exam? I swear mine got eaten by my dog!",
          minutesAgo: 30,
        },
      ],
    }}
  />
</Browser>

## Client

Let's create a simple React component that will call the `getNotifications` function when a button is clicked. The function will generate a list of notifications as described in the schema.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { getNotifications } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [generation, setGeneration] = useState<string>('');

  return (
    <div>
      <button
        onClick={async () => {
          const { notifications } = await getNotifications(
            'Messages during finals week.',
          );

          setGeneration(JSON.stringify(notifications, null, 2));
        }}
      >
        View Notifications
      </button>

      <pre>{generation}</pre>
    </div>
  );
}
```

## Server

Now let's implement the `getNotifications` function. We'll use the `generateObject` function to generate the list of notifications based on the schema we defined earlier.

```typescript filename='app/actions.ts'
'use server';

import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

export async function getNotifications(input: string) {
  'use server';

  const { object: notifications } = await generateObject({
    model: openai('gpt-4-turbo'),
    system: 'You generate three notifications for a messages app.',
    prompt: input,
    schema: z.object({
      notifications: z.array(
        z.object({
          name: z.string().describe('Name of a fictional person.'),
          message: z.string().describe('Do not use emojis or links.'),
          minutesAgo: z.number(),
        }),
      ),
    }),
  });

  return { notifications };
}
```

---
title: Streaming Object Generation
description: Learn to stream object generations using the AI SDK in your Next.js App Router application
---

# Stream Object Generation

<Note>
  This example uses React Server Components (RSC). If you want to client side
  rendering and hooks instead, check out the ["streaming object generation"
  example with
  useObject](/examples/next-pages/basics/streaming-object-generation).
</Note>

Object generation can sometimes take a long time to complete, especially when you're generating a large schema. In such cases, it is useful to stream the object generation process to the client in real-time. This allows the client to display the generated object as it is being generated, rather than have users wait for it to complete before displaying the result.

<Browser>
  <ObjectGeneration
    stream
    object={{
      notifications: [
        {
          name: 'Jamie Roberts',
          message: "Hey! How's the study grind going? Need a coffee boost?",
          minutesAgo: 15,
        },
        {
          name: 'Prof. Morgan',
          message:
            'Reminder: Your term paper is due promptly at 8 AM tomorrow. Please ensure it meets the submission guidelines outlined.',
          minutesAgo: 46,
        },
        {
          name: 'Alex Chen',
          message:
            "Dude, urgent! Borrow your notes for tomorrow's exam? I swear mine got eaten by my dog!",
          minutesAgo: 30,
        },
      ],
    }}
  />
</Browser>

## Client

Let's create a simple React component that will call the `getNotifications` function when a button is clicked. The function will generate a list of notifications as described in the schema.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { generate } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [generation, setGeneration] = useState<string>('');

  return (
    <div>
      <button
        onClick={async () => {
          const { object } = await generate('Messages during finals week.');

          for await (const partialObject of readStreamableValue(object)) {
            if (partialObject) {
              setGeneration(
                JSON.stringify(partialObject.notifications, null, 2),
              );
            }
          }
        }}
      >
        Ask
      </button>

      <pre>{generation}</pre>
    </div>
  );
}
```

## Server

Now let's implement the `getNotifications` function. We'll use the `generateObject` function to generate the list of fictional notifications based on the schema we defined earlier.

```typescript filename='app/actions.ts'
'use server';

import { streamObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';
import { z } from 'zod';

export async function generate(input: string) {
  'use server';

  const stream = createStreamableValue();

  (async () => {
    const { partialObjectStream } = streamObject({
      model: openai('gpt-4-turbo'),
      system: 'You generate three notifications for a messages app.',
      prompt: input,
      schema: z.object({
        notifications: z.array(
          z.object({
            name: z.string().describe('Name of a fictional person.'),
            message: z.string().describe('Do not use emojis or links.'),
            minutesAgo: z.number(),
          }),
        ),
      }),
    });

    for await (const partialObject of partialObjectStream) {
      stream.update(partialObject);
    }

    stream.done();
  })();

  return { object: stream.value };
}
```

---
title: Basics
description: Learn the basics of using the AI SDK in your Next.js App Router application.
---

# Basics

One of the most basic things you can do with language models is to generate text. In this section, you will learn to generate text, and also stream it to the client.

Beyond text, you will also learn to generate structured data by providing a schema of your choice, and also stream it to the client.

---
title: Generation Chat Completion
description: Learn to generate chat completions using the AI SDK in your Next.js App Router application
---

# Generate Chat Completion

Previously, we were able to generate text and objects using either a single message prompt, a system prompt, or a combination of both of them. However, there may be times when you want to generate text based on a series of messages.

A chat completion allows you to generate text based on a series of messages. This series of messages can be any series of interactions between any number of systems, but the most popular and relatable use case has been a series of messages that represent a conversation between a user and a model.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{ role: 'User', content: 'Why is the sky blue?' }}
    outputMessage={{
      role: 'Assistant',
      content: 'The sky is blue because of rayleigh scattering.',
    }}
  />
</Browser>

## Client

Let's create a simple conversation between a user and a model, and place a button that will call `continueConversation`.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { Message, continueConversation } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [conversation, setConversation] = useState<Message[]>([]);
  const [input, setInput] = useState<string>('');

  return (
    <div>
      <div>
        {conversation.map((message, index) => (
          <div key={index}>
            {message.role}: {message.content}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            const { messages } = await continueConversation([
              ...conversation,
              { role: 'user', content: input },
            ]);

            setConversation(messages);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

Now, let's implement the `continueConversation` function that will insert the user's message into the conversation and generate a response.

```typescript filename='app/actions.ts'
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export async function continueConversation(history: Message[]) {
  'use server';

  const { text } = await generateText({
    model: openai('gpt-3.5-turbo'),
    system: 'You are a friendly assistant!',
    messages: history,
  });

  return {
    messages: [
      ...history,
      {
        role: 'assistant' as const,
        content: text,
      },
    ],
  };
}
```

---
title: Stream Chat Completion
description: Learn to stream chat completions using the AI SDK in your Next.js App Router application
---

# Stream Chat Completion

Chat completion can sometimes take a long time to finish, especially when the response is big. In such cases, it is useful to stream the chat completion to the client in real-time. This allows the client to display the new message as it is being generated by the model, rather than have users wait for it to finish.

<Browser>
  <ChatGeneration
    stream
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{ role: 'User', content: 'Why is the sky blue?' }}
    outputMessage={{
      role: 'Assistant',
      content: 'The sky is blue because of rayleigh scattering.',
    }}
  />
</Browser>

## Client

Let's create a simple conversation between a user and a model, and place a button that will call `continueConversation`.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { Message, continueConversation } from './actions';
import { readStreamableValue } from 'ai/rsc';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [conversation, setConversation] = useState<Message[]>([]);
  const [input, setInput] = useState<string>('');

  return (
    <div>
      <div>
        {conversation.map((message, index) => (
          <div key={index}>
            {message.role}: {message.content}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            const { messages, newMessage } = await continueConversation([
              ...conversation,
              { role: 'user', content: input },
            ]);

            let textContent = '';

            for await (const delta of readStreamableValue(newMessage)) {
              textContent = `${textContent}${delta}`;

              setConversation([
                ...messages,
                { role: 'assistant', content: textContent },
              ]);
            }
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

Now, let's implement the `continueConversation` function that will insert the user's message into the conversation and stream back the new message.

```typescript filename='app/actions.ts'
'use server';

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';

export interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export async function continueConversation(history: Message[]) {
  'use server';

  const stream = createStreamableValue();

  (async () => {
    const { textStream } = streamText({
      model: openai('gpt-3.5-turbo'),
      system:
        "You are a dude that doesn't drop character until the DVD commentary.",
      messages: history,
    });

    for await (const text of textStream) {
      stream.update(text);
    }

    stream.done();
  })();

  return {
    messages: history,
    newMessage: stream.value,
  };
}
```

---
title: Chat
description: Learn how to generate chat completions using the AI SDK in your Next.js application
---

# Chat

So far we've learned how to generate text and structured data using single prompts. In this section, we will learn to use `messages` to add a sequence of messages to the language model and generate the response based on the context of the conversation – called chat completion.

---
title: Call Tools
description: Learn to call tools using the AI SDK in your Next.js App Router application
---

# Call Tools

Some models allow developers to provide a list of tools that can be called at any time during a generation. This is useful for extending the capabilites of a language model to either use logic or data to interact with systems external to the model.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is 24 celsius in fahrenheit?',
    }}
    outputMessage={{
      role: 'Assistant',
      content: '24°C is 75.20°F',
    }}
  />
</Browser>

## Client

Let's create a simple conversation between a user and model and place a button that will call `continueConversation`.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { Message, continueConversation } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [conversation, setConversation] = useState<Message[]>([]);
  const [input, setInput] = useState<string>('');

  return (
    <div>
      <div>
        {conversation.map((message, index) => (
          <div key={index}>
            {message.role}: {message.content}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            const { messages } = await continueConversation([
              ...conversation,
              { role: 'user', content: input },
            ]);

            setConversation(messages);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

Now, let's implement the `continueConversation` action that uses `generateText` to generate a response to the user's question. We will use the [`tools`](/docs/reference/ai-sdk-core/generate-text#tools) parameter to specify our own function called `celsiusToFahrenheit` that will convert a user given value in celsius to fahrenheit.

We will use zod to specify the schema for the `celsiusToFahrenheit` function's parameters.

```tsx filename='app/actions.ts'
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

export interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export async function continueConversation(history: Message[]) {
  'use server';

  const { text, toolResults } = await generateText({
    model: openai('gpt-3.5-turbo'),
    system: 'You are a friendly assistant!',
    messages: history,
    tools: {
      celsiusToFahrenheit: {
        description: 'Converts celsius to fahrenheit',
        parameters: z.object({
          value: z.string().describe('The value in celsius'),
        }),
        execute: async ({ value }) => {
          const celsius = parseFloat(value);
          const fahrenheit = celsius * (9 / 5) + 32;
          return `${celsius}°C is ${fahrenheit.toFixed(2)}°F`;
        },
      },
    },
  });

  return {
    messages: [
      ...history,
      {
        role: 'assistant' as const,
        content:
          text || toolResults.map(toolResult => toolResult.result).join('\n'),
      },
    ],
  };
}
```

---
title: Call Tools in Parallel
description: Learn to call tools in parallel using the AI SDK in your Next.js App Router application
---

# Call Tools in Parallel

Some language models support calling tools in parallel. This is particularly useful when multiple tools are independent of each other and can be executed in parallel during the same generation step.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is the weather in Paris and New York?',
    }}
    outputMessage={{
      role: 'Assistant',
      content:
        'The weather is 24°C in New York and 25°C in Paris. It is sunny in both cities.',
    }}
  />
</Browser>

## Client

Let's modify our previous example to call `getWeather` tool for multiple cities in parallel.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { Message, continueConversation } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [conversation, setConversation] = useState<Message[]>([]);
  const [input, setInput] = useState<string>('');

  return (
    <div>
      <div>
        {conversation.map((message, index) => (
          <div key={index}>
            {message.role}: {message.content}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            const { messages } = await continueConversation([
              ...conversation,
              { role: 'user', content: input },
            ]);

            setConversation(messages);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

Let's update the tools object to now use the `getWeather` function instead.

```ts filename='app/actions.ts'
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

export interface Message {
  role: 'user' | 'assistant';
  content: string;
}

function getWeather({ city, unit }) {
  // This function would normally make an
  // API request to get the weather.

  return { value: 25, description: 'Sunny' };
}

export async function continueConversation(history: Message[]) {
  'use server';

  const { text, toolResults } = await generateText({
    model: openai('gpt-3.5-turbo'),
    system: 'You are a friendly weather assistant!',
    messages: history,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        parameters: z.object({
          city: z.string().describe('The city to get the weather for'),
          unit: z
            .enum(['C', 'F'])
            .describe('The unit to display the temperature in'),
        }),
        execute: async ({ city, unit }) => {
          const weather = getWeather({ city, unit });
          return `It is currently ${weather.value}°${unit} and ${weather.description} in ${city}!`;
        },
      },
    },
  });

  return {
    messages: [
      ...history,
      {
        role: 'assistant' as const,
        content:
          text || toolResults.map(toolResult => toolResult.result).join('\n'),
      },
    ],
  };
}
```

---
title: Render Interface during Tool Call
description: Learn to render a React component during a tool call using the AI SDK in your Next.js App Router application
---

# Render Interface During Tool Call

An interesting consequence of language models that can call tools is that this ability can be used to render visual interfaces by streaming React components to the client.

<Browser>
  <ChatGeneration
    history={[
      { role: 'User', content: 'How is it going?' },
      { role: 'Assistant', content: 'All good, how may I help you?' },
    ]}
    inputMessage={{
      role: 'User',
      content: 'What is the weather in San Francisco?',
    }}
    outputMessage={{
      role: 'Assistant',
      content: 'The weather is 24°C and sunny in San Francisco.',
      display: (
        <div className="py-4">
          <WeatherCard
            content={{
              weather: {
                temperature: 24,
                condition: 'Sunny',
              },
            }}
          />
        </div>
      ),
    }}
  />
</Browser>

## Client

We can make a few changes to our previous example where the assistant could get the weather for any city by calling the `getWeather` tool. This time, instead of returning text during the tool call, we will stream a React component that will be rendered on the client using `createStreamableUI` from `ai/rsc`.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { Message, continueConversation } from './actions';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [conversation, setConversation] = useState<Message[]>([]);
  const [input, setInput] = useState<string>('');

  return (
    <div>
      <div>
        {conversation.map((message, index) => (
          <div key={index}>
            {message.role}: {message.content}
            {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            const { messages } = await continueConversation([
              // exclude React components from being sent back to the server:
              ...conversation.map(({ role, content }) => ({ role, content })),
              { role: 'user', content: input },
            ]);

            setConversation(messages);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

```tsx filename='components/weather.tsx'
export async function Weather({ city, unit }) {
  const data = await fetch(
    `https://api.example.com/weather?city=${city}&unit=${unit}`,
  );

  return (
    <div>
      <div>{data.temperature}</div>
      <div>{data.unit}</div>
      <div>{data.description}</div>
    </div>
  );
}
```

## Server

```tsx highlight="13,17,33-34,47" filename='app/actions.tsx'
'use server';

import { Weather } from '@/components/weather';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableUI } from 'ai/rsc';
import { ReactNode } from 'react';
import { z } from 'zod';

export interface Message {
  role: 'user' | 'assistant';
  content: string;
  display?: ReactNode;
}

export async function continueConversation(history: Message[]) {
  const stream = createStreamableUI();

  const { text, toolResults } = await generateText({
    model: openai('gpt-3.5-turbo'),
    system: 'You are a friendly weather assistant!',
    messages: history,
    tools: {
      showWeather: {
        description: 'Show the weather for a given location.',
        parameters: z.object({
          city: z.string().describe('The city to show the weather for.'),
          unit: z
            .enum(['C', 'F'])
            .describe('The unit to display the temperature in'),
        }),
        execute: async ({ city, unit }) => {
          stream.done(<Weather city={city} unit={unit} />);
          return `Here's the weather for ${city}!`;
        },
      },
    },
  });

  return {
    messages: [
      ...history,
      {
        role: 'assistant' as const,
        content:
          text || toolResults.map(toolResult => toolResult.result).join(),
        display: stream.value,
      },
    ],
  };
}
```

---
title: Tools
description: Learn to allow language models use tools using the AI SDK in your Next.js App Router application
---

# Tools

Certain language models have the ability to use external tools to perform tasks, like using a calculator to solve a math problem or using a browser to search for information. The most common way to share tool information with language models is to share a function definition, along with its description, for it to execute and generate a response based on the output.

In this section, you will learn how to use the `tools` parameter to allow language models to call these functions in your Next.js application.

You will also briefly explore rendering React components as part of a function's output, which can be useful for creating user interfaces that go beyond text.

<IndexCards
  cards={[
    {
      title: 'Call Tool',
      href: '/examples/next-app/tools/call-tool',
    },
    {
      title: 'Call Tools in Parallel',
      href: '/examples/next-app/tools/call-tools-in-parallel',
    },
    {
      title: 'Render Interface during Tool Call',
      href: '/examples/next-app/tools/render-interface-during-tool-call',
    },
  ]}
/>

---
title: AI and UI States
description: Learn to manage AI and UI states using the AI SDK in your Next.js App Router application
---

# AI and UI States

In our previous examples, there seems to be a recurring pattern of having a state for the language model on the server, and a state for the UI on the client. However, it can get tricky to manage these two states separately. For example, if the user types something in the input field, we need to update the UI state, but we also need to send the input to the server to update the AI state.

As a result, the `ai/rsc` library provides a way to seamlessly manage both states together using a context provider that wraps the client application and makes the AI state available to all its children. This way, the client application can access and update the AI state directly keeping the two states in sync.

## Client

Let's use layout to wrap the children components of page with the AI context provider.

```tsx filename='app/layout.tsx'
import { ReactNode } from 'react';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  return (
    <html lang="en">
      <body>
        <AI>{children}</AI>
      </body>
    </html>
  );
}
```

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions, useUIState } from 'ai/rsc';
import { generateId } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState();
  const { continueConversation } = useActions();

  return (
    <div>
      <div>
        {conversation.map((message: ClientMessage) => (
          <div key={message.id}>
            {message.role}: {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              { id: generateId(), role: 'user', display: input },
            ]);

            const message = await continueConversation(input);

            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              message,
            ]);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

```tsx filename='app/actions.tsx'
'use server';

import { getMutableAIState, streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { ReactNode } from 'react';
import { z } from 'zod';
import { generateId } from 'ai';
import { Stock } from '@ai-studio/components/stock';

export interface ServerMessage {
  role: 'user' | 'assistant';
  content: string;
}

export interface ClientMessage {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
}

export async function continueConversation(
  input: string,
): Promise<ClientMessage> {
  'use server';

  const history = getMutableAIState();

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    messages: [...history.get(), { role: 'user', content: input }],
    text: ({ content, done }) => {
      if (done) {
        history.done((messages: ServerMessage[]) => [
          ...messages,
          { role: 'assistant', content },
        ]);
      }

      return <div>{content}</div>;
    },
    tools: {
      showStockInformation: {
        description:
          'Get stock information for symbol for the last numOfMonths months',
        parameters: z.object({
          symbol: z
            .string()
            .describe('The stock symbol to get information for'),
          numOfMonths: z
            .number()
            .describe('The number of months to get historical information for'),
        }),
        generate: async ({ symbol, numOfMonths }) => {
          history.done((messages: ServerMessage[]) => [
            ...messages,
            {
              role: 'assistant',
              content: `Showing stock information for ${symbol}`,
            },
          ]);

          return <Stock symbol={symbol} numOfMonths={numOfMonths} />;
        },
      },
    },
  });

  return {
    id: generateId(),
    role: 'assistant',
    display: result.value,
  };
}
```

Finally, create the AI context provider.

```typescript filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ServerMessage, ClientMessage, continueConversation } from './actions';

export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  initialAIState: [],
  initialUIState: [],
});
```

---
title: Save and Restore States
description: Learn to save and restore AI and UI states with onGetUIState and onSetAIState using the AI SDK in your Next.js App Router application
---

# Save and Restore AI State

Sometimes conversations with language models can get interesting and you might want to save the state of so you can revisit it or continue the conversation later.

`createAI` has an experimental callback function called `onSetAIState` that gets called whenever the AI state changes. You can use this to save the AI state to a file or a database.

## Client

```tsx filename='app/layout.tsx'
import { ServerMessage } from './actions';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  // get chat history from database
  const history: ServerMessage[] = getChat();

  return (
    <html lang="en">
      <body>
        <AI initialAIState={history} initialUIState={[]}>
          {children}
        </AI>
      </body>
    </html>
  );
}
```

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions, useUIState } from 'ai/rsc';
import { generateId } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState();
  const { continueConversation } = useActions();

  return (
    <div>
      <div>
        {conversation.map((message: ClientMessage) => (
          <div key={message.id}>
            {message.role}: {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              { id: generateId(), role: 'user', display: input },
            ]);

            const message = await continueConversation(input);

            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              message,
            ]);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

We will use the callback function to listen to state changes and save the conversation once we receive a `done` event.

```tsx filename='app/actions.tsx'
'use server';

import { getAIState, getMutableAIState, streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { ReactNode } from 'react';
import { z } from 'zod';
import { generateId } from 'ai';
import { Stock } from '@ai-studio/components/stock';

export interface ServerMessage {
  role: 'user' | 'assistant' | 'function';
  content: string;
}

export interface ClientMessage {
  id: string;
  role: 'user' | 'assistant' | 'function';
  display: ReactNode;
}

export async function continueConversation(
  input: string,
): Promise<ClientMessage> {
  'use server';

  const history = getMutableAIState();

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    messages: [...history.get(), { role: 'user', content: input }],
    text: ({ content, done }) => {
      if (done) {
        history.done([
          ...history.get(),
          { role: 'user', content: input },
          { role: 'assistant', content },
        ]);
      }

      return <div>{content}</div>;
    },
    tools: {
      showStockInformation: {
        description:
          'Get stock information for symbol for the last numOfMonths months',
        parameters: z.object({
          symbol: z
            .string()
            .describe('The stock symbol to get information for'),
          numOfMonths: z
            .number()
            .describe('The number of months to get historical information for'),
        }),
        generate: async ({ symbol, numOfMonths }) => {
          history.done([
            ...history.get(),
            {
              role: 'function',
              name: 'showStockInformation',
              content: JSON.stringify({ symbol, numOfMonths }),
            },
          ]);

          return <Stock symbol={symbol} numOfMonths={numOfMonths} />;
        },
      },
    },
  });

  return {
    id: generateId(),
    role: 'assistant',
    display: result.value,
  };
}
```

```ts filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ServerMessage, ClientMessage, continueConversation } from './actions';

export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  onSetAIState: async ({ state, done }) => {
    'use server';

    if (done) {
      saveChat(state);
    }
  },
  onGetUIState: async () => {
    'use server';

    const history: ServerMessage[] = getAIState();

    return history.map(({ role, content }) => ({
      id: generateId(),
      role,
      display:
        role === 'function' ? <Stock {...JSON.parse(content)} /> : content,
    }));
  },
});
```

---
title: State Management
description: Learn to manage AI and UI states using the AI SDK in your Next.js App Router application
---

# State Management

In this section you will learn how to manage states between the language model on the server, and the user interface on the client. You will learn how to use `createAI` to initiate a context provider that can be used to manage these two states.

You will also learn about the function callbacks that `createAI` supports, which can be used to save and restore the AI and UI state of your application.

---
title: Route Components
description: Learn to route components using the AI SDK in your Next.js App Router application
---

# Route Components

We've now seen how a language model can call a function and render a component based on a conversation with the user.

When we define multiple functions in [`tools`](/docs/reference/ai-sdk-core/generate-text#tools), it is possible for the model to reason out the right functions to call based on whatever the user's intent is. This means that you can write a bunch of functions without the burden of implementing complex routing logic to run them.

## Client

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions, useUIState } from 'ai/rsc';
import { generateId } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState();
  const { continueConversation } = useActions();

  return (
    <div>
      <div>
        {conversation.map((message: ClientMessage) => (
          <div key={message.id}>
            {message.role}: {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              { id: generateId(), role: 'user', display: input },
            ]);

            const message = await continueConversation(input);

            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              message,
            ]);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

```tsx filename='components/stock.tsx'
export async function Stock({ symbol, numOfMonths }) {
  const data = await fetch(
    `https://api.example.com/stock/${symbol}/${numOfMonths}`,
  );

  return (
    <div>
      <div>{symbol}</div>

      <div>
        {data.timeline.map(data => (
          <div>
            <div>{data.date}</div>
            <div>{data.value}</div>
          </div>
        ))}
      </div>
    </div>
  );
}
```

```tsx filename='components/flight.tsx'
export async function Flight({ flightNumber }) {
  const data = await fetch(`https://api.example.com/flight/${flightNumber}`);

  return (
    <div>
      <div>{flightNumber}</div>
      <div>{data.status}</div>
      <div>{data.source}</div>
      <div>{data.destination}</div>
    </div>
  );
}
```

## Server

```tsx filename='app/actions.tsx'
'use server';

import { getMutableAIState, streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { ReactNode } from 'react';
import { z } from 'zod';
import { generateId } from 'ai';
import { Stock } from '@/components/stock';
import { Flight } from '@/components/flight';

export interface ServerMessage {
  role: 'user' | 'assistant';
  content: string;
}

export interface ClientMessage {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
}

export async function continueConversation(
  input: string,
): Promise<ClientMessage> {
  'use server';

  const history = getMutableAIState();

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    messages: [...history.get(), { role: 'user', content: input }],
    text: ({ content, done }) => {
      if (done) {
        history.done((messages: ServerMessage[]) => [
          ...messages,
          { role: 'assistant', content },
        ]);
      }

      return <div>{content}</div>;
    },
    tools: {
      showStockInformation: {
        description:
          'Get stock information for symbol for the last numOfMonths months',
        parameters: z.object({
          symbol: z
            .string()
            .describe('The stock symbol to get information for'),
          numOfMonths: z
            .number()
            .describe('The number of months to get historical information for'),
        }),
        generate: async ({ symbol, numOfMonths }) => {
          history.done((messages: ServerMessage[]) => [
            ...messages,
            {
              role: 'assistant',
              content: `Showing stock information for ${symbol}`,
            },
          ]);

          return <Stock symbol={symbol} numOfMonths={numOfMonths} />;
        },
      },
      showFlightStatus: {
        description: 'Get the status of a flight',
        parameters: z.object({
          flightNumber: z
            .string()
            .describe('The flight number to get status for'),
        }),
        generate: async ({ flightNumber }) => {
          history.done((messages: ServerMessage[]) => [
            ...messages,
            {
              role: 'assistant',
              content: `Showing flight status for ${flightNumber}`,
            },
          ]);

          return <Flight flightNumber={flightNumber} />;
        },
      },
    },
  });

  return {
    id: generateId(),
    role: 'assistant',
    display: result.value,
  };
}
```

```typescript filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ServerMessage, ClientMessage, continueConversation } from './actions';

export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  initialAIState: [],
  initialUIState: [],
});
```

---
title: Stream Component Updates
description: Learn to stream progressive components updates using the AI SDK in your Next.js App Router application
---

# Stream Component Updates

If you haven't noticed already, in our previous example we've been streaming react components from the server to the client. By streaming the components, we open up the possibility to update these components based on state changes that occur in the server.

## Client

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions, useUIState } from 'ai/rsc';
import { generateId } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState();
  const { continueConversation } = useActions();

  return (
    <div>
      <div>
        {conversation.map((message: ClientMessage) => (
          <div key={message.id}>
            {message.role}: {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              { id: generateId(), role: 'user', display: input },
            ]);

            const message = await continueConversation(input);

            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              message,
            ]);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

```tsx filename='app/actions.tsx'
'use server';

import { getMutableAIState, streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { ReactNode } from 'react';
import { z } from 'zod';
import { generateId } from 'ai';

export interface ServerMessage {
  role: 'user' | 'assistant';
  content: string;
}

export interface ClientMessage {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
}

export async function continueConversation(
  input: string,
): Promise<ClientMessage> {
  'use server';

  const history = getMutableAIState();

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    messages: [...history.get(), { role: 'user', content: input }],
    text: ({ content, done }) => {
      if (done) {
        history.done((messages: ServerMessage[]) => [
          ...messages,
          { role: 'assistant', content },
        ]);
      }

      return <div>{content}</div>;
    },
    tools: {
      deploy: {
        description: 'Deploy repository to vercel',
        parameters: z.object({
          repositoryName: z
            .string()
            .describe('The name of the repository, example: vercel/ai-chatbot'),
        }),
        generate: async function* ({ repositoryName }) {
          yield <div>Cloning repository {repositoryName}...</div>; // [!code highlight:5]
          await new Promise(resolve => setTimeout(resolve, 3000));
          yield <div>Building repository {repositoryName}...</div>;
          await new Promise(resolve => setTimeout(resolve, 2000));
          return <div>{repositoryName} deployed!</div>;
        },
      },
    },
  });

  return {
    id: generateId(),
    role: 'assistant',
    display: result.value,
  };
}
```

```typescript filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ServerMessage, ClientMessage, continueConversation } from './actions';

export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  initialAIState: [],
  initialUIState: [],
});
```

---
title: Recording Token Usage
description: Examples of how to record token usage when streaming user interfaces.
---

# Recording Token Usage

When you're streaming structured data with [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui),
you may want to record the token usage for billing purposes.

## `onFinish` Callback

You can use the `onFinish` callback to record token usage.
It is called when the stream is finished.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions, useUIState } from 'ai/rsc';
import { generateId } from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState();
  const { continueConversation } = useActions();

  return (
    <div>
      <div>
        {conversation.map((message: ClientMessage) => (
          <div key={message.id}>
            {message.role}: {message.display}
          </div>
        ))}
      </div>

      <div>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button
          onClick={async () => {
            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              { id: generateId(), role: 'user', display: input },
            ]);

            const message = await continueConversation(input);

            setConversation((currentConversation: ClientMessage[]) => [
              ...currentConversation,
              message,
            ]);
          }}
        >
          Send Message
        </button>
      </div>
    </div>
  );
}
```

## Server

```tsx filename='app/actions.tsx' highlight={"57-63"}
'use server';

import { createAI, getMutableAIState, streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { ReactNode } from 'react';
import { z } from 'zod';
import { generateId } from 'ai';

export interface ServerMessage {
  role: 'user' | 'assistant';
  content: string;
}

export interface ClientMessage {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
}

export async function continueConversation(
  input: string,
): Promise<ClientMessage> {
  'use server';

  const history = getMutableAIState();

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    messages: [...history.get(), { role: 'user', content: input }],
    text: ({ content, done }) => {
      if (done) {
        history.done((messages: ServerMessage[]) => [
          ...messages,
          { role: 'assistant', content },
        ]);
      }

      return <div>{content}</div>;
    },
    tools: {
      deploy: {
        description: 'Deploy repository to vercel',
        parameters: z.object({
          repositoryName: z
            .string()
            .describe('The name of the repository, example: vercel/ai-chatbot'),
        }),
        generate: async function* ({ repositoryName }) {
          yield <div>Cloning repository {repositoryName}...</div>; // [!code highlight:5]
          await new Promise(resolve => setTimeout(resolve, 3000));
          yield <div>Building repository {repositoryName}...</div>;
          await new Promise(resolve => setTimeout(resolve, 2000));
          return <div>{repositoryName} deployed!</div>;
        },
      },
    },
    onFinish: ({ usage }) => {
      const { promptTokens, completionTokens, totalTokens } = usage;
      // your own logic, e.g. for saving the chat history or recording usage
      console.log('Prompt tokens:', promptTokens);
      console.log('Completion tokens:', completionTokens);
      console.log('Total tokens:', totalTokens);
    },
  });

  return {
    id: generateId(),
    role: 'assistant',
    display: result.value,
  };
}
```

```typescript filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { ServerMessage, ClientMessage, continueConversation } from './actions';

export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  initialAIState: [],
  initialUIState: [],
});
```

---
title: Generative User Interface
description: Learn to build generative user interfaces using the AI SDK in your Next.js App Router application
---

# Generative User Interface

In this section you will learn to use the `streamUI` function to stream generative user interfaces to the client based on the response from the language model.

---
title: Stream Assistant Responses
description: Learn to stream assistant responses using the AI SDK in your Next.js App Router application
---

# Stream Assistant Responses

In this example, you'll learn how to stream responses from OpenAI's [Assistant API](https://platform.openai.com/docs/assistants/overview) using `ai/rsc`.

## Client

In your client component, you will create a simple chat interface that allows users to send messages to the assistant and receive responses. The assistant's responses will be streamed in two parts: the status of the current run and the text content of the messages.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage } from './actions';
import { useActions } from 'ai/rsc';

export default function Home() {
  const [input, setInput] = useState('');
  const [messages, setMessages] = useState<ClientMessage[]>([]);
  const { submitMessage } = useActions();

  const handleSubmission = async () => {
    setMessages(currentMessages => [
      ...currentMessages,
      {
        id: '123',
        status: 'user.message.created',
        text: input,
        gui: null,
      },
    ]);

    const response = await submitMessage(input);
    setMessages(currentMessages => [...currentMessages, response]);
    setInput('');
  };

  return (
    <div className="flex flex-col-reverse">
      <div className="flex flex-row gap-2 p-2 bg-zinc-100 w-full">
        <input
          className="bg-zinc-100 w-full p-2 outline-none"
          value={input}
          onChange={event => setInput(event.target.value)}
          placeholder="Ask a question"
          onKeyDown={event => {
            if (event.key === 'Enter') {
              handleSubmission();
            }
          }}
        />
        <button
          className="p-2 bg-zinc-900 text-zinc-100 rounded-md"
          onClick={handleSubmission}
        >
          Send
        </button>
      </div>

      <div className="flex flex-col h-[calc(100dvh-56px)] overflow-y-scroll">
        <div>
          {messages.map(message => (
            <div key={message.id} className="flex flex-col gap-1 border-b p-2">
              <div className="flex flex-row justify-between">
                <div className="text-sm text-zinc-500">{message.status}</div>
              </div>
              <div>{message.text}</div>
            </div>
          ))}
        </div>
      </div>
    </div>
  );
}
```

```tsx filename='app/message.tsx'
'use client';

import { StreamableValue, useStreamableValue } from 'ai/rsc';

export function Message({ textStream }: { textStream: StreamableValue }) {
  const [text] = useStreamableValue(textStream);

  return <div>{text}</div>;
}
```

## Server

In your server action, you will create a function called `submitMessage` that adds the user's message to the thread. The function will create a new thread if one does not exist and add the user's message to the thread. If a thread already exists, the function will add the user's message to the existing thread. The function will then create a run and stream the assistant's response to the client. Furthermore, the run queue is used to manage multiple runs in the same thread during the lifetime of the server action.

```tsx filename='app/actions.tsx'
'use server';

import { generateId } from 'ai';
import { createStreamableUI, createStreamableValue } from 'ai/rsc';
import { OpenAI } from 'openai';
import { ReactNode } from 'react';
import { Message } from './message';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export interface ClientMessage {
  id: string;
  status: ReactNode;
  text: ReactNode;
}

const ASSISTANT_ID = 'asst_xxxx';
let THREAD_ID = '';
let RUN_ID = '';

export async function submitMessage(question: string): Promise<ClientMessage> {
  const statusUIStream = createStreamableUI('thread.init');

  const textStream = createStreamableValue('');
  const textUIStream = createStreamableUI(
    <Message textStream={textStream.value} />,
  );

  const runQueue = [];

  (async () => {
    if (THREAD_ID) {
      await openai.beta.threads.messages.create(THREAD_ID, {
        role: 'user',
        content: question,
      });

      const run = await openai.beta.threads.runs.create(THREAD_ID, {
        assistant_id: ASSISTANT_ID,
        stream: true,
      });

      runQueue.push({ id: generateId(), run });
    } else {
      const run = await openai.beta.threads.createAndRun({
        assistant_id: ASSISTANT_ID,
        stream: true,
        thread: {
          messages: [{ role: 'user', content: question }],
        },
      });

      runQueue.push({ id: generateId(), run });
    }

    while (runQueue.length > 0) {
      const latestRun = runQueue.shift();

      if (latestRun) {
        for await (const delta of latestRun.run) {
          const { data, event } = delta;

          statusUIStream.update(event);

          if (event === 'thread.created') {
            THREAD_ID = data.id;
          } else if (event === 'thread.run.created') {
            RUN_ID = data.id;
          } else if (event === 'thread.message.delta') {
            data.delta.content?.map(part => {
              if (part.type === 'text') {
                if (part.text) {
                  textStream.append(part.text.value as string);
                }
              }
            });
          } else if (event === 'thread.run.failed') {
            console.error(data);
          }
        }
      }
    }

    statusUIStream.done();
    textStream.done();
  })();

  return {
    id: generateId(),
    status: statusUIStream.value,
    text: textUIStream.value,
  };
}
```

```tsx filename="app/ai.ts"
import { createAI } from 'ai/rsc';
import { submitMessage } from './actions';

export const AI = createAI({
  actions: {
    submitMessage,
  },
  initialAIState: [],
  initialUIState: [],
});
```

And finally, make sure to update your layout component to wrap the children with the `AI` component.

```tsx filename="app/layout.tsx"
import { ReactNode } from 'react';
import { AI } from './ai';

export default function Layout({ children }: { children: ReactNode }) {
  return <AI>{children}</AI>;
}
```

---
title: Stream Assistant Responses with Tools
description: Learn to stream assistant responses with tools using the AI SDK in your Next.js App Router application
---

# Stream Assistant Responses

In this example, you'll learn how to stream responses along with tool calls from OpenAI's [Assistant API](https://platform.openai.com/docs/assistants/overview) using `ai/rsc`.

## Client

In your client component, you will create a simple chat interface that allows users to send messages to the assistant and receive responses. The assistant's responses will be streamed in two parts: the status of the current run and the text content of the messages.

```tsx filename='app/page.tsx'
'use client';

import { useState } from 'react';
import { ClientMessage, submitMessage } from './actions';
import { useActions } from 'ai/rsc';

export default function Home() {
  const [input, setInput] = useState('');
  const [messages, setMessages] = useState<ClientMessage[]>([]);
  const { submitMessage } = useActions();

  const handleSubmission = async () => {
    setMessages(currentMessages => [
      ...currentMessages,
      {
        id: '123',
        status: 'user.message.created',
        text: input,
        gui: null,
      },
    ]);

    const response = await submitMessage(input);
    setMessages(currentMessages => [...currentMessages, response]);
    setInput('');
  };

  return (
    <div className="flex flex-col-reverse">
      <div className="flex flex-row gap-2 p-2 bg-zinc-100 w-full">
        <input
          className="bg-zinc-100 w-full p-2 outline-none"
          value={input}
          onChange={event => setInput(event.target.value)}
          placeholder="Ask a question"
          onKeyDown={event => {
            if (event.key === 'Enter') {
              handleSubmission();
            }
          }}
        />
        <button
          className="p-2 bg-zinc-900 text-zinc-100 rounded-md"
          onClick={handleSubmission}
        >
          Send
        </button>
      </div>

      <div className="flex flex-col h-[calc(100dvh-56px)] overflow-y-scroll">
        <div>
          {messages.map(message => (
            <div key={message.id} className="flex flex-col gap-1 border-b p-2">
              <div className="flex flex-row justify-between">
                <div className="text-sm text-zinc-500">{message.status}</div>
              </div>
              <div className="flex flex-col gap-2">{message.gui}</div>
              <div>{message.text}</div>
            </div>
          ))}
        </div>
      </div>
    </div>
  );
}
```

```tsx filename='app/message.tsx'
'use client';

import { StreamableValue, useStreamableValue } from 'ai/rsc';

export function Message({ textStream }: { textStream: StreamableValue }) {
  const [text] = useStreamableValue(textStream);

  return <div>{text}</div>;
}
```

## Server

In your server action, you will create a function called `submitMessage` that adds the user's message to the thread. The function will create a new thread if one does not exist and add the user's message to the thread. If a thread already exists, the function will add the user's message to the existing thread. The function will then create a run and stream the assistant's response to the client. Furthermore, the run queue is used to manage multiple runs in the same thread during the lifetime of the server action.

In case the assistant requires a tool call, the server action will handle the tool call and return the output to the assistant. In this example, the assistant requires a tool call to search for emails. The server action will search for emails based on the `query` and `has_attachments` parameters and return the output to the both the assistant and the client.

```tsx filename='app/actions.tsx'
'use server';

import { generateId } from 'ai';
import { createStreamableUI, createStreamableValue } from 'ai/rsc';
import { OpenAI } from 'openai';
import { ReactNode } from 'react';
import { searchEmails } from './function';
import { Message } from './message';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export interface ClientMessage {
  id: string;
  status: ReactNode;
  text: ReactNode;
  gui: ReactNode;
}

const ASSISTANT_ID = 'asst_xxxx';
let THREAD_ID = '';
let RUN_ID = '';

export async function submitMessage(question: string): Promise<ClientMessage> {
  const status = createStreamableUI('thread.init');
  const textStream = createStreamableValue('');
  const textUIStream = createStreamableUI(
    <Message textStream={textStream.value} />,
  );
  const gui = createStreamableUI();

  const runQueue = [];

  (async () => {
    if (THREAD_ID) {
      await openai.beta.threads.messages.create(THREAD_ID, {
        role: 'user',
        content: question,
      });

      const run = await openai.beta.threads.runs.create(THREAD_ID, {
        assistant_id: ASSISTANT_ID,
        stream: true,
      });

      runQueue.push({ id: generateId(), run });
    } else {
      const run = await openai.beta.threads.createAndRun({
        assistant_id: ASSISTANT_ID,
        stream: true,
        thread: {
          messages: [{ role: 'user', content: question }],
        },
      });

      runQueue.push({ id: generateId(), run });
    }

    while (runQueue.length > 0) {
      const latestRun = runQueue.shift();

      if (latestRun) {
        for await (const delta of latestRun.run) {
          const { data, event } = delta;

          status.update(event);

          if (event === 'thread.created') {
            THREAD_ID = data.id;
          } else if (event === 'thread.run.created') {
            RUN_ID = data.id;
          } else if (event === 'thread.message.delta') {
            data.delta.content?.map((part: any) => {
              if (part.type === 'text') {
                if (part.text) {
                  textStream.append(part.text.value);
                }
              }
            });
          } else if (event === 'thread.run.requires_action') {
            if (data.required_action) {
              if (data.required_action.type === 'submit_tool_outputs') {
                const { tool_calls } = data.required_action.submit_tool_outputs;
                const tool_outputs = [];

                for (const tool_call of tool_calls) {
                  const { id: toolCallId, function: fn } = tool_call;
                  const { name, arguments: args } = fn;

                  if (name === 'search_emails') {
                    const { query, has_attachments } = JSON.parse(args);

                    gui.append(
                      <div className="flex flex-row gap-2 items-center">
                        <div>
                          Searching for emails: {query}, has_attachments:
                          {has_attachments ? 'true' : 'false'}
                        </div>
                      </div>,
                    );

                    await new Promise(resolve => setTimeout(resolve, 2000));

                    const fakeEmails = searchEmails({ query, has_attachments });

                    gui.append(
                      <div className="flex flex-col gap-2">
                        {fakeEmails.map(email => (
                          <div
                            key={email.id}
                            className="p-2 bg-zinc-100 rounded-md flex flex-row gap-2 items-center justify-between"
                          >
                            <div className="flex flex-row gap-2 items-center">
                              <div>{email.subject}</div>
                            </div>
                            <div className="text-zinc-500">{email.date}</div>
                          </div>
                        ))}
                      </div>,
                    );

                    tool_outputs.push({
                      tool_call_id: toolCallId,
                      output: JSON.stringify(fakeEmails),
                    });
                  }
                }

                const nextRun: any =
                  await openai.beta.threads.runs.submitToolOutputs(
                    THREAD_ID,
                    RUN_ID,
                    {
                      tool_outputs,
                      stream: true,
                    },
                  );

                runQueue.push({ id: generateId(), run: nextRun });
              }
            }
          } else if (event === 'thread.run.failed') {
            console.log(data);
          }
        }
      }
    }

    status.done();
    textUIStream.done();
    gui.done();
  })();

  return {
    id: generateId(),
    status: status.value,
    text: textUIStream.value,
    gui: gui.value,
  };
}
```

```typescript filename='app/ai.ts'
import { createAI } from 'ai/rsc';
import { submitMessage } from './actions';

export const AI = createAI({
  actions: {
    submitMessage,
  },
  initialAIState: [],
  initialUIState: [],
});
```

And finally, make sure to update your layout component to wrap the children with the `AI` component.

```tsx filename="app/layout.tsx"
import { ReactNode } from 'react';
import { AI } from './ai';

export default function Layout({ children }: { children: ReactNode }) {
  return <AI>{children}</AI>;
}
```

---
title: OpenAI Assistants
description: Learn to use the OpenAI Assistant API using the AI SDK in your Next.js App Router application
---

# Assistants

In this section, you will learn to use OpenAI's Assistant API along with `ai/rsc` functions.

---

title: RSC (experimental)
description: Explore examples of how to use the AI SDK RSC in a Next.js application.
---`

RSC (experimental)

<Note type="warning">
  AI SDK RSC is currently experimental. We recommend using [AI SDK
  UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
  RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).
</Note>

The AI SDK provides tools for frameworks that support React Server Components (RSC). The following examples demonstrate how to achieve different use cases while integrating AI into your RSC powered applications.

---
title: Node.js HTTP Server
description: Example of using the AI SDK in a Node.js HTTP server.
---

# Node.js HTTP Server

You can use the AI SDK in a Node.js HTTP server to generate text and stream it to the client.

## Examples

The examples start a simple HTTP server that listens on port 8080. You can e.g. test it using `curl`:

```bash
curl -X POST http://localhost:8080
```

<Note>
  The examples use the OpenAI `gpt-4o` model. Ensure that the OpenAI API key is
  set in the `OPENAI_API_KEY` environment variable.
</Note>

**Full example**: [github.com/vercel/ai/examples/node-http-server](https://github.com/vercel/ai/tree/main/examples/node-http-server)

### Data Stream

You can use the `pipeDataStreamToResponse` method to pipe the stream data to the server response.

```ts filename='index.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { createServer } from 'http';

createServer(async (req, res) => {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  result.pipeDataStreamToResponse(res);
}).listen(8080);
```

### Data Stream With Stream Data

`pipeDataStreamToResponse` can be used with `StreamData` to send additional data to the client.

```ts filename='index.ts' highlight="6-7,12-15,18"
import { openai } from '@ai-sdk/openai';
import { StreamData, streamText } from 'ai';
import { createServer } from 'http';

createServer(async (req, res) => {
  const data = new StreamData();
  data.append('initialized call');

  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
    onFinish() {
      data.append('call completed');
      data.close();
    },
  });

  result.pipeDataStreamToResponse(res, { data });
}).listen(8080);
```

### Text Stream

You can send a text stream to the client using `pipeTextStreamToResponse`.

```ts filename='index.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { createServer } from 'http';

createServer(async (req, res) => {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  result.pipeTextStreamToResponse(res);
}).listen(8080);
```

---
title: Express
description: Example of using the AI SDK in an Express server.
---

# Express

You can use the AI SDK in an [Express](https://expressjs.com/) server to generate and stream text and objects to the client.

## Examples

The examples start a simple HTTP server that listens on port 8080. You can e.g. test it using `curl`:

```bash
curl -X POST http://localhost:8080
```

<Note>
  The examples use the OpenAI `gpt-4o` model. Ensure that the OpenAI API key is
  set in the `OPENAI_API_KEY` environment variable.
</Note>

**Full example**: [github.com/vercel/ai/examples/express](https://github.com/vercel/ai/tree/main/examples/express)

### Data Stream

You can use the `pipeDataStreamToResponse` method to pipe the stream data to the server response.

```ts filename='index.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import express, { Request, Response } from 'express';

const app = express();

app.post('/', async (req: Request, res: Response) => {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  result.pipeDataStreamToResponse(res);
});

app.listen(8080, () => {
  console.log(`Example app listening on port ${8080}`);
});
```

### Data Stream With Stream Data

`pipeDataStreamToResponse` can be used with `StreamData` to send additional data to the client.

```ts filename='index.ts' highlight="8-10,15-18,21"
import { openai } from '@ai-sdk/openai';
import { StreamData, streamText } from 'ai';
import express, { Request, Response } from 'express';

const app = express();

app.post('/', async (req: Request, res: Response) => {
  // use stream data (optional):
  const data = new StreamData();
  data.append('initialized call');

  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
    onFinish() {
      data.append('call completed');
      data.close();
    },
  });

  result.pipeDataStreamToResponse(res, { data });
});

app.listen(8080, () => {
  console.log(`Example app listening on port ${8080}`);
});
```

### Text Stream

You can send a text stream to the client using `pipeTextStreamToResponse`.

```ts filename='index.ts' highlight="13"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import express, { Request, Response } from 'express';

const app = express();

app.post('/', async (req: Request, res: Response) => {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  result.pipeTextStreamToResponse(res);
});

app.listen(8080, () => {
  console.log(`Example app listening on port ${8080}`);
});
```

---
title: Hono
description: Example of using the AI SDK in a Hono server.
---

# Hono

You can use the AI SDK in a [Hono](https://hono.dev/) server to generate and stream text and objects to the client.

## Examples

The examples start a simple HTTP server that listens on port 8080. You can e.g. test it using `curl`:

```bash
curl -X POST http://localhost:8080
```

<Note>
  The examples use the OpenAI `gpt-4o` model. Ensure that the OpenAI API key is
  set in the `OPENAI_API_KEY` environment variable.
</Note>

**Full example**: [github.com/vercel/ai/examples/hono](https://github.com/vercel/ai/tree/main/examples/hono)

### Data Stream

You can use the `toDataStream` method to get a data stream from the result and then pipe it to the response.

```ts filename='index.ts'
import { openai } from '@ai-sdk/openai';
import { serve } from '@hono/node-server';
import { streamText } from 'ai';
import { Hono } from 'hono';
import { stream } from 'hono/streaming';

const app = new Hono();

app.post('/', async c =>
  stream(c, async stream => {
    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
    });

    // Mark the response as a v1 data stream:
    c.header('X-Vercel-AI-Data-Stream', 'v1');
    c.header('Content-Type', 'text/plain; charset=utf-8');

    await stream.pipe(result.toDataStream());
  }),
);

serve({ fetch: app.fetch, port: 8080 });
```

### Data Stream With Stream Data

`toDataStream` can be used with `StreamData` to send additional data to the client.

```ts filename='index.ts' highlight="11-13,18-21,28"
import { openai } from '@ai-sdk/openai';
import { serve } from '@hono/node-server';
import { StreamData, streamText } from 'ai';
import { Hono } from 'hono';
import { stream } from 'hono/streaming';

const app = new Hono();

app.post('/', async c =>
  stream(c, async stream => {
    // use stream data (optional):
    const data = new StreamData();
    data.append('initialized call');

    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
      onFinish() {
        data.append('call completed');
        data.close();
      },
    });

    // Mark the response as a v1 data stream:
    c.header('X-Vercel-AI-Data-Stream', 'v1');
    c.header('Content-Type', 'text/plain; charset=utf-8');

    await stream.pipe(result.toDataStream({ data }));
  }),
);

serve({ fetch: app.fetch, port: 8080 });
```

### Text Stream

You can use the `textStream` property to get a text stream from the result and then pipe it to the response.

```ts filename='index.ts' highlight="18"
import { openai } from '@ai-sdk/openai';
import { serve } from '@hono/node-server';
import { streamText } from 'ai';
import { Hono } from 'hono';
import { stream } from 'hono/streaming';

const app = new Hono();

app.post('/', async c =>
  stream(c, async stream => {
    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
    });

    c.header('Content-Type', 'text/plain; charset=utf-8');

    await stream.pipe(result.textStream);
  }),
);

serve({ fetch: app.fetch, port: 8080 });
```

---
title: Fastify
description: Example of using the AI SDK in a Fastify server.
---

# Fastify

You can use the AI SDK in a [Fastify](https://fastify.dev/) server to generate and stream text and objects to the client.

## Examples

The examples start a simple HTTP server that listens on port 8080. You can e.g. test it using `curl`:

```bash
curl -X POST http://localhost:8080
```

<Note>
  The examples use the OpenAI `gpt-4o` model. Ensure that the OpenAI API key is
  set in the `OPENAI_API_KEY` environment variable.
</Note>

**Full example**: [github.com/vercel/ai/examples/fastify](https://github.com/vercel/ai/tree/main/examples/fastify)

### Data Stream

You can use the `toDataStream` method to get a data stream from the result and then pipe it to the response.

```ts filename='index.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post('/', async function (request, reply) {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  // Mark the response as a v1 data stream:
  reply.header('X-Vercel-AI-Data-Stream', 'v1');
  reply.header('Content-Type', 'text/plain; charset=utf-8');

  return reply.send(result.toDataStream({ data }));
});

fastify.listen({ port: 8080 });
```

### Data Stream With Stream Data

`toDataStream` can be used with `StreamData` to send additional data to the client.

```ts filename='index.ts' highlight="8-10,15-18,25"
import { openai } from '@ai-sdk/openai';
import { StreamData, streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post('/', async function (request, reply) {
  // use stream data (optional):
  const data = new StreamData();
  data.append('initialized call');

  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
    onFinish() {
      data.append('call completed');
      data.close();
    },
  });

  // Mark the response as a v1 data stream:
  reply.header('X-Vercel-AI-Data-Stream', 'v1');
  reply.header('Content-Type', 'text/plain; charset=utf-8');

  return reply.send(result.toDataStream({ data }));
});

fastify.listen({ port: 8080 });
```

### Text Stream

You can use the `textStream` property to get a text stream from the result and then pipe it to the response.

```ts filename='index.ts' highlight="15"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import Fastify from 'fastify';

const fastify = Fastify({ logger: true });

fastify.post('/', async function (request, reply) {
  const result = streamText({
    model: openai('gpt-4o'),
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  reply.header('Content-Type', 'text/plain; charset=utf-8');

  return reply.send(result.textStream);
});

fastify.listen({ port: 8080 });
```

---
title: Nest.js
description: Example of using the AI SDK in a Nest.js server.
---

# Nest.js

You can use the AI SDK in a [Nest.js](https://nestjs.com/) server to generate and stream text and objects to the client.

## Examples

The examples show how to implement a Nest.js controller that uses the AI SDK to stream text and objects to the client.

**Full example**: [github.com/vercel/ai/examples/nest](https://github.com/vercel/ai/tree/main/examples/nest)

### Data Stream

You can use the `pipeDataStreamToResponse` method to get a data stream from the result and then pipe it to the response.

```ts filename='app.controller.ts'
import { Controller, Post, Res } from '@nestjs/common';
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { Response } from 'express';

@Controller()
export class AppController {
  @Post()
  async example(@Res() res: Response) {
    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
    });

    result.pipeDataStreamToResponse(res);
  }
}
```

### Data Stream With Stream Data

`pipeDataStreamToResponse` can be used with `StreamData` to send additional data to the client.

```ts filename='app.controller.ts' highlight="10-11,16-19,22"
import { Controller, Post, Res } from '@nestjs/common';
import { openai } from '@ai-sdk/openai';
import { StreamData, streamText } from 'ai';
import { Response } from 'express';

@Controller()
export class AppController {
  @Post()
  async example(@Res() res: Response) {
    const data = new StreamData();
    data.append('initialized call');

    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
      onFinish() {
        data.append('call completed');
        data.close();
      },
    });

    result.pipeDataStreamToResponse(res, { data });
  }
}
```

### Text Stream

You can use the `pipeTextStreamToResponse` method to get a text stream from the result and then pipe it to the response.

```ts filename='app.controller.ts' highlight="15"
import { Controller, Post, Res } from '@nestjs/common';
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { Response } from 'express';

@Controller()
export class AppController {
  @Post()
  async example(@Res() res: Response) {
    const result = streamText({
      model: openai('gpt-4o'),
      prompt: 'Invent a new holiday and describe its traditions.',
    });

    result.pipeTextStreamToResponse(res);
  }
}
```

---
title: API Servers
description: Explore examples of using the AI SDK in different JavaScript backend frameworks.
---

# API Server Examples

You can use the AI SDK in any JavaScript API server to generate text and stream it to the client.

<IndexCards
  cards={[
    {
      title: 'Node.js HTTP Server',
      description: 'Send AI responses from a Node.js HTTP server.',
      href: '/examples/api-servers/node-js-http-server',
    },
    {
      title: 'Express',
      description: 'Send AI responses from an Express server.',
      href: '/examples/api-servers/express',
    },
    {
      title: 'Hono',
      description: 'Send AI responses from a Hono server.',
      href: '/examples/api-servers/hono',
    },
    {
      title: 'Fastify',
      description: 'Send AI responses from a Fastify server.',
      href: '/examples/api-servers/fastify',
    },
    {
      title: 'Nest.js',
      description: 'Send AI responses from a Nest.js server.',
      href: '/examples/api-servers/nest',
    },
  ]}
/>

---
title: Intercepting Fetch Requests
description: Example of intercepting fetch requests with a provider.
---

# Intercepting Fetch Requests

Many providers support setting a custom `fetch` function using the `fetch` argument in their factory function.

A custom `fetch` function can be used to intercept and modify requests before they are sent to the provider's API,
and to intercept and modify responses before they are returned to the caller.

Use cases for intercepting requests include:

- Logging requests and responses
- Adding authentication headers
- Modifying request bodies
- Caching responses
- Using a custom HTTP client

## Example

```ts file='index.ts' highlight="5-13"
import { generateText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  // example fetch wrapper that logs the input to the API call:
  fetch: async (url, options) => {
    console.log('URL', url);
    console.log('Headers', JSON.stringify(options!.headers, null, 2));
    console.log(
      `Body ${JSON.stringify(JSON.parse(options!.body! as string), null, 2)}`,
    );
    return await fetch(url, options);
  },
});

const { text } = await generateText({
  model: openai('gpt-3.5-turbo'),
  prompt: 'Why is the sky blue?',
});
```

---
title: Providers
description: Examples of using provider functionality in the AI SDK.
---

# Provider Examples

The following section will guide you through the various capabilities of the AI SDKs using minimal code examples.

<IndexCards
  cards={[
    {
      title: 'Intercepting Fetch Requests',
      description: 'Example of intercepting fetch requests with a provider.',
      href: '/examples/providers/intercepting-fetch-requests',
    },
  ]}
/>

---
title: Explore Examples
description: Explore examples of how to use the AI SDK in different frameworks.
---

<div style={{ marginBottom: 120, transform: 'scale(1.2)' }}>
  <MarketingFrameworkCircles />
</div>

# Explore Examples of AI Applications Across Frameworks

The AI SDK is designed to work with different frameworks and environments that run using Typescript or Javascript.

Whether you're integrating AI capabilities in a new framework, or want to get started with your existing framework of choice, you can explore examples of how to use the AI SDK for different use cases based on the framework you choose from the sidebar.

<ExampleCards />

---
title: OpenAI
description: Learn how to use the OpenAI provider for the AI SDK.
---

# OpenAI Provider

The [OpenAI](https://openai.com/) provider contains language model support for the OpenAI chat and completion APIs
and embedding model support for the OpenAI embeddings API.

## Setup

The OpenAI provider is available in the `@ai-sdk/openai` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `openai` from `@ai-sdk/openai`:

```ts
import { openai } from '@ai-sdk/openai';
```

If you need a customized setup, you can import `createOpenAI` from `@ai-sdk/openai` and create a provider instance with your settings:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  // custom settings, e.g.
  compatibility: 'strict', // strict mode, enable when using the OpenAI API
});
```

You can use the following optional settings to customize the OpenAI provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.openai.com/v1`.

- **apiKey** _string_

  API key that is being sent using the `Authorization` header.
  It defaults to the `OPENAI_API_KEY` environment variable.

- **name** _string_

  The provider name. You can set this when using OpenAI compatible providers
  to change the model provider property. Defaults to `openai`.

- **organization** _string_

  OpenAI Organization.

- **project** _string_

  OpenAI project.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

- **compatibility** _"strict" | "compatible"_

  OpenAI compatibility mode. Should be set to `strict` when using the OpenAI API,
  and `compatible` when using 3rd party providers. In `compatible` mode, newer
  information such as `streamOptions` are not being sent, resulting in `NaN`
  token usage. Defaults to 'compatible'.

## Language Models

The OpenAI provider instance is a function that you can invoke to create a language model:

```ts
const model = openai('gpt-4-turbo');
```

It automatically selects the correct API based on the model id.
You can also pass additional settings in the second argument:

```ts
const model = openai('gpt-4-turbo', {
  // additional settings
});
```

The available options depend on the API that's automatically chosen for the model (see below).
If you want to explicitly select a specific model API, you can use `.chat` or `.completion`.

### Example

You can use OpenAI language models to generate text with the `generateText` function:

```ts
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const { text } = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

OpenAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### Chat Models

You can create models that call the [OpenAI chat API](https://platform.openai.com/docs/api-reference/chat) using the `.chat()` factory method.
The first argument is the model id, e.g. `gpt-4`.
The OpenAI chat models support tool calls and some have multi-modal capabilities.

```ts
const model = openai.chat('gpt-3.5-turbo');
```

OpenAI chat models support also some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = openai.chat('gpt-3.5-turbo', {
  logitBias: {
    // optional likelihood for specific tokens
    '50256': -100,
  },
  user: 'test-user', // optional unique user identifier
});
```

The following optional settings are available for OpenAI chat models:

- **logitBias** _Record&lt;number, number&gt;_

  Modifies the likelihood of specified tokens appearing in the completion.

  Accepts a JSON object that maps tokens (specified by their token ID in
  the GPT tokenizer) to an associated bias value from -100 to 100. You
  can use this tokenizer tool to convert text to token IDs. Mathematically,
  the bias is added to the logits generated by the model prior to sampling.
  The exact effect will vary per model, but values between -1 and 1 should
  decrease or increase likelihood of selection; values like -100 or 100
  should result in a ban or exclusive selection of the relevant token.

  As an example, you can pass `{"50256": -100}` to prevent the token from being generated.

- **logProbs** _boolean | number_

  Return the log probabilities of the tokens. Including logprobs will increase
  the response size and can slow down response times. However, it can
  be useful to better understand how the model is behaving.

  Setting to true will return the log probabilities of the tokens that
  were generated.

  Setting to a number will return the log probabilities of the top n
  tokens that were generated.

- **parallelToolCalls** _boolean_

  Whether to enable parallel function calling during tool use. Defaults to `true`.

- **useLegacyFunctionCalls** _boolean_

  Whether to use legacy function calling. Defaults to false.

  Required by some open source inference engines which do not support the `tools` API. May also
  provide a workaround for `parallelToolCalls` resulting in the provider buffering tool calls,
  which causes `streamObject` to be non-streaming.

  Prefer setting `parallelToolCalls: false` over this option.

- **structuredOutputs** _boolean_

  Whether to use [structured outputs](#structured-outputs). Defaults to `false`.

  When enabled, tool calls and object generation will be strict and follow the provided schema.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids).

- **downloadImages** _boolean_

  Automatically download images and pass the image as data to the model.
  OpenAI supports image URLs for public models, so this is only needed for
  private models or when the images are not publicly accessible.
  Defaults to `false`.

#### Structured Outputs

You can enable [OpenAI structured outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) by setting the `structuredOutputs` option to `true`.
Structured outputs are a form of grammar-guided generation.
The JSON schema is used as a grammar and the outputs will always conform to the schema.

```ts highlight="7"
import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import { z } from 'zod';

const result = await generateObject({
  model: openai('gpt-4o-2024-08-06', {
    structuredOutputs: true,
  }),
  schemaName: 'recipe',
  schemaDescription: 'A recipe for lasagna.',
  schema: z.object({
    name: z.string(),
    ingredients: z.array(
      z.object({
        name: z.string(),
        amount: z.string(),
      }),
    ),
    steps: z.array(z.string()),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(JSON.stringify(result.object, null, 2));
```

<Note type="warning">
  OpenAI structured outputs have several
  [limitations](https://openai.com/index/introducing-structured-outputs-in-the-api),
  in particular around the [supported schemas](https://platform.openai.com/docs/guides/structured-outputs/supported-schemas),
  and are therefore opt-in.

For example, optional schema properties are not supported.
You need to change Zod `.nullish()` and `.optional()` to `.nullable()`.

</Note>

#### Predicted Outputs

OpenAI supports [predicted outputs](https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs) for `gpt-4o` and `gpt-4o-mini`.
Predicted outputs help you reduce latency by allowing you to specify a base text that the model should modify.
You can enable predicted outputs by adding the `prediction` option to the `experimental_providerMetadata.openai` object:

```ts highlight="15-18"
const result = streamText({
  model: openai('gpt-4o'),
  messages: [
    {
      role: 'user',
      content: 'Replace the Username property with an Email property.',
    },
    {
      role: 'user',
      content: existingCode,
    },
  ],
  experimental_providerMetadata: {
    openai: {
      prediction: {
        type: 'content',
        content: existingCode,
      },
    },
  },
});
```

#### Image Detail

You can use the `openai` provider metadata to set the [image generation detail](https://platform.openai.com/docs/guides/vision/low-or-high-fidelity-image-understanding) to `high`, `low`, or `auto`:

```ts highlight="13-16"
const result = await generateText({
  model: openai('gpt-4o'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'image',
          image:
            'https://github.com/vercel/ai/blob/main/examples/ai-core/data/comic-cat.png?raw=true',

          // OpenAI specific extension - image detail:
          experimental_providerMetadata: {
            openai: { imageDetail: 'low' },
          },
        },
      ],
    },
  ],
});
```

#### Distillation

OpenAI supports model distillation for some models. If you want to store a generation for use in the distillation process, you can add the `store` option to the `experimental_providerMetadata.openai` object. This will save the generation to the OpenAI platform for later use in distillation.

```typescript highlight="9-13"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const { text, usage } = await generateText({
    model: openai('gpt-4o-mini'),
    prompt: 'Who worked on the original macintosh?',
    experimental_providerMetadata: {
      openai: {
        store: true,
        metadata: {
          custom: 'value',
        },
      },
    },
  });

  console.log(text);
  console.log();
  console.log('Usage:', usage);
}

main().catch(console.error);
```

#### Reasoning Models

OpenAI has introduced the `o1` series of [reasoning models](https://platform.openai.com/docs/guides/reasoning).
Currently, `o1-mini` and `o1-preview` are available.

Reasoning models currently only generate text, have several limitations, and are only supported using `generateText` and `streamText`.

Reasoning models support two additional options:

- You can use request `experimental_providerMetadata` to set the `maxCompletionTokens` option, which determines the maximum number
  of both reasoning and output tokens that the model generates.
- You can use response `experimental_providerMetadata` to access the number of reasoning tokens that the model generated.

```ts highlight="4,7-9,15"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const { text, usage, experimental_providerMetadata } = await generateText({
  model: openai('o1-mini'),
  prompt: 'Invent a new holiday and describe its traditions.',
  experimental_providerMetadata: {
    openai: { maxCompletionTokens: 1000 },
  },
});

console.log(text);
console.log('Usage:', {
  ...usage,
  reasoningTokens: experimental_providerMetadata?.openai?.reasoningTokens,
});
```

#### Prompt Caching

OpenAI has introduced [Prompt Caching](https://platform.openai.com/docs/guides/prompt-caching) for supported models
including `gpt-4o`, `gpt-4o-mini`, `o1-preview`, and `o1-mini`.

- Prompt caching is automatically enabled for these models, when the prompt is 1024 tokens or longer. It does
  not need to be explicitly enabled.
- You can use response `experimental_providerMetadata` to access the number of prompt tokens that were a cache hit.
- Note that caching behavior is dependent on load on OpenAI's infrastructure. Prompt prefixes generally remain in the
  cache following 5-10 minutes of inactivity before they are evicted, but during off-peak periods they may persist for up
  to an hour.

```ts highlight="11"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const { text, usage, experimental_providerMetadata } = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: `A 1024-token or longer prompt...`,
});

console.log(`usage:`, {
  ...usage,
  cachedPromptTokens: experimental_providerMetadata?.openai?.cachedPromptTokens,
});
```

#### Audio Input

With the `gpt-4o-audio-preview` model, you can pass audio files to the model.

<Note type="warning">
  The `gpt-4o-audio-preview` model is currently in preview and requires at least
  some audio inputs. It will not work with non-audio data.
</Note>

```ts highlight="12-14"
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const result = await generateText({
  model: openai('gpt-4o-audio-preview'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the audio saying?' },
        {
          type: 'file',
          mimeType: 'audio/mpeg',
          data: fs.readFileSync('./data/galileo.mp3'),
        },
      ],
    },
  ],
});
```

### Completion Models

You can create models that call the [OpenAI completions API](https://platform.openai.com/docs/api-reference/completions) using the `.completion()` factory method.
The first argument is the model id.
Currently only `gpt-3.5-turbo-instruct` is supported.

```ts
const model = openai.completion('gpt-3.5-turbo-instruct');
```

OpenAI completion models support also some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = openai.completion('gpt-3.5-turbo-instruct', {
  echo: true, // optional, echo the prompt in addition to the completion
  logitBias: {
    // optional likelihood for specific tokens
    '50256': -100,
  },
  suffix: 'some text', // optional suffix that comes after a completion of inserted text
  user: 'test-user', // optional unique user identifier
});
```

The following optional settings are available for OpenAI completion models:

- **echo**: _boolean_

  Echo back the prompt in addition to the completion.

- **logitBias** _Record&lt;number, number&gt;_

  Modifies the likelihood of specified tokens appearing in the completion.

  Accepts a JSON object that maps tokens (specified by their token ID in
  the GPT tokenizer) to an associated bias value from -100 to 100. You
  can use this tokenizer tool to convert text to token IDs. Mathematically,
  the bias is added to the logits generated by the model prior to sampling.
  The exact effect will vary per model, but values between -1 and 1 should
  decrease or increase likelihood of selection; values like -100 or 100
  should result in a ban or exclusive selection of the relevant token.

  As an example, you can pass `{"50256": -100}` to prevent the &lt;|endoftext|&gt;
  token from being generated.

- **logProbs** _boolean | number_

  Return the log probabilities of the tokens. Including logprobs will increase
  the response size and can slow down response times. However, it can
  be useful to better understand how the model is behaving.

  Setting to true will return the log probabilities of the tokens that
  were generated.

  Setting to a number will return the log probabilities of the top n
  tokens that were generated.

- **suffix** _string_

  The suffix that comes after a completion of inserted text.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids).

### Model Capabilities

| Model                  | Image Input         | Audio Input         | Object Generation   | Tool Usage          |
| ---------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `gpt-4o`               | <Check size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gpt-4o-mini`          | <Check size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gpt-4o-audio-preview` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gpt-4-turbo`          | <Check size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gpt-4`                | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gpt-3.5-turbo`        | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `o1-preview`           | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `o1-mini`              | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

## Embedding Models

You can create models that call the [OpenAI embeddings API](https://platform.openai.com/docs/api-reference/embeddings)
using the `.embedding()` factory method.

```ts
const model = openai.embedding('text-embedding-3-large');
```

OpenAI embedding models support several aditional settings.
You can pass them as an options argument:

```ts
const model = openai.embedding('text-embedding-3-large', {
  dimensions: 512 // optional, number of dimensions for the embedding
  user: 'test-user' // optional unique user identifier
})
```

The following optional settings are available for OpenAI embedding models:

- **dimensions**: _number_

  The number of dimensions the resulting output embeddings should have.
  Only supported in text-embedding-3 and later models.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids).

### Model Capabilities

| Model                    | Default Dimensions | Custom Dimensions   |
| ------------------------ | ------------------ | ------------------- |
| `text-embedding-3-large` | 3072               | <Check size={18} /> |
| `text-embedding-3-small` | 1536               | <Check size={18} /> |
| `text-embedding-ada-002` | 1536               | <Cross size={18} /> |

---
title: Azure OpenAI
description: Learn how to use the Azure OpenAI provider for the AI SDK.
---

# Azure OpenAI Provider

The [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) provider contains language model support for the Azure OpenAI chat API.

## Setup

The Azure OpenAI provider is available in the `@ai-sdk/azure` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/azure" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/azure" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/azure" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `azure` from `@ai-sdk/azure`:

```ts
import { azure } from '@ai-sdk/azure';
```

If you need a customized setup, you can import `createAzure` from `@ai-sdk/azure` and create a provider instance with your settings:

```ts
import { createAzure } from '@ai-sdk/azure';

const azure = createAzure({
  resourceName: 'your-resource-name', // Azure resource name
  apiKey: 'your-api-key',
});
```

You can use the following optional settings to customize the OpenAI provider instance:

- **resourceName** _string_

  Azure resource name.
  It defaults to the `AZURE_RESOURCE_NAME` environment variable.

  The resource name is used in the assembled URL: `https://{resourceName}.openai.azure.com/openai/deployments/{modelId}{path}`.
  You can use `baseURL` instead to specify the URL prefix.

- **apiKey** _string_

  API key that is being sent using the `api-key` header.
  It defaults to the `AZURE_API_KEY` environment variable.

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.

  Either this or `resourceName` can be used.
  When a baseURL is provided, the resourceName is ignored.

  With a baseURL, the resolved URL is `{baseURL}/{modelId}{path}`.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

The Azure OpenAI provider instance is a function that you can invoke to create a language model:

```ts
const model = azure('your-deployment-name');
```

You need to pass your deployment name as the first argument.

### Example

You can use OpenAI language models to generate text with the `generateText` function:

```ts
import { azure } from '@ai-sdk/azure';
import { generateText } from 'ai';

const { text } = await generateText({
  model: azure('your-deployment-name'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

OpenAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

<Note>
  Azure OpenAI sends larger chunks than OpenAI. This can lead to the perception
  that the response is slower. See [Troubleshooting: Azure OpenAI Slow To
  Stream](/docs/troubleshooting/common-issues/azure-stream-slow)
</Note>

### Chat Models

<Note>
  The URL for calling Azure chat models will be constructed as follows:
  `https://RESOURCE_NAME.openai.azure.com/openai/deployments/DEPLOYMENT_NAME/chat/completions?api-version=2024-10-01-preview`
</Note>

Azure OpenAI chat models support also some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = azure('your-deployment-name', {
  logitBias: {
    // optional likelihood for specific tokens
    '50256': -100,
  },
  user: 'test-user', // optional unique user identifier
});
```

The following optional settings are available for OpenAI chat models:

- **logitBias** _Record&lt;number, number&gt;_

  Modifies the likelihood of specified tokens appearing in the completion.

  Accepts a JSON object that maps tokens (specified by their token ID in
  the GPT tokenizer) to an associated bias value from -100 to 100. You
  can use this tokenizer tool to convert text to token IDs. Mathematically,
  the bias is added to the logits generated by the model prior to sampling.
  The exact effect will vary per model, but values between -1 and 1 should
  decrease or increase likelihood of selection; values like -100 or 100
  should result in a ban or exclusive selection of the relevant token.

  As an example, you can pass `{"50256": -100}` to prevent the token from being generated.

- **logProbs** _boolean | number_

  Return the log probabilities of the tokens. Including logprobs will increase
  the response size and can slow down response times. However, it can
  be useful to better understand how the model is behaving.

  Setting to true will return the log probabilities of the tokens that
  were generated.

  Setting to a number will return the log probabilities of the top n
  tokens that were generated.

- **parallelToolCalls** _boolean_

  Whether to enable parallel function calling during tool use. Default to true.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. Learn more.

### Completion Models

You can create models that call the completions API using the `.completion()` factory method.
The first argument is the model id.
Currently only `gpt-35-turbo-instruct` is supported.

```ts
const model = azure.completion('your-gpt-35-turbo-instruct-deployment');
```

OpenAI completion models support also some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = azure.completion('your-gpt-35-turbo-instruct-deployment', {
  echo: true, // optional, echo the prompt in addition to the completion
  logitBias: {
    // optional likelihood for specific tokens
    '50256': -100,
  },
  suffix: 'some text', // optional suffix that comes after a completion of inserted text
  user: 'test-user', // optional unique user identifier
});
```

The following optional settings are available for Azure OpenAI completion models:

- **echo**: _boolean_

  Echo back the prompt in addition to the completion.

- **logitBias** _Record&lt;number, number&gt;_

  Modifies the likelihood of specified tokens appearing in the completion.

  Accepts a JSON object that maps tokens (specified by their token ID in
  the GPT tokenizer) to an associated bias value from -100 to 100. You
  can use this tokenizer tool to convert text to token IDs. Mathematically,
  the bias is added to the logits generated by the model prior to sampling.
  The exact effect will vary per model, but values between -1 and 1 should
  decrease or increase likelihood of selection; values like -100 or 100
  should result in a ban or exclusive selection of the relevant token.

  As an example, you can pass `{"50256": -100}` to prevent the &lt;|endoftext|&gt;
  token from being generated.

- **logProbs** _boolean | number_

  Return the log probabilities of the tokens. Including logprobs will increase
  the response size and can slow down response times. However, it can
  be useful to better understand how the model is behaving.

  Setting to true will return the log probabilities of the tokens that
  were generated.

  Setting to a number will return the log probabilities of the top n
  tokens that were generated.

- **suffix** _string_

  The suffix that comes after a completion of inserted text.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. Learn more.

## Embedding Models

You can create models that call the Azure OpenAI embeddings API
using the `.embedding()` factory method.

```ts
const model = azure.embedding('your-embedding-deployment');
```

Azure OpenAI embedding models support several aditional settings.
You can pass them as an options argument:

```ts
const model = azure.embedding('your-embedding-deployment', {
  dimensions: 512 // optional, number of dimensions for the embedding
  user: 'test-user' // optional unique user identifier
})
```

The following optional settings are available for Azure OpenAI embedding models:

- **dimensions**: _number_

  The number of dimensions the resulting output embeddings should have.
  Only supported in text-embedding-3 and later models.

- **user** _string_

  A unique identifier representing your end-user, which can help OpenAI to
  monitor and detect abuse. Learn more.

---
title: Anthropic
description: Learn how to use the Anthropic provider for the AI SDK.
---

# Anthropic Provider

The [Anthropic](https://www.anthropic.com/) provider contains language model support for the [Anthropic Messages API](https://docs.anthropic.com/claude/reference/messages_post).

## Setup

The Anthropic provider is available in the `@ai-sdk/anthropic` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/anthropic" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/anthropic" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/anthropic" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `anthropic` from `@ai-sdk/anthropic`:

```ts
import { anthropic } from '@ai-sdk/anthropic';
```

If you need a customized setup, you can import `createAnthropic` from `@ai-sdk/anthropic` and create a provider instance with your settings:

```ts
import { createAnthropic } from '@ai-sdk/anthropic';

const anthropic = createAnthropic({
  // custom settings
});
```

You can use the following optional settings to customize the Anthropic provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.anthropic.com/v1`.

- **apiKey** _string_

  API key that is being sent using the `x-api-key` header.
  It defaults to the `ANTHROPIC_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create models that call the [Anthropic Messages API](https://docs.anthropic.com/claude/reference/messages_post) using the provider instance.
The first argument is the model id, e.g. `claude-3-haiku-20240307`.
Some models have multi-modal capabilities.

```ts
const model = anthropic('claude-3-haiku-20240307');
```

The following optional settings are available for Anthropic models:

- **cacheControl** _boolean_

  Enable the Anthropic cache control beta.

  You can then use provider metadata to set cache control breakpoints ([example](#example-cache-control))

You can use Anthropic language models to generate text with the `generateText` function:

```ts
import { anthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';

const { text } = await generateText({
  model: anthropic('claude-3-haiku-20240307'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Anthropic language models can also be used in the `streamText`, `generateObject`, and `streamObject` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

<Note>
  The Anthropic API returns streaming tool calls all at once after a delay. This
  causes the `streamObject` function to generate the object fully after a delay
  instead of streaming it incrementally.
</Note>

### Cache Control

You can enable the cache control beta by setting the `cacheControl` option to `true` when creating the model instance.

In the messages and message parts, you can then use the `experimental_providerMetadata` property to set cache control breakpoints.
You need to set the `anthropic` property in the `experimental_providerMetadata` object to `{ cacheControl: { type: 'ephemeral' } }` to set a cache control breakpoint.

The cache creation input tokens are then returned in the `experimental_providerMetadata` object
for `generateText` and `generateObject`, again under the `anthropic` property.
When you use `streamText` or `streamObject`, the response contains a promise
that resolves to the metadata. Alternatively you can receive it in the
`onFinish` callback.

```ts highlight="8,18-20,29-30"
import { anthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';

const errorMessage = '... long error message ...';

const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20240620', {
    cacheControl: true,
  }),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'You are a JavaScript expert.' },
        {
          type: 'text',
          text: `Error message: ${errorMessage}`,
          experimental_providerMetadata: {
            anthropic: { cacheControl: { type: 'ephemeral' } },
          },
        },
        { type: 'text', text: 'Explain the error message.' },
      ],
    },
  ],
});

console.log(result.text);
console.log(result.experimental_providerMetadata?.anthropic);
// e.g. { cacheCreationInputTokens: 2118, cacheReadInputTokens: 0 }
```

You can also use cache control on system messages by providing multiple system messages at the head of your messages array:

```ts highlight="3,9-11"
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20240620', {
    cacheControl: true,
  }),
  messages: [
    {
      role: 'system',
      content: 'Cached system message part',
      experimental_providerMetadata: {
        anthropic: { cacheControl: { type: 'ephemeral' } },
      },
    },
    {
      role: 'system',
      content: 'Uncached system message part',
    },
    {
      role: 'user',
      content: 'User prompt',
    },
  ],
});
```

### Computer Use

Anthropic provides three built-in tools that can be used to interact with external systems:

1. **Bash Tool**: Allows running bash commands.
2. **Text Editor Tool**: Provides functionality for viewing and editing text files.
3. **Computer Tool**: Enables control of keyboard and mouse actions on a computer.

They are available via the `tools` property of the provider instance.

#### Bash Tool

The Bash Tool allows running bash commands. Here's how to create and use it:

```ts
const bashTool = anthropic.tools.bash_20241022({
  execute: async ({ command, restart }) => {
    // Implement your bash command execution logic here
    // Return the result of the command execution
  },
});
```

Parameters:

- `command` (string): The bash command to run. Required unless the tool is being restarted.
- `restart` (boolean, optional): Specifying true will restart this tool.

#### Text Editor Tool

The Text Editor Tool provides functionality for viewing and editing text files:

```ts
const textEditorTool = anthropic.tools.textEditor_20241022({
  execute: async ({
    command,
    path,
    file_text,
    insert_line,
    new_str,
    old_str,
    view_range,
  }) => {
    // Implement your text editing logic here
    // Return the result of the text editing operation
  },
});
```

Parameters:

- `command` ('view' | 'create' | 'str_replace' | 'insert' | 'undo_edit'): The command to run.
- `path` (string): Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.
- `file_text` (string, optional): Required for `create` command, with the content of the file to be created.
- `insert_line` (number, optional): Required for `insert` command. The line number after which to insert the new string.
- `new_str` (string, optional): New string for `str_replace` or `insert` commands.
- `old_str` (string, optional): Required for `str_replace` command, containing the string to replace.
- `view_range` (number[], optional): Optional for `view` command to specify line range to show.

#### Computer Tool

The Computer Tool enables control of keyboard and mouse actions on a computer:

```ts
const computerTool = anthropic.tools.computer_20241022({
  displayWidthPx: 1920,
  displayHeightPx: 1080,
  displayNumber: 0, // Optional, for X11 environments

  execute: async ({ action, coordinate, text }) => {
    // Implement your computer control logic here
    // Return the result of the action

    // Example code:
    switch (action) {
      case 'screenshot': {
        // multipart result:
        return {
          type: 'image',
          data: fs
            .readFileSync('./data/screenshot-editor.png')
            .toString('base64'),
        };
      }
      default: {
        console.log('Action:', action);
        console.log('Coordinate:', coordinate);
        console.log('Text:', text);
        return `executed ${action}`;
      }
    }
  },

  // map to tool result content for LLM consumption:
  experimental_toToolResultContent(result) {
    return typeof result === 'string'
      ? [{ type: 'text', text: result }]
      : [{ type: 'image', data: result.data, mimeType: 'image/png' }];
  },
});
```

Parameters:

- `action` ('key' | 'type' | 'mouse_move' | 'left_click' | 'left_click_drag' | 'right_click' | 'middle_click' | 'double_click' | 'screenshot' | 'cursor_position'): The action to perform.
- `coordinate` (number[], optional): Required for `mouse_move` and `left_click_drag` actions. Specifies the (x, y) coordinates.
- `text` (string, optional): Required for `type` and `key` actions.

These tools can be used in conjunction with the `sonnet-3-5-sonnet-20240620` model to enable more complex interactions and tasks.

### PDF support

Anthropic Sonnet `claude-3-5-sonnet-20241022` supports reading PDF files.
You can pass PDF files as part of the message content using the `file` type:

```ts
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'What is an embedding model according to this document?',
        },
        {
          type: 'file',
          data: fs.readFileSync('./data/ai.pdf'),
          mimeType: 'application/pdf',
        },
      ],
    },
  ],
});
```

The model will have access to the contents of the PDF file and
respond to questions about it.
The PDF file should be passed using the `data` field,
and the `mimeType` should be set to `'application/pdf'`.

### Model Capabilities

See also [Anthropic Model Comparison](https://docs.anthropic.com/en/docs/about-claude/models#model-comparison).

| Model                        | Image Input         | Object Generation   | Tool Usage          | Computer Use        |
| ---------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `claude-3-5-sonnet-20241022` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `claude-3-5-sonnet-20240620` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `claude-3-5-haiku-20241022`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `claude-3-opus-20240229`     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `claude-3-sonnet-20240229`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `claude-3-haiku-20240307`    | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

---
title: Amazon Bedrock
description: Learn how to use the Amazon Bedrock provider.
---

# Amazon Bedrock Provider

The Amazon Bedrock provider for the [AI SDK](https://sdk.vercel.ai/docs) contains language model support for the [Amazon Bedrock](https://aws.amazon.com/bedrock) APIs.

## Setup

The Bedrock provider is available in the `@ai-sdk/amazon-bedrock` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/amazon-bedrock" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/amazon-bedrock" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/amazon-bedrock" dark />
  </Tab>
</Tabs>

### Prerequisites

Access to Amazon Bedrock foundation models isn't granted by default. In order to gain access to a foundation model, an IAM user with sufficient permissions needs to request access to it through the console. Once access is provided to a model, it is available for all users in the account.

See the [Model Access Docs](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for more information.

### Authentication

**Step 1: Creating AWS Access Key and Secret Key**

To get started, you'll need to create an AWS access key and secret key. Here's how:

**Login to AWS Management Console**

- Go to the [AWS Management Console](https://console.aws.amazon.com/) and log in with your AWS account credentials.

**Create an IAM User**

- Navigate to the [IAM dashboard](https://console.aws.amazon.com/iam/home) and click on "Users" in the left-hand navigation menu.
- Click on "Create user" and fill in the required details to create a new IAM user.
- Make sure to select "Programmatic access" as the access type.
- The user account needs the `AmazonBedrockFullAccess` policy attached to it.

**Create Access Key**

- Click on the "Security credentials" tab and then click on "Create access key".
- Click "Create access key" to generate a new access key pair.
- Download the `.csv` file containing the access key ID and secret access key.

**Step 2: Configuring the Access Key and Secret Key**

Within your project add a `.env` file if you don't already have one. This file will be used to set the access key and secret key as environment variables. Add the following lines to the `.env` file:

```makefile
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY
AWS_REGION=YOUR_REGION
```

<Note>
  Many frameworks such as [Next.js](https://nextjs.org/) load the `.env` file
  automatically. If you're using a different framework, you may need to load the
  `.env` file manually using a package like
  [`dotenv`](https://github.com/motdotla/dotenv).
</Note>

Remember to replace `YOUR_ACCESS_KEY_ID`, `YOUR_SECRET_ACCESS_KEY`, and `YOUR_REGION` with the actual values from your AWS account.

## Provider Instance

You can import the default provider instance `bedrock` from `@ai-sdk/amazon-bedrock`:

```ts
import { bedrock } from '@ai-sdk/amazon-bedrock';
```

If you need a customized setup, you can import `createAmazonBedrock` from `@ai-sdk/amazon-bedrock` and create a provider instance with your settings:

```ts
import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';

const bedrock = createAmazonBedrock({
  region: 'us-east-1',
  accessKeyId: 'xxxxxxxxx',
  secretAccessKey: 'xxxxxxxxx',
  sessionToken: 'xxxxxxxxx',
});

// or with bedrockOptions
const bedrock = createAmazonBedrock({
  bedrockOptions: {
    region: 'us-east-1',
    credentials: {
      // ...
    },
  },
});
```

You can use the following optional settings to customize the Amazon Bedrock provider instance:

- **region** _string_

  The AWS region that you want to use for the API calls.
  It uses the `AWS_REGION` environment variable by default.

- **accessKeyId** _string_

  The AWS access key ID that you want to use for the API calls.
  It uses the `AWS_ACCESS_KEY_ID` environment variable by default.

- **secretAccessKey** _string_

  The AWS secret access key that you want to use for the API calls.
  It uses the `AWS_SECRET_ACCESS_KEY` environment variable by default.

- **sessionToken** _string_

  Optional. The AWS session token that you want to use for the API calls.
  It uses the `AWS_SESSION_TOKEN` environment variable by default.

- **bedrockOptions** _object_

  Optional. The configuration options used by the [Amazon Bedrock Library](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-bedrock-runtime/)
  (`BedrockRuntimeClientConfig`), including:

  - **region** _string_
    The AWS region that you want to use for the API calls.

  - **credentials** _object_
    The AWS credentials that you want to use for the API calls.

  When `bedrockOptions` are provided, the `region`, `accessKeyId`, and `secretAccessKey` settings are ignored.

## Language Models

You can create models that call the Bedrock API using the provider instance.
The first argument is the model id, e.g. `meta.llama3-70b-instruct-v1:0`.

```ts
const model = bedrock('meta.llama3-70b-instruct-v1:0');
```

Amazon Bedrock models also support some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = bedrock('anthropic.claude-3-sonnet-20240229-v1:0', {
  additionalModelRequestFields: { top_k: 350 },
});
```

Documentation for additional settings based on the selected model can be found within the [Amazon Bedrock Inference Parameter Documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).

You can use Amazon Bedrock language models to generate text with the `generateText` function:

```ts
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { generateText } from 'ai';

const { text } = await generateText({
  model: bedrock('meta.llama3-70b-instruct-v1:0'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Amazon Bedrock language models can also be used in the `streamText` function
(see [AI SDK Core](/docs/ai-sdk-core)).

### File Inputs

<Note type="warning">
  Amazon Bedrock supports file inputs on in combination with specific models,
  e.g. `anthropic.claude-3-haiku-20240307-v1:0`.
</Note>

The Amazon Bedrock provider supports file inputs, e.g. PDF files.

```ts
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { generateText } from 'ai';

const result = await generateText({
  model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the pdf in detail.' },
        {
          type: 'file',
          data: fs.readFileSync('./data/ai.pdf'),
          mimeType: 'application/pdf',
        },
      ],
    },
  ],
});
```

### Guardrails

You can use the `bedrock` provider metadata to utilize [Amazon Bedrock Guardrails](https://aws.amazon.com/bedrock/guardrails/):

```ts
const result = await generateText({
  bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  experimental_providerMetadata: {
    bedrock: {
      guardrailConfig: {
        guardrailIdentifier: '1abcd2ef34gh',
        guardrailVersion: '1',
        trace: 'enabled' as const,
        streamProcessingMode: 'async',
      },
    },
  },
});
```

Tracing information will be returned in the provider metadata if you have tracing enabled.

```ts
if (result.experimental_providerMetadata?.bedrock.trace) {
  // ...
}
```

See the [Amazon Bedrock Guardrails documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) for more information.

### Model Capabilities

> Note: This model list is ever changing and may not be complete. Refer to the [Amazon Bedrock ](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) documentation for up to date information.

| Model                                       | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `amazon.titan-tg1-large`                    | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `amazon.titan-text-express-v1`              | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `anthropic.claude-3-5-sonnet-20241022-v2:0` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `anthropic.claude-3-5-sonnet-20240620-v1:0` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `anthropic.claude-3-5-haiku-20241022-v1:0`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `anthropic.claude-3-opus-20240229-v1:0`     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `anthropic.claude-3-sonnet-20240229-v1:0`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `anthropic.claude-3-haiku-20240307-v1:0`    | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `anthropic.claude-v2:1`                     | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `cohere.command-r-v1:0`                     | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `cohere.command-r-plus-v1:0`                | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `meta.llama2-13b-chat-v1`                   | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `meta.llama2-70b-chat-v1`                   | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `meta.llama3-8b-instruct-v1:0`              | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `meta.llama3-70b-instruct-v1:0`             | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `mistral.mistral-7b-instruct-v0:2`          | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `mistral.mixtral-8x7b-instruct-v0:1`        | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `mistral.mistral-large-2402-v1:0`           | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| `mistral.mistral-small-2402-v1:0`           | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

## Embedding Models

You can create models that call the Bedrock API [Bedrock API](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html)
using the `.embedding()` factory method.

```ts
const model = bedrock.embedding('amazon.titan-embed-text-v1');
```

Bedrock Titan embedding model amazon.titan-embed-text-v2:0 supports several aditional settings.
You can pass them as an options argument:

```ts
const model = bedrock.embedding('amazon.titan-embed-text-v2:0', {
  dimensions: 512 // optional, number of dimensions for the embedding
  normalize: true // optional  normalize the output embeddings
})
```

The following optional settings are available for Bedrock Titan embedding models:

- **dimensions**: _number_

  The number of dimensions the output embeddings should have. The following values are accepted: 1024 (default), 512, 256.

- **normalize** _boolean_

  Flag indicating whether or not to normalize the output embeddings. Defaults to true.

### Model Capabilities

| Model                          | Default Dimensions | Custom Dimensions   |
| ------------------------------ | ------------------ | ------------------- |
| `amazon.titan-embed-text-v1`   | 1024               | <Cross size={18} /> |
| `amazon.titan-embed-text-v2:0` | 1024               | <Check size={18} /> |

---
title: Google Generative AI
description: Learn how to use Google Generative AI Provider.
---

# Google Generative AI Provider

The [Google Generative AI](https://ai.google/discover/generativeai/) provider contains language and embedding model support for
the [Google Generative AI](https://ai.google.dev/api/rest) APIs.

## Setup

The Google provider is available in the `@ai-sdk/google` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/google" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/google" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/google" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `google` from `@ai-sdk/google`:

```ts
import { google } from '@ai-sdk/google';
```

If you need a customized setup, you can import `createGoogleGenerativeAI` from `@ai-sdk/google` and create a provider instance with your settings:

```ts
import { createGoogleGenerativeAI } from '@ai-sdk/google';

const google = createGoogleGenerativeAI({
  // custom settings
});
```

You can use the following optional settings to customize the Google Generative AI provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://generativelanguage.googleapis.com/v1beta`.

- **apiKey** _string_

  API key that is being sent using the `x-goog-api-key` header.
  It defaults to the `GOOGLE_GENERATIVE_AI_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create models that call the [Google Generative AI API](https://ai.google.dev/api/rest) using the provider instance.
The first argument is the model id, e.g. `gemini-1.5-pro-latest`.
The models support tool calls and some have multi-modal capabilities.

```ts
const model = google('gemini-1.5-pro-latest');
```

<Note>
  You can use fine-tuned models by prefixing the model id with `tunedModels/`,
  e.g. `tunedModels/my-model`.
</Note>

Google Generative AI models support also some model specific settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = google('gemini-1.5-pro-latest', {
  safetySettings: [
    { category: 'HARM_CATEGORY_UNSPECIFIED', threshold: 'BLOCK_LOW_AND_ABOVE' },
  ],
});
```

The following optional settings are available for Google Generative AI models:

- **cachedContent** _string_

  Optional. The name of the cached content used as context to serve the prediction.
  Format: cachedContents/\{cachedContent\}

- **structuredOutputs** _boolean_

  Optional. Enable structured output. Default is true.

  This is useful when the JSON Schema contains elements that are
  not supported by the OpenAPI schema version that
  Google Generative AI uses. You can use this to disable
  structured outputs if you need to.

  See [Troubleshooting: Schema Limitations](#troubleshooting-schema-limitations) for more details.

- **safetySettings** _Array\<\{ category: string; threshold: string \}\>_

  Optional. Safety settings for the model.

  - **category** _string_

    The category of the safety setting. Can be one of the following:

    - `HARM_CATEGORY_HATE_SPEECH`
    - `HARM_CATEGORY_DANGEROUS_CONTENT`
    - `HARM_CATEGORY_HARASSMENT`
    - `HARM_CATEGORY_SEXUALLY_EXPLICIT`

  - **threshold** _string_

    The threshold of the safety setting. Can be one of the following:

    - `HARM_BLOCK_THRESHOLD_UNSPECIFIED`
    - `BLOCK_LOW_AND_ABOVE`
    - `BLOCK_MEDIUM_AND_ABOVE`
    - `BLOCK_ONLY_HIGH`
    - `BLOCK_NONE`

You can use Google Generative AI language models to generate text with the `generateText` function:

```ts
import { google } from '@ai-sdk/google';
import { generateText } from 'ai';

const { text } = await generateText({
  model: google('gemini-1.5-pro-latest'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Google Generative AI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### File Inputs

The Google Generative AI provider supports file inputs, e.g. PDF files.

```ts
import { google } from '@ai-sdk/google';
import { generateText } from 'ai';

const result = await generateText({
  model: google('gemini-1.5-flash'),
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'What is an embedding model according to this document?',
        },
        {
          type: 'file',
          data: fs.readFileSync('./data/ai.pdf'),
          mimeType: 'application/pdf',
        },
      ],
    },
  ],
});
```

<Note>
  The AI SDK will automatically download URLs if you pass them as data, except
  for `https://generativelanguage.googleapis.com/v1beta/files/`. You can use the
  Google Generative AI Files API to upload larger files to that location.
</Note>

See [File Parts](/docs/foundations/prompts#file-parts) for details on how to use files in prompts.

### Cached Content

You can use Google Generative AI language models to cache content:

```ts
import { google } from '@ai-sdk/google';
import { GoogleAICacheManager } from '@google/generative-ai/server';
import { generateText } from 'ai';

const cacheManager = new GoogleAICacheManager(
  process.env.GOOGLE_GENERATIVE_AI_API_KEY,
);

// As of August 23rd, 2024, these are the only models that support caching
type GoogleModelCacheableId =
  | 'models/gemini-1.5-flash-001'
  | 'models/gemini-1.5-pro-001';

const model: GoogleModelCacheableId = 'models/gemini-1.5-pro-001';

const { name: cachedContent } = await cacheManager.create({
  model,
  contents: [
    {
      role: 'user',
      parts: [{ text: '1000 Lasanga Recipes...' }],
    },
  ],
  ttlSeconds: 60 * 5,
});

const { text: veggieLasangaRecipe } = await generateText({
  model: google(model, { cachedContent }),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

const { text: meatLasangaRecipe } = await generateText({
  model: google(model, { cachedContent }),
  prompt: 'Write a meat lasagna recipe for 12 people.',
});
```

### Troubleshooting

#### Schema Limitations

The Google Generative AI API uses a subset of the OpenAPI 3.0 schema,
which does not support features such as unions.
The errors that you get in this case look like this:

`GenerateContentRequest.generation_config.response_schema.properties[occupation].type: must be specified`

By default, structured outputs are enabled (and for tool calling they are required).
You can disable structured outputs for object generation as a workaround:

```ts highlight="3,8"
const result = await generateObject({
  model: google('gemini-1.5-pro-latest', {
    structuredOutputs: false,
  }),
  schema: z.object({
    name: z.string(),
    age: z.number(),
    contact: z.union([
      z.object({
        type: z.literal('email'),
        value: z.string(),
      }),
      z.object({
        type: z.literal('phone'),
        value: z.string(),
      }),
    ]),
  }),
  prompt: 'Generate an example person for testing.',
});
```

### Model Capabilities

| Model                     | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `gemini-1.5-pro-latest`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gemini-1.5-pro`          | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gemini-1.5-flash-latest` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gemini-1.5-flash`        | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

## Embedding Models

You can create models that call the [Google Generative AI embeddings API](https://ai.google.dev/api/embeddings)
using the `.textEmbeddingModel()` factory method.

```ts
const model = google.textEmbeddingModel('text-embedding-004');
```

Google Generative AI embedding models support aditional settings. You can pass them as an options argument:

```ts
const model = google.textEmbeddingModel('text-embedding-004', {
  outputDimensionality: 512, // optional, number of dimensions for the embedding
});
```

The following optional settings are available for Google Generative AI embedding models:

- **outputDimensionality**: _number_

  Optional reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end.

### Model Capabilities

| Model                | Default Dimensions | Custom Dimensions   |
| -------------------- | ------------------ | ------------------- |
| `text-embedding-004` | 768                | <Check size={18} /> |

---
title: Google Vertex AI
description: Learn how to use the Google Vertex AI provider.
---

# Google Vertex Provider

The Google Vertex provider for the [AI SDK](https://sdk.vercel.ai/docs) contains language model support for the [Google Vertex AI](https://cloud.google.com/vertex-ai) APIs.

<Note>
  The Google Vertex provider is not compatible with edge environments.
</Note>

## Setup

The Google provider is available in the `@ai-sdk/google-vertex` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/google-vertex" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/google-vertex" dark />
  </Tab>
  <Tab>
    <Snippet
      text="yarn add @ai-sdk/google-vertex @google-cloud/vertexai"
      dark
    />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `vertex` from `@ai-sdk/google-vertex`:

```ts
import { vertex } from '@ai-sdk/google-vertex';
```

If you need a customized setup, you can import `createVertex` from `@ai-sdk/google-vertex` and create a provider instance with your settings:

```ts
import { createVertex } from '@ai-sdk/google-vertex';

const vertex = createVertex({
  project: 'my-project', // optional
  location: 'us-central1', // optional
});
```

You can use the following optional settings to customize the Google Generative AI provider instance:

- **project** _string_

  The Google Cloud project ID that you want to use for the API calls.
  It uses the `GOOGLE_VERTEX_PROJECT` environment variable by default.

- **location** _string_

  The Google Cloud location that you want to use for the API calls, e.g. `us-central1`.
  It uses the `GOOGLE_VERTEX_LOCATION` environment variable by default.

- **googleAuthOptions** _object_

  Optional. The Authentication options used by the [Google Auth Library](https://github.com/googleapis/google-auth-library-nodejs/):

  - **authClient** _object_
    An `AuthClient` to use.

  - **keyFilename** _string_
    Path to a .json, .pem, or .p12 key file.

  - **keyFile** _string_
    Path to a .json, .pem, or .p12 key file.

  - **credentials** _object_
    Object containing client_email and private_key properties, or the external account client options.

  - **clientOptions** _object_
    Options object passed to the constructor of the client.

  - **scopes** _string | string[]_
    Required scopes for the desired API request.

  - **projectId** _string_
    Your project ID.

  - **universeDomain** _string_
    The default service domain for a given Cloud universe.

## Language Models

You can create models that call the Vertex API using the provider instance.
The first argument is the model id, e.g. `gemini-1.5-pro`.

```ts
const model = vertex('gemini-1.5-pro');
```

<Note>
  If you are using [your own
  models](https://cloud.google.com/vertex-ai/docs/training-overview), the name
  of your model needs to start with `projects/`.
</Note>

Google Vertex models support also some model specific settings that are not part
of the [standard call settings](/docs/ai-sdk-core/settings). You can pass them as
an options argument:

```ts
const model = vertex('gemini-1.5-pro', {
  safetySettings: [
    { category: 'HARM_CATEGORY_UNSPECIFIED', threshold: 'BLOCK_LOW_AND_ABOVE' },
  ],
});
```

The following optional settings are available for Google Vertex models:

- **structuredOutputs** _boolean_

  Optional. Enable structured output. Default is true.

  This is useful when the JSON Schema contains elements that are
  not supported by the OpenAPI schema version that
  Google Vertex uses. You can use this to disable
  structured outputs if you need to.

  See [Troubleshooting: Schema Limitations](#troubleshooting-schema-limitations) for more details.

- **safetySettings** _Array\<\{ category: string; threshold: string \}\>_

  Optional. Safety settings for the model.

  - **category** _string_

    The category of the safety setting. Can be one of the following:

    - `HARM_CATEGORY_UNSPECIFIED`
    - `HARM_CATEGORY_HATE_SPEECH`
    - `HARM_CATEGORY_DANGEROUS_CONTENT`
    - `HARM_CATEGORY_HARASSMENT`
    - `HARM_CATEGORY_SEXUALLY_EXPLICIT`

  - **threshold** _string_

    The threshold of the safety setting. Can be one of the following:

    - `HARM_BLOCK_THRESHOLD_UNSPECIFIED`
    - `BLOCK_LOW_AND_ABOVE`
    - `BLOCK_MEDIUM_AND_ABOVE`
    - `BLOCK_ONLY_HIGH`
    - `BLOCK_NONE`

- **useSearchGrounding** _boolean_

  Optional. When enabled, the model will [use Google search to ground the response](https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview).

You can use Google Vertex language models to generate text with the `generateText` function:

```ts highlight="1,5"
import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-pro'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Google Vertex language models can also be used in the `streamText` and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### File Inputs

The Google Vertex provider supports file inputs, e.g. PDF files.

```ts
import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-pro'),
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'What is an embedding model according to this document?',
        },
        {
          type: 'file',
          data: fs.readFileSync('./data/ai.pdf'),
          mimeType: 'application/pdf',
        },
      ],
    },
  ],
});
```

<Note>
  The AI SDK will automatically download URLs if you pass them as data, except
  for `gs://` URLs. You can use the Google Cloud Storage API to upload larger
  files to that location.
</Note>

See [File Parts](/docs/foundations/prompts#file-parts) for details on how to use files in prompts.

### Search Grounding

With [search grounding](https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview),
the model has access to the latest information using Google search.
Search grounding can e.g. be used to provide answers around current events:

```ts highlight="6,13-15"
import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text, experimental_providerMetadata } = await generateText({
  model: vertex('gemini-1.5-pro', {
    useSearchGrounding: true,
  }),
  prompt:
    'List the top 5 San Francisco news from the past week.' +
    'You must include the date of each article.',
});

// access the grounding metadata
const groundingMetadata =
  experimental_providerMetadata?.vertex.groundingMetadata;
```

### Troubleshooting

#### Schema Limitations

The Google Vertex API uses a subset of the OpenAPI 3.0 schema,
which does not support features such as unions.
The errors that you get in this case look like this:

`GenerateContentRequest.generation_config.response_schema.properties[occupation].type: must be specified`

By default, structured outputs are enabled (and for tool calling they are required).
You can disable structured outputs for object generation as a workaround:

```ts highlight="3,8"
const result = await generateObject({
  model: vertex('gemini-1.5-pro', {
    structuredOutputs: false,
  }),
  schema: z.object({
    name: z.string(),
    age: z.number(),
    contact: z.union([
      z.object({
        type: z.literal('email'),
        value: z.string(),
      }),
      z.object({
        type: z.literal('phone'),
        value: z.string(),
      }),
    ]),
  }),
  prompt: 'Generate an example person for testing.',
});
```

### Model Capabilities

| Model              | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------ | ------------------- | ------------------- | ------------------- | ------------------- |
| `gemini-1.5-flash` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gemini-1.5-pro`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

## Embedding Models

You can create models that call the Google Vertex AI embeddings API using the `.textEmbeddingModel()` factory method:

```ts
const model = vertex.textEmbeddingModel('text-embedding-004');
```

Google Vertex AI embedding models support additional settings. You can pass them as an options argument:

```ts
const model = vertex.textEmbeddingModel('text-embedding-004', {
  outputDimensionality: 512, // optional, number of dimensions for the embedding
});
```

The following optional settings are available for Google Vertex AI embedding models:

- **outputDimensionality**: _number_

  Optional reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end.

### Model Capabilities

| Model                | Max Values Per Call | Parallel Calls      |
| -------------------- | ------------------- | ------------------- |
| `text-embedding-004` | 2048                | <Check size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

---
title: Mistral AI
description: Learn how to use Mistral.
---

# Mistral AI Provider

The [Mistral AI](https://mistral.ai/) provider contains language model support for the Mistral chat API.

## Setup

The Mistral provider is available in the `@ai-sdk/mistral` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/mistral" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/mistral" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/mistral" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `mistral` from `@ai-sdk/mistral`:

```ts
import { mistral } from '@ai-sdk/mistral';
```

If you need a customized setup, you can import `createMistral` from `@ai-sdk/mistral`
and create a provider instance with your settings:

```ts
import { createMistral } from '@ai-sdk/mistral';

const mistral = createMistral({
  // custom settings
});
```

You can use the following optional settings to customize the Mistral provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.mistral.ai/v1`.

- **apiKey** _string_

  API key that is being sent using the `Authorization` header.
  It defaults to the `MISTRAL_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create models that call the [Mistral chat API](https://docs.mistral.ai/api/#operation/createChatCompletion) using a provider instance.
The first argument is the model id, e.g. `mistral-large-latest`.
Some Mistral chat models support tool calls.

```ts
const model = mistral('mistral-large-latest');
```

Mistral chat models also support additional model settings that are not part of the [standard call settings](/docs/ai-sdk-core/settings).
You can pass them as an options argument:

```ts
const model = mistral('mistral-large-latest', {
  safePrompt: true, // optional safety prompt injection
});
```

The following optional settings are available for Mistral models:

- **safePrompt** _boolean_

  Whether to inject a safety prompt before all conversations.

  Defaults to `false`.

### Example

You can use Mistral language models to generate text with the `generateText` function:

```ts
import { mistral } from '@ai-sdk/mistral';
import { generateText } from 'ai';

const { text } = await generateText({
  model: mistral('mistral-large-latest'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Mistral language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### Model Capabilities

| Model                  | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ---------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `pixtral-large-latest` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `mistral-large-latest` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `mistral-small-latest` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `ministral-3b-latest`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `ministral-8b-latest`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `pixtral-12b-2409`     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  The table above lists popular models. Please see the [Mistral
  docs](https://docs.mistral.ai/getting-started/models/models_overview/) for a
  full list of available models.
</Note>

## Embedding Models

You can create models that call the [Mistral embeddings API](https://docs.mistral.ai/api/#operation/createEmbedding)
using the `.embedding()` factory method.

```ts
const model = mistral.embedding('mistral-embed');
```

### Model Capabilities

| Model           | Default Dimensions |
| --------------- | ------------------ |
| `mistral-embed` | 1024               |

---
title: xAI Grok
description: Learn how to use xAI Grok.
---

# xAI Grok Provider

The [xAI Grok](https://x.ai) provider contains language model support for the [xAI API](https://x.ai/api).

## Setup

The xAI Grok provider is available via the `@ai-sdk/xai` module. You can
install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/xai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/xai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/xai" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `xai` from `@ai-sdk/xai`:

```ts
import { xai } from '@ai-sdk/xai';
```

If you need a customized setup, you can import `createXai` from `@ai-sdk/xai`
and create a provider instance with your settings:

```ts
import { createXai } from '@ai-sdk/xai';

const xai = createXai({
  apiKey: 'your-api-key',
});
```

You can use the following optional settings to customize the xAI provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.x.ai/v1`.

- **apiKey** _string_

  API key that is being send using the `Authorization` header. It defaults to
  the `XAI_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create [xAI models](https://console.x.ai) using a provider instance. The
first argument is the model id, e.g. `grok-beta`.

```ts
const model = xai('grok-beta');
```

### Example

You can use xAI language models to generate text with the `generateText` function:

```ts
import { xai } from '@ai-sdk/xai';
import { generateText } from 'ai';

const { text } = await generateText({
  model: xai('grok-beta'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

xAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### Chat Models

xAI chat models also support some model specific settings that are not part of
the [standard call settings](/docs/ai-sdk-core/settings). You can pass them as
an options argument:

```ts
const model = xai('grok-beta', {
  user: 'test-user', // optional unique user identifier
});
```

The following optional settings are available for xAI chat models:

- **user** _string_

  A unique identifier representing your end-user, which can help xAI to
  monitor and detect abuse.

## Model Capabilities

xAI available models are listed in [the xAI console](https://console.x.ai/)
under "View models", including:

| Model              | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------ | ------------------- | ------------------- | ------------------- | ------------------- |
| `grok-beta`        | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `grok-vision-beta` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  You can also pass any available provider model ID as a string if needed.
</Note>

---
title: Cohere
description: Learn how to use the Cohere provider for the AI SDK.
---

# Cohere Provider

The [Cohere](https://cohere.com/) provider contains language and emdedding model support for the Cohere chat API.

## Setup

The Cohere provider is available in the `@ai-sdk/cohere` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/cohere" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/cohere" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/cohere" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `cohere` from `@ai-sdk/cohere`:

```ts
import { cohere } from '@ai-sdk/cohere';
```

If you need a customized setup, you can import `createCohere` from `@ai-sdk/cohere`
and create a provider instance with your settings:

```ts
import { createCohere } from '@ai-sdk/cohere';

const cohere = createCohere({
  // custom settings
});
```

You can use the following optional settings to customize the Cohere provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.cohere.com/v2`.

- **apiKey** _string_

  API key that is being sent using the `Authorization` header.
  It defaults to the `COHERE_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create models that call the [Cohere chat API](https://docs.cohere.com/v2/docs/chat-api) using a provider instance.
The first argument is the model id, e.g. `command-r-plus`.
Some Cohere chat models support tool calls.

```ts
const model = cohere('command-r-plus');
```

### Example

You can use Cohere language models to generate text with the `generateText` function:

```ts
import { cohere } from '@ai-sdk/cohere';
import { generateText } from 'ai';

const { text } = await generateText({
  model: cohere('command-r-plus'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Cohere language models can also be used in the `streamText` function (see [AI SDK Core](/docs/ai-sdk-core)).

### Model Capabilities

| Model            | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ---------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `command-r-plus` | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed. For more see the [Cohere models
  overview](https://docs.cohere.com/v2/docs/models).
</Note>

## Embedding Models

You can create models that call the [Cohere embed API](https://docs.cohere.com/v2/reference/embed)
using the `.embedding()` factory method.

```ts
const model = cohere.embedding('embed-english-v3.0');
```

Cohere embedding models support additional settings. You can pass them as an options argument:

```ts
const model = cohere.embedding('embed-english-v3.0', {
  inputType: 'search_document',
});
```

The following optional settings are available for Cohere embedding models:

- **inputType** _'search_document' | 'search_query' | 'classification' | 'clustering'_

  Specifies the type of input passed to the model. Default is `search_query`.

  - `search_document`: Used for embeddings stored in a vector database for search use-cases.
  - `search_query`: Used for embeddings of search queries run against a vector DB to find relevant documents.
  - `classification`: Used for embeddings passed through a text classifier.
  - `clustering`: Used for embeddings run through a clustering algorithm.

- **truncate** _'NONE' | 'START' | 'END'_

  Specifies how the API will handle inputs longer than the maximum token length.
  Default is `END`.

  - `NONE`: If selected, when the input exceeds the maximum input token length will return an error.
  - `START`: Will discard the start of the input until the remaining input is exactly the maximum input token length for the model.
  - `END`: Will discard the end of the input until the remaining input is exactly the maximum input token length for the model.

### Model Capabilities

| Model                           | Embedding Dimensions |
| ------------------------------- | -------------------- |
| `embed-english-v3.0`            | 1024                 |
| `embed-multilingual-v3.0`       | 1024                 |
| `embed-english-light-v3.0`      | 384                  |
| `embed-multilingual-light-v3.0` | 384                  |
| `embed-english-v2.0`            | 4096                 |
| `embed-english-light-v2.0`      | 1024                 |
| `embed-multilingual-v2.0`       | 768                  |

---
title: Groq
description: Learn how to use Groq.
---

# Groq Provider

The [Groq](https://groq.com/) provider contains language model support for the Groq API.

## Setup

The Groq provider is available via the `@ai-sdk/groq` module.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/groq" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/groq" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/groq" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `groq` from `@ai-sdk/groq`:

```ts
import { groq } from '@ai-sdk/groq';
```

If you need a customized setup, you can import `createGroq` from `@ai-sdk/groq`
and create a provider instance with your settings:

```ts
import { createGroq } from '@ai-sdk/groq';

const groq = createGroq({
  // custom settings
});
```

You can use the following optional settings to customize the Groq provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://api.groq.com/openai/v1`.

- **apiKey** _string_

  API key that is being sent using the `Authorization` header.
  It defaults to the `GROQ_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
  Defaults to the global `fetch` function.
  You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g. testing.

## Language Models

You can create [Groq models](https://console.groq.com/docs/models) using a provider instance.
The first argument is the model id, e.g. `gemma2-9b-it`.

```ts
const model = groq('gemma2-9b-it');
```

### Example

You can use Groq language models to generate text with the `generateText` function:

```ts
import { groq } from '@ai-sdk/groq';
import { generateText } from 'ai';

const { text } = await generateText({
  model: groq('gemma2-9b-it'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

## Model Capabilities

Groq offers [a variety of models with different capabilities](https://console.groq.com/docs/models), including:

| Model                     | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `llama-3.1-70b-versatile` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `llama-3.1-8b-instant`    | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `gemma2-9b-it`            | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `mixtral-8x7b-32768`      | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  The table above lists popular models. You can also pass any available provider
  model ID as a string if needed.
</Note>

---
title: AI SDK Providers
description: Learn how to use AI SDK providers.
---

# AI SDK Providers

The AI SDK comes with several providers that you can use to interact with different language models:

<OfficialModelCards />

There are also [community providers](./community-providers) that have been created using the [Language Model Specification](./community-providers/custom-providers).

<CommunityModelCards />

## Provider support

Not all providers support all AI SDK features. Here's a quick comparison of the capabilities of popular models:

| Provider                                                                 | Model                        | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ------------------------------------------------------------------------ | ---------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4o`                     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4o-mini`                | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4-turbo`                | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `gpt-4`                      | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `o1-preview`                 | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [OpenAI](/providers/ai-sdk-providers/openai)                             | `o1-mini`                    | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-sonnet-20241022` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-sonnet-20240620` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Anthropic](/providers/ai-sdk-providers/anthropic)                       | `claude-3-5-haiku-20241022`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `pixtral-large-latest`       | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `mistral-large-latest`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `mistral-small-latest`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Mistral](/providers/ai-sdk-providers/mistral)                           | `pixtral-12b-2409`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Generative AI](/providers/ai-sdk-providers/google-generative-ai) | `gemini-1.5-flash`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Generative AI](/providers/ai-sdk-providers/google-generative-ai) | `gemini-1.5-pro`             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex)               | `gemini-1.5-flash`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex)               | `gemini-1.5-pro`             | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [xAI Grok](/providers/ai-sdk-providers/xai)                              | `grok-beta`                  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [xAI Grok](/providers/ai-sdk-providers/xai)                              | `grok-vision-beta`           | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-405b-reasoning`   | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-70b-versatile`    | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `llama-3.1-8b-instant`       | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `mixtral-8x7b-32768`         | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| [Groq](/providers/ai-sdk-providers/groq)                                 | `gemma2-9b-it`               | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  This table is not exhaustive. Additional models can be found in the provider
  documentation pages and on the provider websites.
</Note>

---
title: Perplexity
description: Use the Perplexity OpenAI compatible API with the AI SDK.
---

# Perplexity Provider

[Perplexity](https://docs.perplexity.ai/) is a search engine that uses LLMs to answer questions.
It offers an OpenAI compatible API that you can use with the AI SDK.

## Setup

The Perplexity provider is available via the `@ai-sdk/openai` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use Perplexity, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const perplexity = createOpenAI({
  name: 'perplexity',
  apiKey: process.env.PERPLEXITY_API_KEY ?? '',
  baseURL: 'https://api.perplexity.ai/',
});
```

## Language Models

You can create [Perplexity models](https://docs.perplexity.ai/docs/model-cards) using a provider instance.
The first argument is the model id, e.g. `llama-3-sonar-large-32k-online`.

```ts
const model = perplexity('llama-3-sonar-large-32k-online');
```

### Example

You can use Perplexity language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const perplexity = createOpenAI({
  name: 'perplexity',
  apiKey: process.env.PERPLEXITY_API_KEY ?? '',
  baseURL: 'https://api.perplexity.ai/',
});

const { text } = await generateText({
  model: perplexity('llama-3-sonar-large-32k-online'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Perplexity language models can also be used in the `streamText` function.

---
title: Fireworks
description: Use the Fireworks OpenAI compatible API with the AI SDK.
---

# Fireworks Provider

[Fireworks](https://fireworks.ai/) is a platform for running and testing LLMs.
It offers an OpenAI compatible API that you can use with the AI SDK.

## Setup

The Fireworks provider is available via the `@ai-sdk/openai` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use Fireworks, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const fireworks = createOpenAI({
  name: 'fireworks',
  apiKey: process.env.FIREWORKS_API_KEY ?? '',
  baseURL: 'https://api.fireworks.ai/inference/v1',
});
```

## Language Models

You can create [Fireworks models](https://fireworks.ai/models) using a provider instance.
The first argument is the model id, e.g. `accounts/fireworks/models/firefunction-v1`.

```ts
const model = fireworks('accounts/fireworks/models/firefunction-v1');
```

### Example

You can use Fireworks language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai'
import { generateText } from 'ai'

const fireworks = createOpenAI({
  name: 'fireworks',
  apiKey: process.env.FIREWORKS_API_KEY ?? '',
  baseURL: 'https://api.fireworks.ai/inference/v1'
})

const { text } = await generateText({
  model: fireworks('accounts/fireworks/models/firefunction-v1')
  prompt: 'Write a vegetarian lasagna recipe for 4 people.'
})
```

Fireworks language models can also be used in the `streamText`, `generateObject`, and `streamObject` functions.

---
title: Together.ai
description: Use the Together.ai OpenAI compatible API with the AI SDK.
---

# Together.ai Provider

[Together.ai](https://together.ai/) is a platform for running and testing LLMs.
It offers an OpenAI compatible API that you can use with the AI SDK.

## Setup

The Together.ai provider is available via the `@ai-sdk/openai` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use Together.ai, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const togetherai = createOpenAI({
  name: 'togetherai',
  apiKey: process.env.TOGETHER_AI_API_KEY ?? '',
  baseURL: 'https://api.together.xyz/v1/',
});
```

## Language Models

You can create [Together.ai models](https://together.ai/models) using a provider instance.
The first argument is the model id, e.g. `google/gemma-2-9b-it`.

```ts
const model = togetherai('google/gemma-2-9b-it');
```

### Example

You can use Together.ai language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai'
import { generateText } from 'ai'

const togetherai = createOpenAI({
  name: 'togetherai',
  apiKey: process.env.TOGETHER_AI_API_KEY ?? '',
  baseURL: 'https://api.together.xyz/v1/'
})

const { text } = await generateText({
  model: togetherai('google/gemma-2-9b-it')
  prompt: 'Write a vegetarian lasagna recipe for 4 people.'
})
```

Together.ai language models can also be used in the `streamText` function.

---
title: LM Studio
description: Use the LM Studio OpenAI compatible API with the AI SDK.
---

# LM Studio Provider

[LM Studio](https://lmstudio.ai/) is user interface for running local models.

It contains an OpenAI compatible API server that you can use with the AI SDK.
You can start the local server under the [Local Server tab](https://lmstudio.ai/docs/basics/server) in the LM Studio UI ("Start Server" button).

## Setup

The LM Studio provider is available via the `@ai-sdk/openai` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use LM Studio, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const lmstudio = createOpenAI({
  name: 'lmstudio',
  baseURL: 'http://localhost:1234/v1',
  maxRetries: 1, // immediately error if the server is not running
});
```

<Note>
  LM Studio uses port `1234` by default, but you can change in the [app's Local
  Server tab](https://lmstudio.ai/docs/basics/server).
</Note>

## Language Models

You can interact with local LLMs in [LM Studio](https://lmstudio.ai/docs/basics/server#endpoints-overview) using a provider instance.
The first argument is the model id, e.g. `llama-3.2-1b`.

```ts
const model = lmstudio('llama-3.2-1b');
```

###### To be able to use a model, you need to [download it first](https://lmstudio.ai/docs/basics/download-model).

### Example

You can use LM Studio language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const lmstudio = createOpenAI({
  name: 'lmstudio',
  apiKey: 'not-needed',
  baseURL: 'https://localhost:1234/v1',
});

const { text } = await generateText({
  model: lmstudio('llama-3.2-1b'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

LM Studio language models can also be used with `streamText`.

## Embedding Models

You can create models that call the [LM Studio embeddings API](https://lmstudio.ai/docs/basics/server#endpoints-overview)
using the `.embedding()` factory method.

```ts
const model = lmstudio.embedding('text-embedding-nomic-embed-text-v1.5');
```

### Example - Embedding a Single Value

```tsx
import { createOpenAI } from '@ai-sdk/openai';
import { embed } from 'ai';

const lmstudio = createOpenAI({
  name: 'lmstudio',
  apiKey: 'not-needed',
  baseURL: 'https://localhost:1234/v1',
});

// 'embedding' is a single embedding object (number[])
const { embedding } = await embed({
  model: lmstudio.embedding('text-embedding-nomic-embed-text-v1.5'),
  value: 'sunny day at the beach',
});
```

### Example - Embedding Many Values

When loading data, e.g. when preparing a data store for retrieval-augmented generation (RAG),
it is often useful to embed many values at once (batch embedding).

The AI SDK provides the [`embedMany`](/docs/reference/ai-sdk-core/embed-many) function for this purpose.
Similar to `embed`, you can use it with embeddings models,
e.g. `lmstudio.embedding('text-embedding-nomic-embed-text-v1.5')` or `lmstudio.embedding('text-embedding-bge-small-en-v1.5')`.

```tsx
import { createOpenAI } from '@ai-sdk/openai';
import { embedMany } from 'ai';

// 'embeddings' is an array of embedding objects (number[][]).
// It is sorted in the same order as the input values.
const { embeddings } = await embedMany({
  model: lmstudio.embedding('text-embedding-nomic-embed-text-v1.5'),
  values: [
    'sunny day at the beach',
    'rainy afternoon in the city',
    'snowy night in the mountains',
  ],
});
```

---
title: Baseten
description: Use a Baseten OpenAI compatible API with the AI SDK.
---

# Baseten Provider

[Baseten](https://baseten.co/) is a platform for running and testing LLMs.
It allows you to deploy models that are OpenAI API compatible that you can use with the AI SDK.

## Setup

The Baseten provider is available via the `@ai-sdk/openai` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use Baseten, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const BASETEN_MODEL_ID = '<deployment-id>';
const BASETEN_DEPLOYMENT_ID = null;

// see https://docs.baseten.co/api-reference/openai for more information
const basetenExtraPayload = {
  model_id: BASETEN_MODEL_ID,
  deployment_id: BASETEN_DEPLOYMENT_ID,
};

const baseten = createOpenAI({
  name: 'baseten',
  apiKey: process.env.BASETEN_API_KEY ?? '',
  baseURL: 'https://bridge.baseten.co/v1/direct',
  fetch: async (url, request) => {
    const bodyWithBasetenPayload = JSON.stringify({
      ...JSON.parse(request.body),
      baseten: basetenExtraPayload,
    });
    return await fetch(url, { ...request, body: bodyWithBasetenPayload });
  },
});
```

Be sure to have your `BASETEN_API_KEY` set in your environment and the model `deployment id` ready. The `deployment_id` will be given after you have deployed the model on Baseten.

## Language Models

You can create [Baseten models](https://baseten.co/models) using a provider instance.
The first argument is the served model name, e.g. `ultravox`.

```ts
const model = baseten('ultravox');
```

### Example

You can use Baseten language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const BASETEN_MODEL_ID = '<deployment-id>';
const BASETEN_DEPLOYMENT_ID = null;

// see https://docs.baseten.co/api-reference/openai for more information
const basetenExtraPayload = {
  model_id: BASETEN_MODEL_ID,
  deployment_id: BASETEN_DEPLOYMENT_ID,
};

const baseten = createOpenAI({
  name: 'baseten',
  apiKey: process.env.BASETEN_API_KEY ?? '',
  baseURL: 'https://bridge.baseten.co/v1/direct',
  fetch: async (url, request) => {
    const bodyWithBasetenPayload = JSON.stringify({
      ...JSON.parse(request.body),
      baseten: basetenExtraPayload,
    });
    return await fetch(url, { ...request, body: bodyWithBasetenPayload });
  },
});

const { text } = await generateText({
  model: baseten('ultravox'),
  prompt: 'Tell me about yourself in one sentence',
});

console.log(text);
```

Baseten language models are also able to generate text in a streaming fashion with the `streamText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

const BASETEN_MODEL_ID = '<deployment-id>';
const BASETEN_DEPLOYMENT_ID = null;

// see https://docs.baseten.co/api-reference/openai for more information
const basetenExtraPayload = {
  model_id: BASETEN_MODEL_ID,
  deployment_id: BASETEN_DEPLOYMENT_ID,
};

const baseten = createOpenAI({
  name: 'baseten',
  apiKey: process.env.BASETEN_API_KEY ?? '',
  baseURL: 'https://bridge.baseten.co/v1/direct',
  fetch: async (url, request) => {
    const bodyWithBasetenPayload = JSON.stringify({
      ...JSON.parse(request.body),
      baseten: basetenExtraPayload,
    });
    return await fetch(url, { ...request, body: bodyWithBasetenPayload });
  },
});

const result = streamText({
  model: baseten('ultravox'),
  prompt: 'Tell me about yourself in one sentence',
});

for await (const message of result.textStream) {
  console.log(message);
}
```

Baseten language models can also be used in the `generateObject`, and `streamObject` functions.

---
title: OpenAI compatible Providers
description: Use OpenAI compatible providers with the AI SDK.
---

# OpenAI compatible Providers

You can use the [OpenAI Provider](/providers/ai-sdk-providers/openai) to use language model providers that implement the OpenAI API.
We provide detailed documentation for the following OpenAI compatible providers:

- [Perplexity](/providers/openai-compatible-providers/perplexity)
- [Fireworks](/providers/openai-compatible-providers/fireworks)
- [TogetherAI](/providers/openai-compatible-providers/togetherai)
- [LM Studio](/providers/openai-compatible-providers/lmstudio)
- [Baseten](/providers/openai-compatible-providers/baseten)

The general setup and provider instance creation is the same for all OpenAI compatible providers.

## Setup

The OpenAI provider is available via the `@ai-sdk/openai` module. You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai" dark />
  </Tab>
</Tabs>

## Provider Instance

To use an OpenAI compatible provider, you can create a custom provider instance with the `createOpenAI` function from `@ai-sdk/openai`:

```ts
import { createOpenAI } from '@ai-sdk/openai';

const provider = createOpenAI({
  name: 'provider-name',
  apiKey: process.env.PROVIDER_API_KEY ?? '',
  baseURL: 'https://api.provider.com/v1',
});
```

## Language Models

You can create provider models using a provider instance.
The first argument is the model id, e.g. `model-id`.

```ts
const model = provider('model-id');
```

### Example

You can use provider language models to generate text with the `generateText` function:

```ts
import { createOpenAI } from '@ai-sdk/openai'
import { generateText } from 'ai'

const provider = createOpenAI({
  name: 'provider-name',
  apiKey: process.env.PROVIDER_API_KEY ?? '',
  baseURL: 'https://api.provider.com/v1'
})

const { text } = await generateText({
  model: provider('model-id')
  prompt: 'Write a vegetarian lasagna recipe for 4 people.'
})
```

---
title: Writing a Custom Provider
description: Learn how to write a custom provider for the AI SDK
---

# Writing a Custom Provider

The AI SDK provides a [Language Model Specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1).
You can write your own provider that adheres to the specification and it will be compatible with the AI SDK.

You can find the Language Model Specification in the [AI SDK repository](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1).
It can be imported from `'@ai-sdk/provider'`. We also provide utilities that make it easier to implement a custom provider. You can find them in the `@ai-sdk/provider-utils` package ([source code](https://github.com/vercel/ai/tree/main/packages/provider-utils)).

<Note>
  If you open-source a provider, we'd love to promote it here. Please send us a
  PR to add it to the [Community Providers](/providers/community-providers)
  section.
</Note>

## Provider Implementation Guide

Implementing a custom language model provider involves several steps:

- Creating an entry point
- Adding a language model implementation
- Mapping the input (prompt, tools, settings)
- Processing the results (generate, streaming, tool calls)
- Supporting object generation

The best way to get started is to copy a reference implementation and modify it to fit your needs.
Check out the [Mistral reference implementation](https://github.com/vercel/ai/tree/main/packages/mistral)
to see how the project is structured, and feel free to copy the setup.

### Creating an Entry Point

Each AI SDK provider should follow the pattern of using a factory function that returns a provider instance
and provide a default instance.

```ts filename="custom-provider.ts"
import {
  generateId,
  loadApiKey,
  withoutTrailingSlash,
} from '@ai-sdk/provider-utils';
import { CustomChatLanguageModel } from './custom-chat-language-model';
import { CustomChatModelId, CustomChatSettings } from './custom-chat-settings';

// model factory function with additional methods and properties
export interface CustomProvider {
  (
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ): CustomChatLanguageModel;

  // explicit method for targeting a specific API in case there are several
  chat(
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ): CustomChatLanguageModel;
}

// optional settings for the provider
export interface CustomProviderSettings {
  /**
Use a different URL prefix for API calls, e.g. to use proxy servers.
   */
  baseURL?: string;

  /**
API key.
   */
  apiKey?: string;

  /**
Custom headers to include in the requests.
     */
  headers?: Record<string, string>;
}

// provider factory function
export function createCustomProvider(
  options: CustomProviderSettings = {},
): CustomProvider {
  const createModel = (
    modelId: CustomChatModelId,
    settings: CustomChatSettings = {},
  ) =>
    new CustomChatLanguageModel(modelId, settings, {
      provider: 'custom.chat',
      baseURL:
        withoutTrailingSlash(options.baseURL) ?? 'https://custom.ai/api/v1',
      headers: () => ({
        Authorization: `Bearer ${loadApiKey({
          apiKey: options.apiKey,
          environmentVariableName: 'CUSTOM_API_KEY',
          description: 'Custom Provider',
        })}`,
        ...options.headers,
      }),
      generateId: options.generateId ?? generateId,
    });

  const provider = function (
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ) {
    if (new.target) {
      throw new Error(
        'The model factory function cannot be called with the new keyword.',
      );
    }

    return createModel(modelId, settings);
  };

  provider.chat = createModel;

  return provider as CustomProvider;
}

/**
 * Default custom provider instance.
 */
export const customProvider = createCustomProvider();
```

### Implementing the Language Model

A [language model](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts) needs to implement:

- metadata fields
  - `specificationVersion: 'v1'` - always `'v1'`
  - `provider: string` - name of the provider
  - `modelId: string` - unique identifier of the model
  - `defaultObjectGenerationMode` - default object generation mode, e.g. "json"
- `doGenerate` method
- `doStream` method

Check out the [Mistral language model](https://github.com/vercel/ai/blob/main/packages/mistral/src/mistral-chat-language-model.ts) as an example.

At a high level, both `doGenerate` and `doStream` methods should:

1. **Map the prompt and the settings to the format required by the provider API.** This can be extracted, e.g. the Mistral provider contains a `getArgs` method.
2. **Call the provider API.** You could e.g. use fetch calls or a library offered by the provider.
3. **Process the results.** You need to convert the response to the format required by the AI SDK.

### Errors

The AI SDK provides [standardized errors](https://github.com/vercel/ai/tree/main/packages/provider/src/errors) that should be used by providers where possible. This will make it easy for user to debug them.

### Retries, timeouts, and abort signals

The AI SDK will handle retries, timeouts, and aborting requests in a unified way. The model classes should not implement retries or timeouts themselves. Instead, they should use the `abortSignal` parameter to determine when the call should be aborted, and they should throw `ApiCallErrors` (or similar) with a correct `isRetryable` flag when errors such as network errors occur.

---
title: Ollama
description: Learn how to use the Ollama provider.
---

# Ollama Provider

[sgomez/ollama-ai-provider](https://github.com/sgomez/ollama-ai-provider) is a community provider that uses [Ollama](https://ollama.com/) to provide language model support for the AI SDK.

## Setup

The Ollama provider is available in the `ollama-ai-provider` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add ollama-ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install ollama-ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add ollama-ai-provider" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `ollama` from `ollama-ai-provider`:

```ts
import { ollama } from 'ollama-ai-provider';
```

If you need a customized setup, you can import `createOllama` from `ollama-ai-provider` and create a provider instance with your settings:

```ts
import { createOllama } from 'ollama-ai-provider';

const ollama = createOllama({
  // optional settings, e.g.
  baseURL: 'https://api.ollama.com',
});
```

You can use the following optional settings to customize the Ollama provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `http://localhost:11434/api`.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

## Language Models

You can create models that call the [Ollama Chat Completion API](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion) using the provider instance.
The first argument is the model id, e.g. `phi3`. Some models have multi-modal capabilities.

```ts
const model = ollama('phi3');
```

You can find more models on the [Ollama Library](https://ollama.com/library) homepage.

### Model Capabilities

This provider is capable of generating and streaming text and objects. Object generation may fail depending
on the model used and the schema used.

The following models have been tested with image inputs:

- llava
- llava-llama3
- llava-phi3
- moondream

## Embedding Models

You can create models that call the [Ollama embeddings API](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings)
using the `.embedding()` factory method.

```ts
const model = ollama.embedding('nomic-embed-text');
```

---
title: Chrome AI
description: Learn how to use the Chrome built-in Language Model.
---

# ChromeAI

[jeasonstudio/chrome-ai](https://github.com/jeasonstudio/chrome-ai) is a community provider that uses [Chrome Built-in AI](https://developer.chrome.com/docs/ai/built-in) to provide language model support for the AI SDK.

<Note type="warning">
  This module is under development and may contain errors and frequent
  incompatible changes.
</Note>

## Setup

The ChromeAI provider is available in the `chrome-ai` module. You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add chrome-ai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install chrome-ai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add chrome-ai" dark />
  </Tab>
</Tabs>

### Enabling AI in Chrome

<Note type="warning">
  Chrome's implementation of [built-in AI with Gemini
  Nano](https://developer.chrome.com/docs/ai/built-in) is experimental and will
  change as they test and address feedback.
</Note>

Chrome built-in AI is a preview feature, you need to use chrome version 127 or greater, now in [dev](https://www.google.com/chrome/dev/?extra=devchannel) or [canary](https://www.google.com/chrome/canary/) channel, [may release on stable chanel at Jul 17, 2024](https://chromestatus.com/roadmap).

After then, you should turn on these flags:

- [chrome://flags/#prompt-api-for-gemini-nano](chrome://flags/#prompt-api-for-gemini-nano): `Enabled`
- [chrome://flags/#optimization-guide-on-device-model](chrome://flags/#optimization-guide-on-device-model): `Enabled BypassPrefRequirement`
- [chrome://components/](chrome://components/): Click `Optimization Guide On Device Model` to download the model.

## Language Models

The `chromeai` provider instance is a function that you can invoke to create a language model:

```ts
import { chromeai } from 'chrome-ai';

const model = chromeai();
```

It automatically selects the correct model id. You can also pass additional settings in the second argument:

```ts
import { chromeai } from 'chrome-ai';

const model = chromeai('generic', {
  // additional settings
  temperature: 0.5,
  topK: 5,
});
```

You can use the following optional settings to customize:

- **modelId** _'text' | 'generic'_

  Used to distinguish models of Gemini Nano, there is no difference in the current version.

- **temperature** _number_

  The value is passed through to the provider. The range depends on the provider and model.
  For most providers, `0` means almost deterministic results, and higher values mean more randomness.

- **topK** _number_

  Only sample from the top K options for each subsequent token.

  Used to remove "long tail" low probability responses.
  Recommended for advanced use cases only. You usually only need to use temperature.

## Examples

You can use Chrome built-in language models to generate text with the `generateText` or `streamText` function:

```javascript
import { generateText } from 'ai';
import { chromeai } from 'chrome-ai';

const { text } = await generateText({
  model: chromeai(),
  prompt: 'Who are you?',
});

console.log(text); //  I am a large language model, trained by Google.
```

```javascript
import { streamText } from 'ai';
import { chromeai } from 'chrome-ai';

const { textStream } = streamText({
  model: chromeai(),
  prompt: 'Who are you?',
});

let result = '';
for await (const textPart of textStream) {
  result = textPart;
}

console.log(result);
//  I am a large language model, trained by Google.
```

Chrome built-in language models can also be used in the `generateObject` function:

```javascript
import { generateObject } from 'ai';
import { chromeai } from 'chrome-ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: chromeai('text'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(
        z.object({
          name: z.string(),
          amount: z.string(),
        }),
      ),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(object);
// { recipe: {...} }
```

<Note>
  Due to model reasons, `toolCall` and `streamObject` are not supported. We are
  making an effort to implement these functions by prompt engineering.
</Note>

---
title: AnthropicVertex
description: Learn how to use Anthropic Vertex.
---

# AnthropicVertex Provider

[nalaso/anthropic-vertex-ai](https://github.com/nalaso/anthropic-vertex-ai) is a community provider that uses Anthropic models through Vertex AI to provide language model support for the AI SDK.

## Setup

The AnthropicVertex provider is available in the `anthropic-vertex-ai` module. You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add anthropic-vertex-ai" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install anthropic-vertex-ai" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add anthropic-vertex-ai" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `anthropicVertex` from `anthropic-vertex-ai`:

```ts
import { anthropicVertex } from 'anthropic-vertex-ai';
```

If you need a customized setup, you can import `createAnthropicVertex` from `anthropic-vertex-ai` and create a provider instance with your settings:

```ts
import { createAnthropicVertex } from 'anthropic-vertex-ai';

const anthropicVertex = createAnthropicVertex({
  region: 'us-central1',
  projectId: 'your-project-id',
  // other options
});
```

You can use the following optional settings to customize the AnthropicVertex provider instance:

- **region** _string_

  Your Google Vertex region. Defaults to the `GOOGLE_VERTEX_REGION` environment variable.

- **projectId** _string_

  Your Google Vertex project ID. Defaults to the `GOOGLE_VERTEX_PROJECT_ID` environment variable.

- **googleAuth** _GoogleAuth_

  Optional. The Authentication options provided by google-auth-library.

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g., to use proxy servers.
  The default prefix is `https://{region}-aiplatform.googleapis.com/v1`.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom fetch implementation. You can use it as a middleware to intercept requests,
  or to provide a custom fetch implementation for e.g., testing.

## Language Models

You can create models that call the Anthropic API through Vertex AI using the provider instance.
The first argument is the model ID, e.g., `claude-3-sonnet@20240229`:

```ts
const model = anthropicVertex('claude-3-sonnet@20240229');
```

### Example: Generate Text

You can use AnthropicVertex language models to generate text with the `generateText` function:

```ts
import { anthropicVertex } from 'anthropic-vertex-ai';
import { generateText } from 'ai';

const { text } = await generateText({
  model: anthropicVertex('claude-3-sonnet@20240229'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

AnthropicVertex language models can also be used in the `streamText`, `generateObject`, and `streamObject` functions
(see [AI SDK Core](/docs/ai-sdk-core) for more information).

### Model Capabilities

| Model                        | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ---------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `claude-3-5-sonnet@20240620` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `claude-3-opus@20240229`     | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `claude-3-sonnet@20240229`   | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `claude-3-haiku@20240307`    | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

## Environment Variables

To use the AnthropicVertex provider, you need to set up the following environment variables:

- `GOOGLE_VERTEX_REGION`: Your Google Vertex region (e.g., 'us-central1')
- `GOOGLE_VERTEX_PROJECT_ID`: Your Google Cloud project ID

Make sure to set these variables in your environment or in a `.env` file in your project root.

## Authentication

The AnthropicVertex provider uses Google Cloud authentication. Make sure you have set up your Google Cloud credentials properly. You can either use a service account key file or default application credentials.

For more information on setting up authentication, refer to the [Google Cloud Authentication guide](https://cloud.google.com/docs/authentication).

---
title: FriendliAI
description: Learn how to use the Friendli Provider for the AI SDK.
---

# Friendli Provider

The [FriendliAI](https://friendli.ai/) provider supports both open-source LLMs via [Friendli Serverless Endpoints](https://friendli.ai/products/serverless-endpoints) and custom models via [Dedicated Endpoints](https://friendli.ai/products/dedicated-endpoints).

It creates language model objects that can be used with the `generateText`, `streamText`, `generateObject`, and `streamObject` functions.

## Setup

The Friendli provider is available via the `@friendliai/ai-provider` module.
You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @friendliai/ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @friendliai/ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @friendliai/ai-provider" dark />
  </Tab>
</Tabs>

### Credentials

The tokens required for model usage can be obtained from the [Friendli suite](https://suite.friendli.ai/).

To use the provider, you need to set the `FRIENDLI_TOKEN` environment variable with your personal access token.

```bash
export FRIENDLI_TOKEN="YOUR_FRIENDLI_TOKEN"
```

Check the [FriendliAI documentation](https://friendli.ai/docs/guides/personal_access_tokens) for more information.

## Provider Instance

You can import the default provider instance `friendliai` from `@friendliai/ai-provider`:

```ts
import { friendli } from '@friendliai/ai-provider';
```

## Language Models

You can create [FriendliAI models](https://friendli.ai/docs/guides/serverless_endpoints/text_generation#model-supports) using a provider instance.
The first argument is the model id, e.g. `meta-llama-3.1-8b-instruct`.

```ts
const model = friendli('meta-llama-3.1-70b-instruct');
```

### Example: Generating text

You can use FriendliAI language models to generate text with the `generateText` function:

```ts
import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai'

const { text } = await generateText({
  model: friendli('meta-llama-3.1-8b-instruct')
  prompt: 'What is the meaning of life?',
})
```

### Example: Using built-in tools

<Note type="warning">Built-in tools are currently in beta.</Note>

If you use `@friendliai/ai-provider`, you can use the [built-in tools](https://friendli.ai/docs/guides/serverless_endpoints/tools/built_in_tools) via the `tools` option.

Built-in tools allow models to use tools to generate better answers. For example, a `web:search` tool can provide up-to-date answers to current questions.

```ts highlight="1,8,9,10,11,12,13,14"
import { friendli } from '@friendliai/ai-provider';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: friendli('meta-llama-3.1-70b-instruct', {
      tools: [
        { type: 'web:search' },
        { type: 'math:calculator' },
        { type: 'code:python-interpreter' }, // and more tools..!!
      ],
    }),
    messages,
  });

  return result.toDataStreamResponse();
}
```

### Example: Generating text with Dedicated Endpoints

To use a custom model via a dedicated endpoint, you can use the `friendli.dedicated` instance with the endpoint id, e.g. `zbimjgovmlcb`

```ts
import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai';

const { text } = await generateText({
  model: friendli.dedicated('YOUR_ENDPOINT_ID'),
  prompt: 'What is the meaning of life?',
});
```

FriendliAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions.
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

### Model Capabilities

| Model                         | Image Input         | Object Generation   | Tool Usage          | Tool Streaming      |
| ----------------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
| `meta-llama-3.1-70b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `meta-llama-3.1-8b-instruct`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `mixtral-8x7b-instruct-v0-1`  | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
  To access [more models](https://friendli.ai/models), visit the [Friendli
  Dedicated Endpoints
  documentation](https://friendli.ai/docs/guides/dedicated_endpoints/quickstart)
  to deploy your custom models.
</Note>

### OpenAI Compatibility

You can also use `@ai-sdk/openai` as the APIs are OpenAI-compatible.

```ts
import { createOpenAI } from '@ai-sdk/openai';

const friendli = createOpenAI({
  baseURL: 'https://api.friendli.ai/serverless/v1',
  apiKey: process.env.FRIENDLI_TOKEN,
});
```

If you are using dedicated endpoints

```ts
import { createOpenAI } from '@ai-sdk/openai';

const friendli = createOpenAI({
  baseURL: 'https://api.friendli.ai/dedicated/v1',
  apiKey: process.env.FRIENDLI_TOKEN,
});
```

---
title: Portkey
description: Portkey Provider for the AI SDK
---

# Portkey Provider

[Portkey](https://portkey.ai/?utm_source=vercel&utm_medium=docs&utm_campaign=integration) natively integrates with the AI SDK to make your apps production-ready and reliable. Import Portkey's Vercel package and use it as a provider in your Vercel AI app to enable all of Portkey's features:

- Full-stack **observability** and **tracing** for all requests
- Interoperability across **250+ LLMs**
- Built-in **50+** state-of-the-art guardrails
- Simple & semantic **caching** to save costs & time
- Conditional request routing with fallbacks, load-balancing, automatic retries, and more
- Continuous improvement based on user feedback

Learn more at [Portkey docs for the AI SDK](https://docs.portkey.ai/docs/integrations/libraries/vercel)

## Setup

The Portkey provider is available in the `@portkey-ai/vercel-provider` module. You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @portkey-ai/vercel-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @portkey-ai/vercel-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @portkey-ai/vercel-provider" dark />
  </Tab>
</Tabs>

## Provider Instance

To create a Portkey provider instance, use the `createPortkey` function:

```typescript
import { createPortkey } from '@portkey-ai/vercel-provider';

const portkeyConfig = {
  provider: 'openai', //enter provider of choice
  api_key: 'OPENAI_API_KEY', //enter the respective provider's api key
  override_params: {
    model: 'gpt-4', //choose from 250+ LLMs
  },
};

const portkey = createPortkey({
  apiKey: 'YOUR_PORTKEY_API_KEY',
  config: portkeyConfig,
});
```

You can find your Portkey API key in the [Portkey Dashboard](https://app.portkey.ai).

## Language Models

Portkey supports both chat and completion models. Use `portkey.chatModel()` for chat models and `portkey.completionModel()` for completion models:

```typescript
const chatModel = portkey.chatModel('');
const completionModel = portkey.completionModel('');
```

Note: You can provide an empty string as the model name if you've defined it in the `portkeyConfig`.

## Examples

You can use Portkey language models with the `generateText` or `streamText` function:

### `generateText`

```javascript
import { createPortkey } from '@portkey-ai/vercel-provider';
import { generateText } from 'ai';

const portkey = createPortkey({
  apiKey: 'YOUR_PORTKEY_API_KEY',
  config: portkeyConfig,
});

const { text } = await generateText({
  model: portkey.chatModel(''),
  prompt: 'What is Portkey?',
});

console.log(text);
```

### `streamText`

```javascript
import { createPortkey } from '@portkey-ai/vercel-provider';
import { streamText } from 'ai';

const portkey = createPortkey({
  apiKey: 'YOUR_PORTKEY_API_KEY',
  config: portkeyConfig,
});

const result = streamText({
  model: portkey.completionModel(''),
  prompt: 'Invent a new holiday and describe its traditions.',
});

for await (const chunk of result) {
  console.log(chunk);
}
```

Note:

- Portkey supports `Tool` use with the AI SDK
- `generatObject` and `streamObject` are currently not supported.

## Advanced Features

Portkey offers several advanced features to enhance your AI applications:

1. **Interoperability**: Easily switch between 250+ AI models by changing the provider and model name in your configuration.

2. **Observability**: Access comprehensive analytics and logs for all your requests.

3. **Reliability**: Implement caching, fallbacks, load balancing, and conditional routing.

4. **Guardrails**: Enforce LLM behavior in real-time with input and output checks.

5. **Security and Compliance**: Set budget limits and implement fine-grained user roles and permissions.

For detailed information on these features and advanced configuration options, please refer to the [Portkey documentation](https://docs.portkey.ai/docs/integrations/libraries/vercel).

## Additional Resources

- [Portkey Documentation](https://docs.portkey.ai/docs/integrations/libraries/vercel)
- [Twitter](https://twitter.com/portkeyai)
- [Discord Community](https://discord.gg/JHPt4C7r)
- [Portkey Dashboard](https://app.portkey.ai)

---
title: Cloudflare Workers AI
description: Learn how to use the Cloudflare Workers AI provider with the AI SDK.
---

# Cloudflare Workers AI

[threepointone/workers-ai-provider](https://github.com/threepointone/workers-ai-provider/) is a community provider that allows you to use Cloudflare's [Workers AI](https://ai.cloudflare.com/) models with the AI SDK.

## Setup

The Cloudflare Workers AI provider is available in the `workers-ai-provider` module. You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add workers-ai-provider" />
  </Tab>
  <Tab>
    <Snippet text="npm install workers-ai-provider" />
  </Tab>
  <Tab>
    <Snippet text="yarn add workers-ai-provider" />
  </Tab>
</Tabs>

Then, setup an AI binding in your Cloudflare Workers project `wrangler.toml` file:

```bash filename="wrangler.toml"
# ...
[ai]
binding = "AI"
# ...
```

## Provider Instance

To create a `workersai` provider instance, use the `createWorkersAI` function, passing in the AI binding as an option:

```typescript
import { createWorkersAI } from 'workers-ai-provider';

const workersai = createWorkersAI({ binding: env.AI });
```

## Language Models

To create a model instance, call the provider instance and specify the model you would like to use as the first argument. You can also pass additional settings in the second argument:

```typescript highlight="4-7"
import { createWorkersAI } from 'workers-ai-provider';

const workersai = createWorkersAI({ binding: env.AI });
const model = workersai('@cf/meta/llama-3.1-8b-instruct', {
  // additional settings
  safePrompt: true,
});
```

You can use the following optional settings to customize:

- **safePrompt** _boolean_

  Whether to inject a safety prompt before all conversations. Defaults to `false`

### Examples

You can use Cloudflare Workers AI language models to generate text with the **`generateText`** or **`streamText`** function:

#### `generateText`

```typescript
import { createWorkersAI } from 'workers-ai-provider';
import { generateText } from 'ai';

type Env = {
  AI: Ai;
};

export default {
  async fetch(_: Request, env: Env) {
    const workersai = createWorkersAI({ binding: env.AI });
    const result = await generateText({
      model: workersai('@cf/meta/llama-2-7b-chat-int8'),
      prompt: 'Write a 50-word essay about hello world.',
    });

    return new Response(result.text);
  },
};
```

#### `streamText`

```typescript
import { createWorkersAI } from 'workers-ai-provider';
import { streamText } from 'ai';

type Env = {
  AI: Ai;
};

export default {
  async fetch(_: Request, env: Env) {
    const workersai = createWorkersAI({ binding: env.AI });
    const result = streamText({
      model: workersai('@cf/meta/llama-2-7b-chat-int8'),
      prompt: 'Write a 50-word essay about hello world.',
    });

    return result.toTextStreamResponse({
      headers: {
        // add these headers to ensure that the
        // response is chunked and streamed
        'Content-Type': 'text/x-unknown',
        'content-encoding': 'identity',
        'transfer-encoding': 'chunked',
      },
    });
  },
};
```

#### `generateObject`

Some Cloudflare Workers AI language models can also be used with the `generateObject` function:

```typescript
import { createWorkersAI } from 'workers-ai-provider';
import { generateObject } from 'ai';
import { z } from 'zod';

type Env = {
  AI: Ai;
};

export default {
  async fetch(_: Request, env: Env) {
    const workersai = createWorkersAI({ binding: env.AI });
    const result = await generateObject({
      model: workersai('@cf/meta/llama-3.1-8b-instruct'),
      prompt: 'Generate a Lasagna recipe',
      schema: z.object({
        recipe: z.object({
          ingredients: z.array(z.string()),
          description: z.string(),
        }),
      }),
    });

    return Response.json(result.object);
  },
};
```

<Note>`tools` and `streamObject` are currently not supported.</Note>

---
title: Crosshatch
description: Crosshatch Provider for the AI SDK
---

# Crosshatch Provider

The [Crosshatch](https://crosshatch.io) provider supports secure inference from popular language models with permissioned access to data users share, giving responses personalized with complete user context.

It creates language model objects that can be used with the `generateText`, `streamText`, `generateObject` and `streamObject` functions.

## Setup

The Crosshatch provider is available via the `@crosshatch/ai-provider` module.
You can install it with:

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @crosshatch/ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @crosshatch/ai-provider" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @crosshatch/ai-provider" dark />
  </Tab>
</Tabs>

The [Crosshatch](https://crosshatch.io/) provider supports all of their available models such as OpenAI's GPT and Anthropic's Claude. This provider also supports the querying interface for controlling Crosshatch's custom data integration behaviors. This provider wraps the existing underlying providers ([@ai-sdk/openai](/providers/ai-sdk-providers/openai), [@ai-sdk/anthropic](/providers/ai-sdk-providers/openai).

### Credentials

The Crosshatch provider is authenticated by user-specific tokens, enabling permissioned access to personalized inference.

You can obtain synthetic and test user tokens from the [your Crosshatch developer dashboard](https://platform.crosshatch.io/).

Production user tokens are provisioned and accessed with the [Link SDK](https://www.npmjs.com/package/@crosshatch/link) using your Crosshatch developer client id.

## Provider Instance

To create a Crosshatch provider instance, use the `createCrosshatch` function:

```ts
import createCrosshatch from '@crosshatch/ai-provider';
```

## Language Models

You can create [Crosshatch models](https://docs.crosshatch.io/endpoints/ai#supported-model-providers) using a provider instance.

```ts
import { createCrosshatch } from '@crosshatch/ai-provider';
const crosshatch = createCrosshatch();
```

To create a model instance, call the provider instance and specify the model you would like to use in the first argument. In the second argument, specify the user auth token, desired context, and model arguments.
You can use Crosshatch to get generated text based on permissioned user context and your favorite language model.

### Example: Generate Text with Context

This example uses `gpt-4o-mini` to generate text.

```ts
import { generateText } from 'ai';
import createCrosshatch from '@crosshatch/ai-provider':
const crosshatch = createCrosshatch();

const { text } = await generateText({
  model: crosshatch.languageModel("gpt-4o-mini", {
    token: 'YOUR_ACCESS_TOKEN',
    replace: {
      restaurants: {
        select: ["entity_name", "entity_city", "entity_region"],
        from: "personalTimeline",
        where: [
          { field: "event", op: "=", value: "confirmed" },
          { field: "entity_subtype2", op: "=", value: "RESTAURANTS" }
        ],
        groupby: ["entity_name", "entity_city", "entity_region"],
        orderby: "count DESC",
        limit: 5
      }
    }
  }),
  system: `The user recently ate at these restaurants: {restaurants}`,
  messages: [{role: "user", content: "Where should I stay in Paris?"}]
});
```

### Example: Recommend Items based on Context

Use crosshatch to re-rank items based on recent user purchases.

```ts
import { streamObject } from 'ai';
import createCrosshatch from `@crosshatch/ai-provider`
const crosshatch = createCrosshatch();

const itemSummaries = [...]; // list of items
const ids = (itemSummaries?.map(({ itemId }) => itemId) ?? []) as string[];

const { elementStream } = streamObject({
  output: "array",
  mode: "json",
  model: crosshatch.languageModel("gpt-4o-mini", {
    token,
    replace: {
      "orders": {
        select: ["originalTimestamp", "entity_name", "order_total", "order_summary"],
        from: "personalTimeline",
        where: [{ field: "event", op: "=", value: "purchased" }],
        orderBy: [{ field: "originalTimestamp", dir: "desc" }],
        limit: 5,
      },
    },
  }),
  system: `Rerank the following items based on alignment with users recent purchases {orders}`,
  messages: [{role: "user", content: "Heres a list of item: ${JSON.stringify(itemSummaries)"},],
  schema: jsonSchema<{ id: string; reason: string }>({
    type: "object",
    properties: {
      id: { type: "string", enum: ids },
      reason: { type: "string", description: "Explain your ranking." },
    },
  }),
})
```

---
title: LLamaCpp
description: Learn how to use Llama CPP.
---

# LLamaCpp Provider

<Note type="warning">
  The LlamaCpp provider is a prototype that is currently unmaintained.
</Note>

[nnance/llamacpp-ai-provider](https://github.com/nnance/llamacpp-ai-provider) is a community provider that uses [Llama.cpp](https://github.com/ggerganov/llama.cpp) to provide language model support for the AI SDK.

---
title: Community Providers
description: Learn how to use Language Model Specification.
---

# Community Providers

The AI SDK provides a [Language Model Specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1).
You can [write your own provider](./community-providers/custom-providers) that adheres to the specification and it will be compatible with the AI SDK.

Here are the community providers that implement the Language Model Specification:

<CommunityModelCards />

---
title: LangChain
description: Learn how to use LangChain with the AI SDK.
---

# LangChain

[LangChain](https://js.langchain.com/docs/) is a framework for developing applications powered by language models.
It provides tools and abstractions for working with AI models, agents, vector stores, and other data sources for retrieval augmented generation (RAG).
However, LangChain does not provide a way to easily build UIs or a standard way to stream data to the client.

## Example: Completion

Here is a basic example that uses both the AI SDK and LangChain together with the [Next.js](https://nextjs.org/docs) App Router.

The AI SDK [`LangChainAdapter`](/docs/reference/stream-helpers/langchain-adapter) uses the result from [LangChain ExpressionLanguage streaming](https://js.langchain.com/docs/expression_language/streaming) to pipe text to the client.
`LangChainAdapter.toDataStreamResponse()` is compatible with the LangChain Expression Language `.stream()` function response.

```tsx filename="app/api/completion/route.ts" highlight={"16"}
import { ChatOpenAI } from '@langchain/openai';
import { LangChainAdapter } from 'ai';

export const maxDuration = 60;

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const model = new ChatOpenAI({
    model: 'gpt-3.5-turbo-0125',
    temperature: 0,
  });

  const stream = await model.stream(prompt);

  return LangChainAdapter.toDataStreamResponse(stream);
}
```

Then, we use the AI SDK's [`useCompletion`](/docs/ai-sdk-ui/completion) method in the page component to handle the completion:

```tsx filename="app/page.tsx"
'use client';

import { useCompletion } from 'ai/react';

export default function Chat() {
  const { completion, input, handleInputChange, handleSubmit } =
    useCompletion();

  return (
    <div>
      {completion}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

## More Examples

You can find additional examples in the AI SDK [examples/next-langchain](https://github.com/vercel/ai/tree/main/examples/next-langchain) folder.

---
title: LlamaIndex
description: Learn how to use LlamaIndex with the Vercel AI SDK.
---

# LlamaIndex

[LlamaIndex](https://ts.llamaindex.ai/) is a framework for building LLM-powered applications. LlamaIndex helps you ingest, structure, and access private or domain-specific data. LlamaIndex.TS offers the core features of LlamaIndex for Python for popular runtimes like Node.js (official support), Vercel Edge Functions (experimental), and Deno (experimental).

## Example: Completion

Here is a basic example that uses both Vercel AI SDK and LlamaIndex together with the [Next.js](https://nextjs.org/docs) App Router.

The AI SDK [`LlamaIndexAdapter`](/docs/reference/stream-helpers/llamaindex-adapter) uses the stream result from calling the `chat` method on a [LlamaIndex ChatEngine](https://ts.llamaindex.ai/modules/chat_engine) or the `query` method on a [LlamaIndex QueryEngine](https://ts.llamaindex.ai/modules/query_engines) to pipe text to the client.

```tsx filename="app/api/completion/route.ts" highlight="17"
import { OpenAI, SimpleChatEngine } from 'llamaindex';
import { LlamaIndexAdapter } from 'ai';

export const maxDuration = 60;

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const llm = new OpenAI({ model: 'gpt-4o' });
  const chatEngine = new SimpleChatEngine({ llm });

  const stream = await chatEngine.chat({
    message: prompt,
    stream: true,
  });

  return LlamaIndexAdapter.toDataStreamResponse(stream);
}
```

Then, we use the Vercel AI SDK's [`useCompletion`](/docs/ai-sdk-ui/completion) method in the page component to handle the completion:

```tsx filename="app/page.tsx"
'use client';

import { useCompletion } from 'ai/react';

export default function Chat() {
  const { completion, input, handleInputChange, handleSubmit } =
    useCompletion();

  return (
    <div>
      {completion}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

## More Examples

[create-llama](https://github.com/run-llama/create-llama) is the easiest way to get started with LlamaIndex. It uses the Vercel AI SDK to connect to LlamaIndex in all its generated code.

---
title: Adapters
description: Learn how to use AI SDK Adapters.
---

# Adapters

Adapters are lightweight integrations that enable you to use
the AI SDK UI functions (`useChat` and `useCompletion`)
with 3rd party libraries.

The following adapters are currently available:

- [LangChain](/providers/adapters/langchain)
- [LlamaIndex](/providers/adapters/llamaindex)

---
title: Observability Integrations
description: AI SDK Integration for monitoring and tracing LLM applications
---

# Observability Integrations

Several LLM observability providers offer integrations with the AI SDK telemetry data:

- [Langfuse](https://langfuse.com/docs/integrations/vercel-ai-sdk)
- [LangSmith](https://docs.smith.langchain.com/observability/how_to_guides/tracing/trace_with_vercel_ai_sdk)
- [Braintrust](https://www.braintrust.dev/docs/cookbook/recipes/OTEL-logging)
- [Laminar](https://docs.lmnr.ai/tracing/vercel-ai-sdk)

There are also providers that provide monitoring and tracing for the AI SDK through model wrappers:

- [Literal AI](https://docs.literalai.com/integrations/vercel-ai-sdk)

<Note>
  Do you have an observability integration that supports the AI SDK and has an
  integration guide? Please open a pull request to add it to the list.
</Note>