!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.1/docs/integrations/platforms/)
    
    *   [Providers](/v0.1/docs/integrations/platforms/)
    *   [Anthropic](/v0.1/docs/integrations/platforms/anthropic/)
    *   [AWS](/v0.1/docs/integrations/platforms/aws/)
    *   [Google](/v0.1/docs/integrations/platforms/google/)
    *   [Microsoft](/v0.1/docs/integrations/platforms/microsoft/)
    *   [OpenAI](/v0.1/docs/integrations/platforms/openai/)
*   [Components](/v0.1/docs/integrations/components/)
    
    *   [LLMs](/v0.1/docs/integrations/llms/)
        
    *   [Chat models](/v0.1/docs/integrations/chat/)
        
        *   [Chat models](/v0.1/docs/integrations/chat/)
        *   [Alibaba Tongyi](/v0.1/docs/integrations/chat/alibaba_tongyi/)
        *   [Anthropic](/v0.1/docs/integrations/chat/anthropic/)
        *   [Anthropic Tools](/v0.1/docs/integrations/chat/anthropic_tools/)
        *   [Azure OpenAI](/v0.1/docs/integrations/chat/azure/)
        *   [Baidu Wenxin](/v0.1/docs/integrations/chat/baidu_wenxin/)
        *   [Bedrock](/v0.1/docs/integrations/chat/bedrock/)
        *   [Cloudflare Workers AI](/v0.1/docs/integrations/chat/cloudflare_workersai/)
        *   [Cohere](/v0.1/docs/integrations/chat/cohere/)
        *   [Fake LLM](/v0.1/docs/integrations/chat/fake/)
        *   [Fireworks](/v0.1/docs/integrations/chat/fireworks/)
        *   [Friendli](/v0.1/docs/integrations/chat/friendli/)
        *   [Google GenAI](/v0.1/docs/integrations/chat/google_generativeai/)
        *   [(Legacy) Google PaLM/VertexAI](/v0.1/docs/integrations/chat/google_palm/)
        *   [Google Vertex AI](/v0.1/docs/integrations/chat/google_vertex_ai/)
        *   [Groq](/v0.1/docs/integrations/chat/groq/)
        *   [Llama CPP](/v0.1/docs/integrations/chat/llama_cpp/)
        *   [Minimax](/v0.1/docs/integrations/chat/minimax/)
        *   [Mistral AI](/v0.1/docs/integrations/chat/mistral/)
        *   [NIBittensorChatModel](/v0.1/docs/integrations/chat/ni_bittensor/)
        *   [Ollama](/v0.1/docs/integrations/chat/ollama/)
        *   [Ollama Functions](/v0.1/docs/integrations/chat/ollama_functions/)
        *   [OpenAI](/v0.1/docs/integrations/chat/openai/)
        *   [PremAI](/v0.1/docs/integrations/chat/premai/)
        *   [PromptLayer OpenAI](/v0.1/docs/integrations/chat/prompt_layer_openai/)
        *   [TogetherAI](/v0.1/docs/integrations/chat/togetherai/)
        *   [WebLLM](/v0.1/docs/integrations/chat/web_llm/)
        *   [YandexGPT](/v0.1/docs/integrations/chat/yandex/)
        *   [ZhipuAI](/v0.1/docs/integrations/chat/zhipuai/)
    *   [Document loaders](/v0.1/docs/integrations/document_loaders/)
        
    *   [Document transformers](/v0.1/docs/integrations/document_transformers/)
        
    *   [Document compressors](/v0.1/docs/integrations/document_compressors/)
        
    *   [Text embedding models](/v0.1/docs/integrations/text_embedding/)
        
    *   [Vector stores](/v0.1/docs/integrations/vectorstores/)
        
    *   [Retrievers](/v0.1/docs/integrations/retrievers/)
        
    *   [Tools](/v0.1/docs/integrations/tools/)
        
    *   [Agents and toolkits](/v0.1/docs/integrations/toolkits/)
        
    *   [Chat Memory](/v0.1/docs/integrations/chat_memory/)
        
    *   [Stores](/v0.1/docs/integrations/stores/)
        

*   [](/v0.1/)
*   [Components](/v0.1/docs/integrations/components/)
*   [Chat models](/v0.1/docs/integrations/chat/)
*   Mistral AI

On this page

ChatMistralAI
=============

[Mistral AI](https://mistral.ai/) is a research organization and hosting platform for LLMs. They're most known for their family of 7B models ([`mistral7b` // `mistral-tiny`](https://mistral.ai/news/announcing-mistral-7b/), [`mixtral8x7b` // `mistral-small`](https://mistral.ai/news/mixtral-of-experts/)).

The LangChain implementation of Mistral's models uses their hosted generation API, making it easier to access their models without needing to run them locally.

Models[​](#models "Direct link to Models")
------------------------------------------

Mistral's API offers access to two of their open source, and proprietary models:

*   `open-mistral-7b` (aka `mistral-tiny-2312`)
*   `open-mixtral-8x7b` (aka `mistral-small-2312`)
*   `mistral-small-latest` (aka `mistral-small-2402`) (default)
*   `mistral-medium-latest` (aka `mistral-medium-2312`)
*   `mistral-large-latest` (aka `mistral-large-2402`)

See [this page](https://docs.mistral.ai/guides/model-selection/) for an up to date list.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

In order to use the Mistral API you'll need an API key. You can sign up for a Mistral account and create an API key [here](https://console.mistral.ai/).

You'll first need to install the [`@langchain/mistralai`](https://www.npmjs.com/package/@langchain/mistralai) package:

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/mistralai

    yarn add @langchain/mistralai

    pnpm add @langchain/mistralai

tip

We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.

Usage[​](#usage "Direct link to Usage")
---------------------------------------

When sending chat messages to mistral, there are a few requirements to follow:

*   The first message can __not__ be an assistant (ai) message.
*   Messages __must__ alternate between user and assistant (ai) messages.
*   Messages can __not__ end with an assistant (ai) or system message.

    import { ChatMistralAI } from "@langchain/mistralai";import { ChatPromptTemplate } from "@langchain/core/prompts";const model = new ChatMistralAI({  apiKey: process.env.MISTRAL_API_KEY,  model: "mistral-small",});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant"],  ["human", "{input}"],]);const chain = prompt.pipe(model);const response = await chain.invoke({  input: "Hello",});console.log("response", response);/**response AIMessage {  lc_namespace: [ 'langchain_core', 'messages' ],  content: "Hello! I'm here to help answer any questions you might have or provide information on a variety of topics. How can I assist you today?\n" +    '\n' +    'Here are some common tasks I can help with:\n' +    '\n' +    '* Setting alarms or reminders\n' +    '* Sending emails or messages\n' +    '* Making phone calls\n' +    '* Providing weather information\n' +    '* Creating to-do lists\n' +    '* Offering suggestions for restaurants, movies, or other local activities\n' +    '* Providing definitions and explanations for words or concepts\n' +    '* Translating text into different languages\n' +    '* Playing music or podcasts\n' +    '* Setting timers\n' +    '* Providing directions or traffic information\n' +    '* And much more!\n' +    '\n' +    "Let me know how I can help you specifically, and I'll do my best to make your day easier and more productive!\n" +    '\n' +    'Best regards,\n' +    'Your helpful assistant.',  name: undefined,  additional_kwargs: {}} */

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

info

You can see a LangSmith trace of this example [here](https://smith.langchain.com/public/d69d0db9-f29e-45aa-a40d-b53f6273d7d0/r)

### Streaming[​](#streaming "Direct link to Streaming")

Mistral's API also supports streaming token responses. The example below demonstrates how to use this feature.

    import { ChatMistralAI } from "@langchain/mistralai";import { ChatPromptTemplate } from "@langchain/core/prompts";import { StringOutputParser } from "@langchain/core/output_parsers";const model = new ChatMistralAI({  apiKey: process.env.MISTRAL_API_KEY,  model: "mistral-small",});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant"],  ["human", "{input}"],]);const outputParser = new StringOutputParser();const chain = prompt.pipe(model).pipe(outputParser);const response = await chain.stream({  input: "Hello",});for await (const item of response) {  console.log("stream item:", item);}/**stream item:stream item: Hello! I'm here to help answer any questions youstream item:  might have or assist you with any task you'd like tostream item:  accomplish. I can provide informationstream item:  on a wide range of topicsstream item: , from math and science to history and literature. I canstream item:  also help you manage your schedule, set reminders, andstream item:  much more. Is there something specific you need help with? Letstream item:  me know!stream item: */

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`
*   [StringOutputParser](https://api.js.langchain.com/classes/langchain_core_output_parsers.StringOutputParser.html) from `@langchain/core/output_parsers`

info

You can see a LangSmith trace of this example [here](https://smith.langchain.com/public/061d90f2-ac7e-44c5-8790-8b23299f9217/r)

### Tool calling[​](#tool-calling "Direct link to Tool calling")

Mistral's API now supports tool calling and JSON mode! The examples below demonstrates how to use them, along with how to use the `withStructuredOutput` method to easily compose structured output LLM calls.

    import { ChatMistralAI } from "@langchain/mistralai";import { ChatPromptTemplate } from "@langchain/core/prompts";import { JsonOutputKeyToolsParser } from "@langchain/core/output_parsers/openai_tools";import { z } from "zod";import { StructuredTool } from "@langchain/core/tools";const calculatorSchema = z.object({  operation: z    .enum(["add", "subtract", "multiply", "divide"])    .describe("The type of operation to execute."),  number1: z.number().describe("The first number to operate on."),  number2: z.number().describe("The second number to operate on."),});// Extend the StructuredTool class to create a new toolclass CalculatorTool extends StructuredTool {  name = "calculator";  description = "A simple calculator tool";  schema = calculatorSchema;  async _call(input: z.infer<typeof calculatorSchema>) {    return JSON.stringify(input);  }}// Or you can convert the tool to a JSON schema using// a library like zod-to-json-schema// Uncomment the lines below to use tools this way.// import { zodToJsonSchema } from "zod-to-json-schema";// const calculatorJsonSchema = zodToJsonSchema(calculatorSchema);const model = new ChatMistralAI({  apiKey: process.env.MISTRAL_API_KEY,  model: "mistral-large",});// Bind the tool to the modelconst modelWithTool = model.bind({  tools: [new CalculatorTool()],});const prompt = ChatPromptTemplate.fromMessages([  [    "system",    "You are a helpful assistant who always needs to use a calculator.",  ],  ["human", "{input}"],]);// Define an output parser that can handle tool responsesconst outputParser = new JsonOutputKeyToolsParser({  keyName: "calculator",  returnSingle: true,});// Chain your prompt, model, and output parser togetherconst chain = prompt.pipe(modelWithTool).pipe(outputParser);const response = await chain.invoke({  input: "What is 2 + 2?",});console.log(response);/*{ operation: 'add', number1: 2, number2: 2 } */

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`
*   [JsonOutputKeyToolsParser](https://api.js.langchain.com/classes/langchain_core_output_parsers_openai_tools.JsonOutputKeyToolsParser.html) from `@langchain/core/output_parsers/openai_tools`
*   [StructuredTool](https://api.js.langchain.com/classes/langchain_core_tools.StructuredTool.html) from `@langchain/core/tools`

### `.withStructuredOutput({ ... })`[​](#withstructuredoutput-- "Direct link to withstructuredoutput--")

info

The `.withStructuredOutput` method is in beta. It is actively being worked on, so the API may change.

Using the `.withStructuredOutput` method, you can easily make the LLM return structured output, given only a Zod or JSON schema:

note

The Mistral tool calling API requires descriptions for each tool field. If descriptions are not supplied, the API will error.

    import { ChatMistralAI } from "@langchain/mistralai";import { ChatPromptTemplate } from "@langchain/core/prompts";import { z } from "zod";const calculatorSchema = z  .object({    operation: z      .enum(["add", "subtract", "multiply", "divide"])      .describe("The type of operation to execute."),    number1: z.number().describe("The first number to operate on."),    number2: z.number().describe("The second number to operate on."),  })  .describe("A simple calculator tool");const model = new ChatMistralAI({  apiKey: process.env.MISTRAL_API_KEY,  model: "mistral-large",});// Pass the schema and tool name to the withStructuredOutput methodconst modelWithTool = model.withStructuredOutput(calculatorSchema);const prompt = ChatPromptTemplate.fromMessages([  [    "system",    "You are a helpful assistant who always needs to use a calculator.",  ],  ["human", "{input}"],]);// Chain your prompt and model togetherconst chain = prompt.pipe(modelWithTool);const response = await chain.invoke({  input: "What is 2 + 2?",});console.log(response);/*  { operation: 'add', number1: 2, number2: 2 }*//** * You can supply a "name" field to give the LLM additional context * around what you are trying to generate. You can also pass * 'includeRaw' to get the raw message back from the model too. */const includeRawModel = model.withStructuredOutput(calculatorSchema, {  name: "calculator",  includeRaw: true,});const includeRawChain = prompt.pipe(includeRawModel);const includeRawResponse = await includeRawChain.invoke({  input: "What is 2 + 2?",});console.log(JSON.stringify(includeRawResponse, null, 2));/*  {    "raw": {      "kwargs": {        "content": "",        "additional_kwargs": {          "tool_calls": [            {              "id": "null",              "type": "function",              "function": {                "name": "calculator",                "arguments": "{\"operation\": \"add\", \"number1\": 2, \"number2\": 2}"              }            }          ]        }      }    },    "parsed": {      "operation": "add",      "number1": 2,      "number2": 2    }  }*/

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

### Using JSON schema:[​](#using-json-schema "Direct link to Using JSON schema:")

    import { ChatMistralAI } from "@langchain/mistralai";import { ChatPromptTemplate } from "@langchain/core/prompts";const calculatorJsonSchema = {  type: "object",  properties: {    operation: {      type: "string",      enum: ["add", "subtract", "multiply", "divide"],      description: "The type of operation to execute.",    },    number1: { type: "number", description: "The first number to operate on." },    number2: {      type: "number",      description: "The second number to operate on.",    },  },  required: ["operation", "number1", "number2"],  description: "A simple calculator tool",};const model = new ChatMistralAI({  apiKey: process.env.MISTRAL_API_KEY,  model: "mistral-large",});// Pass the schema and tool name to the withStructuredOutput methodconst modelWithTool = model.withStructuredOutput(calculatorJsonSchema);const prompt = ChatPromptTemplate.fromMessages([  [    "system",    "You are a helpful assistant who always needs to use a calculator.",  ],  ["human", "{input}"],]);// Chain your prompt and model togetherconst chain = prompt.pipe(modelWithTool);const response = await chain.invoke({  input: "What is 2 + 2?",});console.log(response);/*  { operation: 'add', number1: 2, number2: 2 }*/

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

### Tool calling agent[​](#tool-calling-agent "Direct link to Tool calling agent")

The larger Mistral models not only support tool calling, but can also be used in the [Tool Calling agent](/v0.1/docs/modules/agents/agent_types/tool_calling/). Here's an example:

    import { z } from "zod";import { ChatMistralAI } from "@langchain/mistralai";import { DynamicStructuredTool } from "@langchain/core/tools";import { AgentExecutor, createToolCallingAgent } from "langchain/agents";import { ChatPromptTemplate } from "@langchain/core/prompts";const llm = new ChatMistralAI({  temperature: 0,  model: "mistral-large-latest",});// Prompt template must have "input" and "agent_scratchpad input variables"const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant"],  ["placeholder", "{chat_history}"],  ["human", "{input}"],  ["placeholder", "{agent_scratchpad}"],]);const currentWeatherTool = new DynamicStructuredTool({  name: "get_current_weather",  description: "Get the current weather in a given location",  schema: z.object({    location: z.string().describe("The city and state, e.g. San Francisco, CA"),  }),  func: async () => Promise.resolve("28 °C"),});const agent = await createToolCallingAgent({  llm,  tools: [currentWeatherTool],  prompt,});const agentExecutor = new AgentExecutor({  agent,  tools: [currentWeatherTool],});const input = "What's the weather like in Paris?";const { output } = await agentExecutor.invoke({ input });console.log(output);/*   The current weather in Paris is 28 °C.*/

#### API Reference:

*   [ChatMistralAI](https://api.js.langchain.com/classes/langchain_mistralai.ChatMistralAI.html) from `@langchain/mistralai`
*   [DynamicStructuredTool](https://api.js.langchain.com/classes/langchain_core_tools.DynamicStructuredTool.html) from `@langchain/core/tools`
*   [AgentExecutor](https://api.js.langchain.com/classes/langchain_agents.AgentExecutor.html) from `langchain/agents`
*   [createToolCallingAgent](https://api.js.langchain.com/functions/langchain_agents.createToolCallingAgent.html) from `langchain/agents`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Minimax

](/v0.1/docs/integrations/chat/minimax/)[

Next

NIBittensorChatModel

](/v0.1/docs/integrations/chat/ni_bittensor/)

*   [Models](#models)
*   [Setup](#setup)
*   [Usage](#usage)
    *   [Streaming](#streaming)
    *   [Tool calling](#tool-calling)
    *   [`.withStructuredOutput({ ... })`](#withstructuredoutput--)
    *   [Using JSON schema:](#using-json-schema)
    *   [Tool calling agent](#tool-calling-agent)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.