!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.1/docs/integrations/platforms/)
    
    *   [Providers](/v0.1/docs/integrations/platforms/)
    *   [Anthropic](/v0.1/docs/integrations/platforms/anthropic/)
    *   [AWS](/v0.1/docs/integrations/platforms/aws/)
    *   [Google](/v0.1/docs/integrations/platforms/google/)
    *   [Microsoft](/v0.1/docs/integrations/platforms/microsoft/)
    *   [OpenAI](/v0.1/docs/integrations/platforms/openai/)
*   [Components](/v0.1/docs/integrations/components/)
    
    *   [LLMs](/v0.1/docs/integrations/llms/)
        
    *   [Chat models](/v0.1/docs/integrations/chat/)
        
        *   [Chat models](/v0.1/docs/integrations/chat/)
        *   [Alibaba Tongyi](/v0.1/docs/integrations/chat/alibaba_tongyi/)
        *   [Anthropic](/v0.1/docs/integrations/chat/anthropic/)
        *   [Anthropic Tools](/v0.1/docs/integrations/chat/anthropic_tools/)
        *   [Azure OpenAI](/v0.1/docs/integrations/chat/azure/)
        *   [Baidu Wenxin](/v0.1/docs/integrations/chat/baidu_wenxin/)
        *   [Bedrock](/v0.1/docs/integrations/chat/bedrock/)
        *   [Cloudflare Workers AI](/v0.1/docs/integrations/chat/cloudflare_workersai/)
        *   [Cohere](/v0.1/docs/integrations/chat/cohere/)
        *   [Fake LLM](/v0.1/docs/integrations/chat/fake/)
        *   [Fireworks](/v0.1/docs/integrations/chat/fireworks/)
        *   [Friendli](/v0.1/docs/integrations/chat/friendli/)
        *   [Google GenAI](/v0.1/docs/integrations/chat/google_generativeai/)
        *   [(Legacy) Google PaLM/VertexAI](/v0.1/docs/integrations/chat/google_palm/)
        *   [Google Vertex AI](/v0.1/docs/integrations/chat/google_vertex_ai/)
        *   [Groq](/v0.1/docs/integrations/chat/groq/)
        *   [Llama CPP](/v0.1/docs/integrations/chat/llama_cpp/)
        *   [Minimax](/v0.1/docs/integrations/chat/minimax/)
        *   [Mistral AI](/v0.1/docs/integrations/chat/mistral/)
        *   [NIBittensorChatModel](/v0.1/docs/integrations/chat/ni_bittensor/)
        *   [Ollama](/v0.1/docs/integrations/chat/ollama/)
        *   [Ollama Functions](/v0.1/docs/integrations/chat/ollama_functions/)
        *   [OpenAI](/v0.1/docs/integrations/chat/openai/)
        *   [PremAI](/v0.1/docs/integrations/chat/premai/)
        *   [PromptLayer OpenAI](/v0.1/docs/integrations/chat/prompt_layer_openai/)
        *   [TogetherAI](/v0.1/docs/integrations/chat/togetherai/)
        *   [WebLLM](/v0.1/docs/integrations/chat/web_llm/)
        *   [YandexGPT](/v0.1/docs/integrations/chat/yandex/)
        *   [ZhipuAI](/v0.1/docs/integrations/chat/zhipuai/)
    *   [Document loaders](/v0.1/docs/integrations/document_loaders/)
        
    *   [Document transformers](/v0.1/docs/integrations/document_transformers/)
        
    *   [Document compressors](/v0.1/docs/integrations/document_compressors/)
        
    *   [Text embedding models](/v0.1/docs/integrations/text_embedding/)
        
    *   [Vector stores](/v0.1/docs/integrations/vectorstores/)
        
    *   [Retrievers](/v0.1/docs/integrations/retrievers/)
        
    *   [Tools](/v0.1/docs/integrations/tools/)
        
    *   [Agents and toolkits](/v0.1/docs/integrations/toolkits/)
        
    *   [Chat Memory](/v0.1/docs/integrations/chat_memory/)
        
    *   [Stores](/v0.1/docs/integrations/stores/)
        

*   [](/v0.1/)
*   [Components](/v0.1/docs/integrations/components/)
*   [Chat models](/v0.1/docs/integrations/chat/)
*   Groq

On this page

ChatGroq
========

Setup[​](#setup "Direct link to Setup")
---------------------------------------

In order to use the Groq API you'll need an API key. You can sign up for a Groq account and create an API key [here](https://wow.groq.com/).

You'll first need to install the [`@langchain/groq`](https://www.npmjs.com/package/@langchain/groq) package:

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/groq

    yarn add @langchain/groq

    pnpm add @langchain/groq

tip

We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.

Usage[​](#usage "Direct link to Usage")
---------------------------------------

    import { ChatGroq } from "@langchain/groq";import { ChatPromptTemplate } from "@langchain/core/prompts";const model = new ChatGroq({  apiKey: process.env.GROQ_API_KEY,});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant"],  ["human", "{input}"],]);const chain = prompt.pipe(model);const response = await chain.invoke({  input: "Hello",});console.log("response", response);/**response AIMessage {  content: "Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or a question you have?",} */

#### API Reference:

*   [ChatGroq](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html) from `@langchain/groq`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

info

You can see a LangSmith trace of this example [here](https://smith.langchain.com/public/2ba59207-1383-4e42-b6a6-c1ddcfcd5710/r)

Tool calling[​](#tool-calling "Direct link to Tool calling")
------------------------------------------------------------

Groq chat models support calling multiple functions to get all required data to answer a question. Here's an example:

    import { ChatGroq } from "@langchain/groq";// Mocked out function, could be a database/API call in productionfunction getCurrentWeather(location: string, _unit?: string) {  if (location.toLowerCase().includes("tokyo")) {    return JSON.stringify({ location, temperature: "10", unit: "celsius" });  } else if (location.toLowerCase().includes("san francisco")) {    return JSON.stringify({      location,      temperature: "72",      unit: "fahrenheit",    });  } else {    return JSON.stringify({ location, temperature: "22", unit: "celsius" });  }}// Bind function to the model as a toolconst chat = new ChatGroq({  model: "mixtral-8x7b-32768",  maxTokens: 128,}).bind({  tools: [    {      type: "function",      function: {        name: "get_current_weather",        description: "Get the current weather in a given location",        parameters: {          type: "object",          properties: {            location: {              type: "string",              description: "The city and state, e.g. San Francisco, CA",            },            unit: { type: "string", enum: ["celsius", "fahrenheit"] },          },          required: ["location"],        },      },    },  ],  tool_choice: "auto",});const res = await chat.invoke([  ["human", "What's the weather like in San Francisco?"],]);console.log(res.additional_kwargs.tool_calls);/*  [    {      id: 'call_01htk055jpftwbb9tvphyf9bnf',      type: 'function',      function: {        name: 'get_current_weather',        arguments: '{"location":"San Francisco, CA"}'      }    }  ]*/

#### API Reference:

*   [ChatGroq](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html) from `@langchain/groq`

### `.withStructuredOutput({ ... })`[​](#withstructuredoutput-- "Direct link to withstructuredoutput--")

info

The `.withStructuredOutput` method is in beta. It is actively being worked on, so the API may change.

You can also use the `.withStructuredOutput({ ... })` method to coerce `ChatGroq` into returning a structured output.

The method allows for passing in either a Zod object, or a valid JSON schema (like what is returned from [`zodToJsonSchema`](https://www.npmjs.com/package/zod-to-json-schema)).

Using the method is simple. Just define your LLM and call `.withStructuredOutput({ ... })` on it, passing the desired schema.

Here is an example using a Zod schema and the `functionCalling` mode (default mode):

    import { ChatPromptTemplate } from "@langchain/core/prompts";import { ChatGroq } from "@langchain/groq";import { z } from "zod";const model = new ChatGroq({  temperature: 0,  model: "mixtral-8x7b-32768",});const calculatorSchema = z.object({  operation: z.enum(["add", "subtract", "multiply", "divide"]),  number1: z.number(),  number2: z.number(),});const modelWithStructuredOutput = model.withStructuredOutput(calculatorSchema);const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are VERY bad at math and must always use a calculator."],  ["human", "Please help me!! What is 2 + 2?"],]);const chain = prompt.pipe(modelWithStructuredOutput);const result = await chain.invoke({});console.log(result);/*  { operation: 'add', number1: 2, number2: 2 }*//** * You can also specify 'includeRaw' to return the parsed * and raw output in the result. */const includeRawModel = model.withStructuredOutput(calculatorSchema, {  name: "calculator",  includeRaw: true,});const includeRawChain = prompt.pipe(includeRawModel);const includeRawResult = await includeRawChain.invoke({});console.log(includeRawResult);/*  {    raw: AIMessage {      content: '',      additional_kwargs: {        tool_calls: [          {            "id": "call_01htk094ktfgxtkwj40n0ehg61",            "type": "function",            "function": {              "name": "calculator",              "arguments": "{\"operation\": \"add\", \"number1\": 2, \"number2\": 2}"            }          }        ]      },      response_metadata: {        "tokenUsage": {          "completionTokens": 197,          "promptTokens": 1214,          "totalTokens": 1411        },        "finish_reason": "tool_calls"      }    },    parsed: { operation: 'add', number1: 2, number2: 2 }  }*/

#### API Reference:

*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`
*   [ChatGroq](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html) from `@langchain/groq`

Streaming[​](#streaming "Direct link to Streaming")
---------------------------------------------------

Groq's API also supports streaming token responses. The example below demonstrates how to use this feature.

    import { ChatGroq } from "@langchain/groq";import { ChatPromptTemplate } from "@langchain/core/prompts";import { StringOutputParser } from "@langchain/core/output_parsers";const model = new ChatGroq({  apiKey: process.env.GROQ_API_KEY,});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant"],  ["human", "{input}"],]);const outputParser = new StringOutputParser();const chain = prompt.pipe(model).pipe(outputParser);const response = await chain.stream({  input: "Hello",});let res = "";for await (const item of response) {  res += item;  console.log("stream:", res);}/**stream: Hellostream: Hello!stream: Hello! Istream: Hello! I'stream: Hello! I'mstream: Hello! I'm happystream: Hello! I'm happy tostream: Hello! I'm happy to assiststream: Hello! I'm happy to assist youstream: Hello! I'm happy to assist you instream: Hello! I'm happy to assist you in anystream: Hello! I'm happy to assist you in any waystream: Hello! I'm happy to assist you in any way Istream: Hello! I'm happy to assist you in any way I canstream: Hello! I'm happy to assist you in any way I can.stream: Hello! I'm happy to assist you in any way I can. Isstream: Hello! I'm happy to assist you in any way I can. Is therestream: Hello! I'm happy to assist you in any way I can. Is there somethingstream: Hello! I'm happy to assist you in any way I can. Is there something specificstream: Hello! I'm happy to assist you in any way I can. Is there something specific youstream: Hello! I'm happy to assist you in any way I can. Is there something specific you needstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need helpstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help withstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with orstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or astream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or a questionstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or a question youstream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or a question you havestream: Hello! I'm happy to assist you in any way I can. Is there something specific you need help with or a question you have? */

#### API Reference:

*   [ChatGroq](https://api.js.langchain.com/classes/langchain_groq.ChatGroq.html) from `@langchain/groq`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`
*   [StringOutputParser](https://api.js.langchain.com/classes/langchain_core_output_parsers.StringOutputParser.html) from `@langchain/core/output_parsers`

info

You can see a LangSmith trace of this example [here](https://smith.langchain.com/public/72832eb5-b9ae-4ce0-baa2-c2e95eca61a7/r)

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Google Vertex AI

](/v0.1/docs/integrations/chat/google_vertex_ai/)[

Next

Llama CPP

](/v0.1/docs/integrations/chat/llama_cpp/)

*   [Setup](#setup)
*   [Usage](#usage)
*   [Tool calling](#tool-calling)
    *   [`.withStructuredOutput({ ... })`](#withstructuredoutput--)
*   [Streaming](#streaming)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.