!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to add ad-hoc tool calling capability to LLMs and Chat Models

On this page

How to add ad-hoc tool calling capability to LLMs and Chat Models
=================================================================

Prerequisites

This guide assumes familiarity with the following concepts:

*   [LangChain Expression Language (LCEL)](/v0.2/docs/concepts/#langchain-expression-language)
*   [Chaining runnables](/v0.2/docs/how_to/sequence/)
*   [Tool calling](/v0.2/docs/how_to/tool_calling/)

In this guide we’ll build a Chain that does not rely on any special model APIs (like tool calling, which we showed in the [Quickstart](/v0.2/docs/how_to/tool_calling)) and instead just prompts the model directly to invoke tools.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

We’ll need to install the following packages:

*   npm
*   yarn
*   pnpm

    npm i @langchain/core zod

    yarn add @langchain/core zod

    pnpm add @langchain/core zod

#### Set environment variables[​](#set-environment-variables "Direct link to Set environment variables")

    # Optional, use LangSmith for best-in-class observabilityLANGSMITH_API_KEY=your-api-keyLANGCHAIN_TRACING_V2=true

Create a tool[​](#create-a-tool "Direct link to Create a tool")
---------------------------------------------------------------

First, we need to create a tool to call. For this example, we will create a custom tool from a function. For more information on all details related to creating custom tools, please see [this guide](/v0.2/docs/how_to/custom_tools).

    import { StructuredTool } from "@langchain/core/tools";import { z } from "zod";class Multiply extends StructuredTool {  schema = z.object({    first_int: z.number(),    second_int: z.number(),  });  name = "multiply";  description = "Multiply two integers together.";  async _call(input: z.infer<typeof this.schema>) {    return (input.first_int * input.second_int).toString();  }}const multiply = new Multiply();

    console.log(multiply.name);console.log(multiply.description);

    multiplyMultiply two integers together.

    await multiply.invoke({ first_int: 4, second_int: 5 });

    20

Creating our prompt[​](#creating-our-prompt "Direct link to Creating our prompt")
---------------------------------------------------------------------------------

We’ll want to write a prompt that specifies the tools the model has access to, the arguments to those tools, and the desired output format of the model. In this case we’ll instruct it to output a JSON blob of the form `{"name": "...", "arguments": {...}}`.

    import { renderTextDescription } from "langchain/tools/render";const renderedTools = renderTextDescription([multiply]);

    import { ChatPromptTemplate } from "@langchain/core/prompts";const systemPrompt = `You are an assistant that has access to the following set of tools. Here are the names and descriptions for each tool:{rendered_tools}Given the user input, return the name and input of the tool to use. Return your response as a JSON blob with 'name' and 'arguments' keys.`;const prompt = ChatPromptTemplate.fromMessages([  ["system", systemPrompt],  ["user", "{input}"],]);

Adding an output parser[​](#adding-an-output-parser "Direct link to Adding an output parser")
---------------------------------------------------------------------------------------------

We’ll use the `JsonOutputParser` for parsing our models output to JSON.

### Pick your chat model:

*   OpenAI
*   Anthropic
*   FireworksAI
*   MistralAI
*   Groq
*   VertexAI

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai 

    yarn add @langchain/openai 

    pnpm add @langchain/openai 

#### Add environment variables

    OPENAI_API_KEY=your-api-key

#### Instantiate the model

    import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({  model: "gpt-3.5-turbo",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/anthropic 

    yarn add @langchain/anthropic 

    pnpm add @langchain/anthropic 

#### Add environment variables

    ANTHROPIC_API_KEY=your-api-key

#### Instantiate the model

    import { ChatAnthropic } from "@langchain/anthropic";const model = new ChatAnthropic({  model: "claude-3-sonnet-20240229",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/community 

    yarn add @langchain/community 

    pnpm add @langchain/community 

#### Add environment variables

    FIREWORKS_API_KEY=your-api-key

#### Instantiate the model

    import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const model = new ChatFireworks({  model: "accounts/fireworks/models/firefunction-v1",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/mistralai 

    yarn add @langchain/mistralai 

    pnpm add @langchain/mistralai 

#### Add environment variables

    MISTRAL_API_KEY=your-api-key

#### Instantiate the model

    import { ChatMistralAI } from "@langchain/mistralai";const model = new ChatMistralAI({  model: "mistral-large-latest",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/groq 

    yarn add @langchain/groq 

    pnpm add @langchain/groq 

#### Add environment variables

    GROQ_API_KEY=your-api-key

#### Instantiate the model

    import { ChatGroq } from "@langchain/groq";const model = new ChatGroq({  model: "mixtral-8x7b-32768",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/google-vertexai 

    yarn add @langchain/google-vertexai 

    pnpm add @langchain/google-vertexai 

#### Add environment variables

    GOOGLE_APPLICATION_CREDENTIALS=credentials.json

#### Instantiate the model

    import { ChatVertexAI } from "@langchain/google-vertexai";const model = new ChatVertexAI({  model: "gemini-1.5-pro",  temperature: 0});

    import { JsonOutputParser } from "@langchain/core/output_parsers";const chain = prompt.pipe(model).pipe(new JsonOutputParser());await chain.invoke({  input: "what's thirteen times 4",  rendered_tools: renderedTools,});

    { name: 'multiply', arguments: [ 13, 4 ] }

Invoking the tool[​](#invoking-the-tool "Direct link to Invoking the tool")
---------------------------------------------------------------------------

We can invoke the tool as part of the chain by passing along the model-generated “arguments” to it:

    import { RunnableLambda, RunnablePick } from "@langchain/core/runnables";const chain = prompt  .pipe(model)  .pipe(new JsonOutputParser())  .pipe(new RunnablePick("arguments"))  .pipe(    new RunnableLambda({      func: (input) =>        multiply.invoke({          first_int: input[0],          second_int: input[1],        }),    })  );await chain.invoke({  input: "what's thirteen times 4",  rendered_tools: renderedTools,});

    52

Choosing from multiple tools[​](#choosing-from-multiple-tools "Direct link to Choosing from multiple tools")
------------------------------------------------------------------------------------------------------------

Suppose we have multiple tools we want the chain to be able to choose from:

    class Add extends StructuredTool {  schema = z.object({    first_int: z.number(),    second_int: z.number(),  });  name = "add";  description = "Add two integers together.";  async _call(input: z.infer<typeof this.schema>) {    return (input.first_int + input.second_int).toString();  }}const add = new Add();class Exponentiate extends StructuredTool {  schema = z.object({    first_int: z.number(),    second_int: z.number(),  });  name = "exponentiate";  description = "Exponentiate the base to the exponent power.";  async _call(input: z.infer<typeof this.schema>) {    return Math.pow(input.first_int, input.second_int).toString();  }}const exponentiate = new Exponentiate();

With function calling, we can do this like so:

If we want to run the model selected tool, we can do so using a function that returns the tool based on the model output. Specifically, our function will action return it’s own subchain that gets the “arguments” part of the model output and passes it to the chosen tool:

    import { StructuredToolInterface } from "@langchain/core/tools";const tools = [add, exponentiate, multiply];const toolChain = (modelOutput) => {  const toolMap: Record<string, StructuredToolInterface> = Object.fromEntries(    tools.map((tool) => [tool.name, tool])  );  const chosenTool = toolMap[modelOutput.name];  return new RunnablePick("arguments").pipe(    new RunnableLambda({      func: (input) =>        chosenTool.invoke({          first_int: input[0],          second_int: input[1],        }),    })  );};const toolChainRunnable = new RunnableLambda({  func: toolChain,});const renderedTools = renderTextDescription(tools);const systemPrompt = `You are an assistant that has access to the following set of tools. Here are the names and descriptions for each tool:{rendered_tools}Given the user input, return the name and input of the tool to use. Return your response as a JSON blob with 'name' and 'arguments' keys.`;const prompt = ChatPromptTemplate.fromMessages([  ["system", systemPrompt],  ["user", "{input}"],]);const chain = prompt  .pipe(model)  .pipe(new JsonOutputParser())  .pipe(toolChainRunnable);await chain.invoke({  input: "what's 3 plus 1132",  rendered_tools: renderedTools,});

    1135

Returning tool inputs[​](#returning-tool-inputs "Direct link to Returning tool inputs")
---------------------------------------------------------------------------------------

It can be helpful to return not only tool outputs but also tool inputs. We can easily do this with LCEL by `RunnablePassthrough.assign`\-ing the tool output. This will take whatever the input is to the RunnablePassrthrough components (assumed to be a dictionary) and add a key to it while still passing through everything that’s currently in the input:

    import { RunnablePassthrough } from "@langchain/core/runnables";const chain = prompt  .pipe(model)  .pipe(new JsonOutputParser())  .pipe(RunnablePassthrough.assign({ output: toolChainRunnable }));await chain.invoke({  input: "what's 3 plus 1132",  rendered_tools: renderedTools,});

    { name: 'add', arguments: [ 3, 1132 ], output: '1135' }

What’s next?[​](#whats-next "Direct link to What’s next?")
----------------------------------------------------------

This how-to guide shows the “happy path” when the model correctly outputs all the required tool information.

In reality, if you’re using more complex tools, you will start encountering errors from the model, especially for models that have not been fine tuned for tool calling and for less capable models.

You will need to be prepared to add strategies to improve the output from the model; e.g.,

*   Provide few shot examples.
*   Add error handling (e.g., catch the exception and feed it back to the LLM to ask it to correct its previous output).

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to return structured data from a model

](/v0.2/docs/how_to/structured_output)[

Next

How to create a custom chat model class

](/v0.2/docs/how_to/custom_chat)

*   [Setup](#setup)
*   [Create a tool](#create-a-tool)
*   [Creating our prompt](#creating-our-prompt)
*   [Adding an output parser](#adding-an-output-parser)
*   [Invoking the tool](#invoking-the-tool)
*   [Choosing from multiple tools](#choosing-from-multiple-tools)
*   [Returning tool inputs](#returning-tool-inputs)
*   [What’s next?](#whats-next)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.