!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Get started](/v0.1/docs/get_started/)
    
    *   [Introduction](/v0.1/docs/get_started/introduction/)
    *   [Installation](/v0.1/docs/get_started/installation/)
    *   [Quickstart](/v0.1/docs/get_started/quickstart/)
*   [LangChain Expression Language](/v0.1/docs/expression_language/)
    
    *   [Get started](/v0.1/docs/expression_language/get_started/)
    *   [Why use LCEL?](/v0.1/docs/expression_language/why/)
    *   [Interface](/v0.1/docs/expression_language/interface/)
    *   [Streaming](/v0.1/docs/expression_language/streaming/)
    *   [How to](/v0.1/docs/expression_language/how_to/routing/)
        
    *   [Cookbook](/v0.1/docs/expression_language/cookbook/)
        
    *   [LangChain Expression Language (LCEL)](/v0.1/docs/expression_language/)
*   [Modules](/v0.1/docs/modules/)
    
    *   [Model I/O](/v0.1/docs/modules/model_io/)
        
    *   [Retrieval](/v0.1/docs/modules/data_connection/)
        
    *   [Chains](/v0.1/docs/modules/chains/)
    *   [Agents](/v0.1/docs/modules/agents/)
        
        *   [Quick start](/v0.1/docs/modules/agents/quick_start/)
        *   [Concepts](/v0.1/docs/modules/agents/concepts/)
        *   [Agent Types](/v0.1/docs/modules/agents/agent_types/)
            
        *   [How-to](/v0.1/docs/modules/agents/how_to/custom_agent/)
            
            *   [Custom agent](/v0.1/docs/modules/agents/how_to/custom_agent/)
            *   [Returning structured output](/v0.1/docs/modules/agents/how_to/agent_structured/)
            *   [Subscribing to events](/v0.1/docs/modules/agents/how_to/callbacks/)
            *   [Cancelling requests](/v0.1/docs/modules/agents/how_to/cancelling_requests/)
            *   [Custom LLM Agent](/v0.1/docs/modules/agents/how_to/custom_llm_agent/)
            *   [Custom LLM Agent (with a ChatModel)](/v0.1/docs/modules/agents/how_to/custom_llm_chat_agent/)
            *   [Custom MRKL agent](/v0.1/docs/modules/agents/how_to/custom_mrkl_agent/)
            *   [Handle parsing errors](/v0.1/docs/modules/agents/how_to/handle_parsing_errors/)
            *   [Access intermediate steps](/v0.1/docs/modules/agents/how_to/intermediate_steps/)
            *   [Logging and tracing](/v0.1/docs/modules/agents/how_to/logging_and_tracing/)
            *   [Cap the max number of iterations](/v0.1/docs/modules/agents/how_to/max_iterations/)
            *   [Streaming](/v0.1/docs/modules/agents/how_to/streaming/)
            *   [Timeouts for agents](/v0.1/docs/modules/agents/how_to/timeouts/)
        *   [Agents](/v0.1/docs/modules/agents/)
        *   [Tools](/v0.1/docs/modules/agents/tools/)
            
    *   [More](/v0.1/docs/modules/memory/)
        
*   [Security](/v0.1/docs/security/)
*   [Guides](/v0.1/docs/guides/)
    
*   [Ecosystem](/v0.1/docs/ecosystem/)
    
*   [LangGraph](/v0.1/docs/langgraph/)
*   * * *
    

*   [](/v0.1/)
*   [Modules](/v0.1/docs/modules/)
*   [Agents](/v0.1/docs/modules/agents/)
*   How-to
*   Custom LLM Agent (with a ChatModel)

Custom LLM Agent (with a ChatModel)
===================================

This notebook goes through how to create your own custom agent based on a chat model.

An LLM chat agent consists of three parts:

*   PromptTemplate: This is the prompt template that can be used to instruct the language model on what to do
*   ChatModel: This is the language model that powers the agent
*   `stop` sequence: Instructs the LLM to stop generating as soon as this string is found
*   OutputParser: This determines how to parse the LLMOutput into an AgentAction or AgentFinish object

The LLMAgent is used in an AgentExecutor. This AgentExecutor can largely be thought of as a loop that:

1.  Passes user input and any previous steps to the Agent (in this case, the LLMAgent)
2.  If the Agent returns an `AgentFinish`, then return that directly to the user
3.  If the Agent returns an `AgentAction`, then use that to call a tool and get an `Observation`
4.  Repeat, passing the `AgentAction` and `Observation` back to the Agent until an `AgentFinish` is emitted.

`AgentAction` is a response that consists of `action` and `action_input`. `action` refers to which tool to use, and `action_input` refers to the input to that tool. `log` can also be provided as more context (that can be used for logging, tracing, etc).

`AgentFinish` is a response that contains the final message to be sent back to the user. This should be used to end an agent run.

With LCEL
=========

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/openai

    yarn add @langchain/openai

    pnpm add @langchain/openai

    import { AgentExecutor } from "langchain/agents";import { formatLogToString } from "langchain/agents/format_scratchpad/log";import { ChatOpenAI } from "@langchain/openai";import { Calculator } from "@langchain/community/tools/calculator";import { PromptTemplate } from "@langchain/core/prompts";import { AgentAction, AgentFinish, AgentStep } from "@langchain/core/agents";import { BaseMessage, HumanMessage } from "@langchain/core/messages";import { InputValues } from "@langchain/core/memory";import { RunnableSequence } from "@langchain/core/runnables";import { SerpAPI } from "@langchain/community/tools/serpapi";/** * Instantiate the chat model and bind the stop token * @important The stop token must be set, if not the LLM will happily continue generating text forever. */const model = new ChatOpenAI({ temperature: 0 }).bind({  stop: ["\nObservation"],});/** Define the tools */const tools = [  new SerpAPI(process.env.SERPAPI_API_KEY, {    location: "Austin,Texas,United States",    hl: "en",    gl: "us",  }),  new Calculator(),];/** Create the prefix prompt */const PREFIX = `Answer the following questions as best you can. You have access to the following tools:{tools}`;/** Create the tool instructions prompt */const TOOL_INSTRUCTIONS_TEMPLATE = `Use the following format in your response:Question: the input question you must answerThought: you should always think about what to doAction: the action to take, should be one of [{tool_names}]Action Input: the input to the actionObservation: the result of the action... (this Thought/Action/Action Input/Observation can repeat N times)Thought: I now know the final answerFinal Answer: the final answer to the original input question`;/** Create the suffix prompt */const SUFFIX = `Begin!Question: {input}Thought:`;async function formatMessages(  values: InputValues): Promise<Array<BaseMessage>> {  /** Check input and intermediate steps are both inside values */  if (!("input" in values) || !("intermediate_steps" in values)) {    throw new Error("Missing input or agent_scratchpad from values.");  }  /** Extract and case the intermediateSteps from values as Array<AgentStep> or an empty array if none are passed */  const intermediateSteps = values.intermediate_steps    ? (values.intermediate_steps as Array<AgentStep>)    : [];  /** Call the helper `formatLogToString` which returns the steps as a string  */  const agentScratchpad = formatLogToString(intermediateSteps);  /** Construct the tool strings */  const toolStrings = tools    .map((tool) => `${tool.name}: ${tool.description}`)    .join("\n");  const toolNames = tools.map((tool) => tool.name).join(",\n");  /** Create templates and format the instructions and suffix prompts */  const prefixTemplate = new PromptTemplate({    template: PREFIX,    inputVariables: ["tools"],  });  const instructionsTemplate = new PromptTemplate({    template: TOOL_INSTRUCTIONS_TEMPLATE,    inputVariables: ["tool_names"],  });  const suffixTemplate = new PromptTemplate({    template: SUFFIX,    inputVariables: ["input"],  });  /** Format both templates by passing in the input variables */  const formattedPrefix = await prefixTemplate.format({    tools: toolStrings,  });  const formattedInstructions = await instructionsTemplate.format({    tool_names: toolNames,  });  const formattedSuffix = await suffixTemplate.format({    input: values.input,  });  /** Construct the final prompt string */  const formatted = [    formattedPrefix,    formattedInstructions,    formattedSuffix,    agentScratchpad,  ].join("\n");  /** Return the message as a HumanMessage. */  return [new HumanMessage(formatted)];}/** Define the custom output parser */function customOutputParser(message: BaseMessage): AgentAction | AgentFinish {  const text = message.content;  if (typeof text !== "string") {    throw new Error(      `Message content is not a string. Received: ${JSON.stringify(        text,        null,        2      )}`    );  }  /** If the input includes "Final Answer" return as an instance of `AgentFinish` */  if (text.includes("Final Answer:")) {    const parts = text.split("Final Answer:");    const input = parts[parts.length - 1].trim();    const finalAnswers = { output: input };    return { log: text, returnValues: finalAnswers };  }  /** Use RegEx to extract any actions and their values */  const match = /Action: (.*)\nAction Input: (.*)/s.exec(text);  if (!match) {    throw new Error(`Could not parse LLM output: ${text}`);  }  /** Return as an instance of `AgentAction` */  return {    tool: match[1].trim(),    toolInput: match[2].trim().replace(/^"+|"+$/g, ""),    log: text,  };}/** Define the Runnable with LCEL */const runnable = RunnableSequence.from([  {    input: (values: InputValues) => values.input,    intermediate_steps: (values: InputValues) => values.steps,  },  formatMessages,  model,  customOutputParser,]);/** Pass the runnable to the `AgentExecutor` class as the agent */const executor = new AgentExecutor({  agent: runnable,  tools,});console.log("Loaded agent.");const input = `Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?`;console.log(`Executing with input "${input}"...`);const result = await executor.invoke({ input });console.log(`Got output ${result.output}`);/** * Got output Harry Styles' current age raised to the 0.23 power is approximately 2.1156502324195268. */

#### API Reference:

*   [AgentExecutor](https://api.js.langchain.com/classes/langchain_agents.AgentExecutor.html) from `langchain/agents`
*   [formatLogToString](https://api.js.langchain.com/functions/langchain_agents_format_scratchpad_log.formatLogToString.html) from `langchain/agents/format_scratchpad/log`
*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [Calculator](https://api.js.langchain.com/classes/langchain_community_tools_calculator.Calculator.html) from `@langchain/community/tools/calculator`
*   [PromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.PromptTemplate.html) from `@langchain/core/prompts`
*   [AgentAction](https://api.js.langchain.com/types/langchain_core_agents.AgentAction.html) from `@langchain/core/agents`
*   [AgentFinish](https://api.js.langchain.com/types/langchain_core_agents.AgentFinish.html) from `@langchain/core/agents`
*   [AgentStep](https://api.js.langchain.com/types/langchain_core_agents.AgentStep.html) from `@langchain/core/agents`
*   [BaseMessage](https://api.js.langchain.com/classes/langchain_core_messages.BaseMessage.html) from `@langchain/core/messages`
*   [HumanMessage](https://api.js.langchain.com/classes/langchain_core_messages.HumanMessage.html) from `@langchain/core/messages`
*   [InputValues](https://api.js.langchain.com/types/langchain_core_memory.InputValues.html) from `@langchain/core/memory`
*   [RunnableSequence](https://api.js.langchain.com/classes/langchain_core_runnables.RunnableSequence.html) from `@langchain/core/runnables`
*   [SerpAPI](https://api.js.langchain.com/classes/langchain_community_tools_serpapi.SerpAPI.html) from `@langchain/community/tools/serpapi`

With `LLMChain`
===============

    import {  AgentActionOutputParser,  AgentExecutor,  LLMSingleActionAgent,} from "langchain/agents";import { LLMChain } from "langchain/chains";import { ChatOpenAI } from "@langchain/openai";import { Calculator } from "@langchain/community/tools/calculator";import {  BaseChatPromptTemplate,  SerializedBasePromptTemplate,  renderTemplate,} from "@langchain/core/prompts";import { AgentAction, AgentFinish, AgentStep } from "@langchain/core/agents";import { BaseMessage, HumanMessage } from "@langchain/core/messages";import { InputValues } from "@langchain/core/memory";import { PartialValues } from "@langchain/core/utils/types";import { Tool } from "@langchain/core/tools";import { SerpAPI } from "@langchain/community/tools/serpapi";const PREFIX = `Answer the following questions as best you can. You have access to the following tools:`;const formatInstructions = (  toolNames: string) => `Use the following format in your response:Question: the input question you must answerThought: you should always think about what to doAction: the action to take, should be one of [${toolNames}]Action Input: the input to the actionObservation: the result of the action... (this Thought/Action/Action Input/Observation can repeat N times)Thought: I now know the final answerFinal Answer: the final answer to the original input question`;const SUFFIX = `Begin!Question: {input}Thought:{agent_scratchpad}`;class CustomPromptTemplate extends BaseChatPromptTemplate {  tools: Tool[];  constructor(args: { tools: Tool[]; inputVariables: string[] }) {    super({ inputVariables: args.inputVariables });    this.tools = args.tools;  }  _getPromptType(): string {    return "chat";  }  async formatMessages(values: InputValues): Promise<BaseMessage[]> {    /** Construct the final template */    const toolStrings = this.tools      .map((tool) => `${tool.name}: ${tool.description}`)      .join("\n");    const toolNames = this.tools.map((tool) => tool.name).join("\n");    const instructions = formatInstructions(toolNames);    const template = [PREFIX, toolStrings, instructions, SUFFIX].join("\n\n");    /** Construct the agent_scratchpad */    const intermediateSteps = values.intermediate_steps as AgentStep[];    const agentScratchpad = intermediateSteps.reduce(      (thoughts, { action, observation }) =>        thoughts +        [action.log, `\nObservation: ${observation}`, "Thought:"].join("\n"),      ""    );    const newInput = { agent_scratchpad: agentScratchpad, ...values };    /** Format the template. */    const formatted = renderTemplate(template, "f-string", newInput);    return [new HumanMessage(formatted)];  }  partial(_values: PartialValues): Promise<BaseChatPromptTemplate> {    throw new Error("Not implemented");  }  serialize(): SerializedBasePromptTemplate {    throw new Error("Not implemented");  }}class CustomOutputParser extends AgentActionOutputParser {  lc_namespace = ["langchain", "agents", "custom_llm_agent_chat"];  async parse(text: string): Promise<AgentAction | AgentFinish> {    if (text.includes("Final Answer:")) {      const parts = text.split("Final Answer:");      const input = parts[parts.length - 1].trim();      const finalAnswers = { output: input };      return { log: text, returnValues: finalAnswers };    }    const match = /Action: (.*)\nAction Input: (.*)/s.exec(text);    if (!match) {      throw new Error(`Could not parse LLM output: ${text}`);    }    return {      tool: match[1].trim(),      toolInput: match[2].trim().replace(/^"+|"+$/g, ""),      log: text,    };  }  getFormatInstructions(): string {    throw new Error("Not implemented");  }}export const run = async () => {  const model = new ChatOpenAI({ temperature: 0 });  const tools = [    new SerpAPI(process.env.SERPAPI_API_KEY, {      location: "Austin,Texas,United States",      hl: "en",      gl: "us",    }),    new Calculator(),  ];  const llmChain = new LLMChain({    prompt: new CustomPromptTemplate({      tools,      inputVariables: ["input", "agent_scratchpad"],    }),    llm: model,  });  const agent = new LLMSingleActionAgent({    llmChain,    outputParser: new CustomOutputParser(),    stop: ["\nObservation"],  });  const executor = new AgentExecutor({    agent,    tools,  });  console.log("Loaded agent.");  const input = `Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?`;  console.log(`Executing with input "${input}"...`);  const result = await executor.invoke({ input });  console.log(`Got output ${result.output}`);};run();

#### API Reference:

*   [AgentActionOutputParser](https://api.js.langchain.com/classes/langchain_agents.AgentActionOutputParser.html) from `langchain/agents`
*   [AgentExecutor](https://api.js.langchain.com/classes/langchain_agents.AgentExecutor.html) from `langchain/agents`
*   [LLMSingleActionAgent](https://api.js.langchain.com/classes/langchain_agents.LLMSingleActionAgent.html) from `langchain/agents`
*   [LLMChain](https://api.js.langchain.com/classes/langchain_chains.LLMChain.html) from `langchain/chains`
*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [Calculator](https://api.js.langchain.com/classes/langchain_community_tools_calculator.Calculator.html) from `@langchain/community/tools/calculator`
*   [BaseChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.BaseChatPromptTemplate.html) from `@langchain/core/prompts`
*   [SerializedBasePromptTemplate](https://api.js.langchain.com/types/langchain_core_prompts.SerializedBasePromptTemplate.html) from `@langchain/core/prompts`
*   [renderTemplate](https://api.js.langchain.com/functions/langchain_core_prompts.renderTemplate.html) from `@langchain/core/prompts`
*   [AgentAction](https://api.js.langchain.com/types/langchain_core_agents.AgentAction.html) from `@langchain/core/agents`
*   [AgentFinish](https://api.js.langchain.com/types/langchain_core_agents.AgentFinish.html) from `@langchain/core/agents`
*   [AgentStep](https://api.js.langchain.com/types/langchain_core_agents.AgentStep.html) from `@langchain/core/agents`
*   [BaseMessage](https://api.js.langchain.com/classes/langchain_core_messages.BaseMessage.html) from `@langchain/core/messages`
*   [HumanMessage](https://api.js.langchain.com/classes/langchain_core_messages.HumanMessage.html) from `@langchain/core/messages`
*   [InputValues](https://api.js.langchain.com/types/langchain_core_memory.InputValues.html) from `@langchain/core/memory`
*   [PartialValues](https://api.js.langchain.com/types/langchain_core_utils_types.PartialValues.html) from `@langchain/core/utils/types`
*   [Tool](https://api.js.langchain.com/classes/langchain_core_tools.Tool.html) from `@langchain/core/tools`
*   [SerpAPI](https://api.js.langchain.com/classes/langchain_community_tools_serpapi.SerpAPI.html) from `@langchain/community/tools/serpapi`

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Custom LLM Agent

](/v0.1/docs/modules/agents/how_to/custom_llm_agent/)[

Next

Custom MRKL agent

](/v0.1/docs/modules/agents/how_to/custom_mrkl_agent/)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.