!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [Tutorials](/v0.2/docs/tutorials/)
*   Build an Agent

On this page

Build an Agent
==============

By themselves, language models can't take actions - they just output text. A big use case for LangChain is creating **agents**. Agents are systems that use an LLM as a reasoning enginer to determine which actions to take and what the inputs to those actions should be. The results of those actions can then be fed back into the agent and it determine whether more actions are needed, or whether it is okay to finish.

In this tutorial we will build an agent that can interact with multiple different tools: one being a local database, the other being a search engine. You will be able to ask this agent questions, watch it call tools, and have conversations with it.

Concepts[​](#concepts "Direct link to Concepts")
------------------------------------------------

Concepts we will cover are:

*   Using [language models](/v0.2/docs/concepts/#chat-models), in particular their tool calling ability
*   Creating a [Retriever](/v0.2/docs/concepts/#retrievers) to expose specific information to our agent
*   Using a Search [Tool](/v0.2/docs/concepts/#tools) to look up things online
*   Using [LangGraph Agents](/v0.2/docs/concepts/#agents) which use an LLM to think about what to do and then execute upon that
*   Debugging and tracing your application using [LangSmith](/v0.2/docs/concepts/#langsmith)

Setup: LangSmith[​](#setup-langsmith "Direct link to Setup: LangSmith")
-----------------------------------------------------------------------

By definition, agents take a self-determined, input-dependent sequence of steps before returning a user-facing output. This makes debugging these systems particularly tricky, and observability particularly important. [LangSmith](https://smith.langchain.com) is especially useful for such cases.

When building with LangChain, all steps will automatically be traced in LangSmith. To set up LangSmith we just need set the following environment variables:

    export LANGCHAIN_TRACING_V2="true"export LANGCHAIN_API_KEY="<your-api-key>"

Define tools[​](#define-tools "Direct link to Define tools")
------------------------------------------------------------

We first need to create the tools we want to use. We will use two tools: [Tavily](https://app.tavily.com) (to search online) and then a retriever over a local index we will create.

### [Tavily](https://app.tavily.com)[​](#tavily "Direct link to tavily")

We have a built-in tool in LangChain to easily use Tavily search engine as tool. Note that this requires a Tavily API key set as an environment variable named `TAVILY_API_KEY` - they have a free tier, but if you don’t have one or don’t want to create one, you can always ignore this step.

    import { TavilySearchResults } from "@langchain/community/tools/tavily_search";const searchTool = new TavilySearchResults();const toolResult = await searchTool.invoke("what is the weather in SF?");console.log(toolResult);/*  [{"title":"Weather in December 2023 in San Francisco, California, USA","url":"https://www.timeanddate.com/weather/@5391959/historic?month=12&year=2023","content":"Currently: 52 °F. Broken clouds. (Weather station: San Francisco International Airport, USA). See more current weather Select month: December 2023 Weather in San Francisco — Graph °F Sun, Dec 17 Lo:55 6 pm Hi:57 4 Mon, Dec 18 Lo:54 12 am Hi:55 7 Lo:54 6 am Hi:55 10 Lo:57 12 pm Hi:64 9 Lo:63 6 pm Hi:64 14 Tue, Dec 19 Lo:61","score":0.96006},...]*/

### Retriever[​](#retriever "Direct link to Retriever")

We will also create a retriever over some data of our own. For a deeper explanation of each step here, see our [how to guides](/v0.2/docs/how_to/).

    import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";import { MemoryVectorStore } from "langchain/vectorstores/memory";import { OpenAIEmbeddings } from "@langchain/openai";const loader = new CheerioWebBaseLoader(  "https://docs.smith.langchain.com/user_guide");const rawDocs = await loader.load();const splitter = new RecursiveCharacterTextSplitter({  chunkSize: 1000,  chunkOverlap: 200,});const docs = await splitter.splitDocuments(rawDocs);const vectorstore = await MemoryVectorStore.fromDocuments(  docs,  new OpenAIEmbeddings());const retriever = vectorstore.asRetriever();const retrieverResult = await retriever.getRelevantDocuments(  "how to upload a dataset");console.log(retrieverResult[0]);/*  Document {    pageContent: "your application progresses through the beta testing phase, it's essential to continue collecting data to refine and improve its performance. LangSmith enables you to add runs as examples to datasets (from both the project page and within an annotation queue), expanding your test coverage on real-world scenarios. This is a key benefit in having your logging system and your evaluation/testing system in the same platform.Production​Closely inspecting key data points, growing benchmarking datasets, annotating traces, and drilling down into important data in trace view are workflows you’ll also want to do once your app hits production. However, especially at the production stage, it’s crucial to get a high-level overview of application performance with respect to latency, cost, and feedback scores. This ensures that it's delivering desirable results at scale.Monitoring and A/B Testing​LangSmith provides monitoring charts that allow you to track key metrics over time. You can expand to",    metadata: {      source: 'https://docs.smith.langchain.com/user_guide',      loc: { lines: [Object] }    }  }*/

Now that we have populated our index that we will do doing retrieval over, we can easily turn it into a tool (the format needed for an agent to properly use it):

    import { createRetrieverTool } from "langchain/tools/retriever";const retrieverTool = createRetrieverTool(retriever, {  name: "langsmith_search",  description:    "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!",});

### Tools[​](#tools "Direct link to Tools")

Now that we have created both, we can create a list of tools that we will use downstream:

    const tools = [searchTool, retrieverTool];

Create the agent[​](#create-the-agent "Direct link to Create the agent")
------------------------------------------------------------------------

Now that we have defined the tools, we can create the agent. We will be using an OpenAI Functions agent - for more information on this type of agent, as well as other options, see [this guide](https://js.langchain.com/v0.1/docs/modules/agents/agent_types/).

First, we choose the LLM we want to be guiding the agent.

    import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({  model: "gpt-3.5-turbo",  temperature: 0,});

Next, we choose the prompt we want to use to guide the agent:

    import type { ChatPromptTemplate } from "@langchain/core/prompts";import { pull } from "langchain/hub";// Get the prompt to use - you can modify this!// If you want to see the prompt in full, you can at:// https://smith.langchain.com/hub/hwchase17/openai-functions-agentconst prompt = await pull<ChatPromptTemplate>(  "hwchase17/openai-functions-agent");

Now, we can initalize the agent with the LLM, the prompt, and the tools. The agent is responsible for taking in input and deciding what actions to take. Crucially, the Agent does not execute those actions - that is done by the AgentExecutor (next step). For more information about how to thing about these components, see our [conceptual guide](/v0.2/docs/concepts#agents).

    import { createOpenAIFunctionsAgent } from "langchain/agents";const agent = await createOpenAIFunctionsAgent({  llm,  tools,  prompt,});

Finally, we combine the agent (the brains) with the tools inside the AgentExecutor (which will repeatedly call the agent and execute tools). For more information about how to thing about these components, see our [conceptual guide](/v0.2/docs/concepts#agents).

    import { AgentExecutor } from "langchain/agents";const agentExecutor = new AgentExecutor({  agent,  tools,});

Run the agent[​](#run-the-agent "Direct link to Run the agent")
---------------------------------------------------------------

We can now run the agent on a few queries! Note that for now, these are all stateless queries (it won’t remember previous interactions).

    const result1 = await agentExecutor.invoke({  input: "hi!",});console.log(result1);/*  [chain/start] [1:chain:AgentExecutor] Entering Chain run with input: {    "input": "hi!"  }  [chain/end] [1:chain:AgentExecutor] [1.36s] Exiting Chain run with output: {    "output": "Hello! How can I assist you today?"  }  {    input: 'hi!',    output: 'Hello! How can I assist you today?'  }*/

    const result2 = await agentExecutor.invoke({  input: "how can langsmith help with testing?",});console.log(result2);/*  [chain/start] [1:chain:AgentExecutor] Entering Chain run with input: {    "input": "how can langsmith help with testing?"  }  [chain/end] [1:chain:AgentExecutor > 2:chain:RunnableAgent > 7:parser:OpenAIFunctionsAgentOutputParser] [66ms] Exiting Chain run with output: {    "tool": "langsmith_search",    "toolInput": {      "query": "how can LangSmith help with testing?"    },    "log": "Invoking \"langsmith_search\" with {\"query\":\"how can LangSmith help with testing?\"}\n",    "messageLog": [      {        "lc": 1,        "type": "constructor",        "id": [          "langchain_core",          "messages",          "AIMessage"        ],        "kwargs": {          "content": "",          "additional_kwargs": {            "function_call": {              "name": "langsmith_search",              "arguments": "{\"query\":\"how can LangSmith help with testing?\"}"            }          }        }      }    ]  }  [tool/start] [1:chain:AgentExecutor > 8:tool:langsmith_search] Entering Tool run with input: "{"query":"how can LangSmith help with testing?"}"  [retriever/start] [1:chain:AgentExecutor > 8:tool:langsmith_search > 9:retriever:VectorStoreRetriever] Entering Retriever run with input: {    "query": "how can LangSmith help with testing?"  }  [retriever/end] [1:chain:AgentExecutor > 8:tool:langsmith_search > 9:retriever:VectorStoreRetriever] [294ms] Exiting Retriever run with output: {    "documents": [      {        "pageContent": "You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring​After all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be assigned string tags or key-value metadata, allowing you to attach correlation ids or AB test variants, and filter runs accordingly.We’ve also made it possible to associate feedback programmatically with runs. This means that if your application has a thumbs up/down button on it, you can use that to log feedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing — mirroring the",        "metadata": {          "source": "https://docs.smith.langchain.com/user_guide",          "loc": {            "lines": {              "from": 11,              "to": 11            }          }        }      },      {        "pageContent": "the time that we do… it’s so helpful. We can use LangSmith to debug:An unexpected end resultWhy an agent is loopingWhy a chain was slower than expectedHow many tokens an agent usedDebugging​Debugging LLMs, chains, and agents can be tough. LangSmith helps solve the following pain points:What was the exact input to the LLM?​LLM calls are often tricky and non-deterministic. The inputs/outputs may seem straightforward, given they are technically string → string (or chat messages → chat message), but this can be misleading as the input string is usually constructed from a combination of user input and auxiliary functions.Most inputs to an LLM call are a combination of some type of fixed template along with input variables. These input variables could come directly from user input or from an auxiliary function (like retrieval). By the time these input variables go into the LLM they will have been converted to a string format, but often times they are not naturally represented as a string",        "metadata": {          "source": "https://docs.smith.langchain.com/user_guide",          "loc": {            "lines": {              "from": 3,              "to": 3            }          }        }      },      {        "pageContent": "inputs, and see what happens. At some point though, our application is performing\nwell and we want to be more rigorous about testing changes. We can use a dataset\nthat we’ve constructed along the way (see above). Alternatively, we could spend some\ntime constructing a small dataset by hand. For these situations, LangSmith simplifies",        "metadata": {          "source": "https://docs.smith.langchain.com/user_guide",          "loc": {            "lines": {              "from": 4,              "to": 7            }          }        }      },      {        "pageContent": "feedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing — mirroring the debug mode approach.We’ve provided several examples in the LangSmith documentation for extracting insights from logged runs. In addition to guiding you on performing this task yourself, we also provide examples of integrating with third parties for this purpose. We're eager to expand this area in the coming months! If you have ideas for either -- an open-source way to evaluate, or are building a company that wants to do analytics over these runs, please reach out.Exporting datasets​LangSmith makes it easy to curate datasets. However, these aren’t just useful inside LangSmith; they can be exported for use in other contexts. Notable applications include exporting for use in OpenAI Evals or fine-tuning, such as with FireworksAI.To set up tracing in Deno, web browsers, or other runtime",        "metadata": {          "source": "https://docs.smith.langchain.com/user_guide",          "loc": {            "lines": {              "from": 11,              "to": 11            }          }        }      }    ]  }  [chain/start] [1:chain:AgentExecutor > 10:chain:RunnableAgent] Entering Chain run with input: {    "input": "how can langsmith help with testing?",    "steps": [      {        "action": {          "tool": "langsmith_search",          "toolInput": {            "query": "how can LangSmith help with testing?"          },          "log": "Invoking \"langsmith_search\" with {\"query\":\"how can LangSmith help with testing?\"}\n",          "messageLog": [            {              "lc": 1,              "type": "constructor",              "id": [                "langchain_core",                "messages",                "AIMessage"              ],              "kwargs": {                "content": "",                "additional_kwargs": {                  "function_call": {                    "name": "langsmith_search",                    "arguments": "{\"query\":\"how can LangSmith help with testing?\"}"                  }                }              }            }          ]        },        "observation": "You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring​After all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be assigned string tags or key-value metadata, allowing you to attach correlation ids or AB test variants, and filter runs accordingly.We’ve also made it possible to associate feedback programmatically with runs. This means that if your application has a thumbs up/down button on it, you can use that to log feedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing — mirroring the\n\nthe time that we do… it’s so helpful. We can use LangSmith to debug:An unexpected end resultWhy an agent is loopingWhy a chain was slower than expectedHow many tokens an agent usedDebugging​Debugging LLMs, chains, and agents can be tough. LangSmith helps solve the following pain points:What was the exact input to the LLM?​LLM calls are often tricky and non-deterministic. The inputs/outputs may seem straightforward, given they are technically string → string (or chat messages → chat message), but this can be misleading as the input string is usually constructed from a combination of user input and auxiliary functions.Most inputs to an LLM call are a combination of some type of fixed template along with input variables. These input variables could come directly from user input or from an auxiliary function (like retrieval). By the time these input variables go into the LLM they will have been converted to a string format, but often times they are not naturally represented as a string\n\ninputs, and see what happens. At some point though, our application is performing\nwell and we want to be more rigorous about testing changes. We can use a dataset\nthat we’ve constructed along the way (see above). Alternatively, we could spend some\ntime constructing a small dataset by hand. For these situations, LangSmith simplifies\n\nfeedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing — mirroring the debug mode approach.We’ve provided several examples in the LangSmith documentation for extracting insights from logged runs. In addition to guiding you on performing this task yourself, we also provide examples of integrating with third parties for this purpose. We're eager to expand this area in the coming months! If you have ideas for either -- an open-source way to evaluate, or are building a company that wants to do analytics over these runs, please reach out.Exporting datasets​LangSmith makes it easy to curate datasets. However, these aren’t just useful inside LangSmith; they can be exported for use in other contexts. Notable applications include exporting for use in OpenAI Evals or fine-tuning, such as with FireworksAI.To set up tracing in Deno, web browsers, or other runtime"      }    ]  }  [chain/end] [1:chain:AgentExecutor] [5.83s] Exiting Chain run with output: {    "input": "how can langsmith help with testing?",    "output": "LangSmith can help with testing in several ways:\n\n1. Debugging: LangSmith can be used to debug unexpected end results, agent loops, slow chains, and token usage. It helps in pinpointing underperforming data points and tracking performance over time.\n\n2. Monitoring: LangSmith can monitor applications by logging all traces, visualizing latency and token usage statistics, and troubleshooting specific issues as they arise. It also allows for associating feedback programmatically with runs, which can be used to track performance over time.\n\n3. Exporting Datasets: LangSmith makes it easy to curate datasets, which can be exported for use in other contexts such as OpenAI Evals or fine-tuning with FireworksAI.\n\nOverall, LangSmith simplifies the process of testing changes, constructing datasets, and extracting insights from logged runs, making it a valuable tool for testing and evaluation."  }  {    input: 'how can langsmith help with testing?',    output: 'LangSmith can help with testing in several ways:\n' +      '\n' +      '1. Initial Test Set: LangSmith allows developers to create datasets of inputs and reference outputs to run tests on their LLM applications. These test cases can be uploaded in bulk, created on the fly, or exported from application traces.\n' +      '\n' +      "2. Comparison View: When making changes to your applications, LangSmith provides a comparison view to see whether you've regressed with respect to your initial test cases. This is helpful for evaluating changes in prompts, retrieval strategies, or model choices.\n" +      '\n' +      '3. Monitoring and A/B Testing: LangSmith provides monitoring charts to track key metrics over time and allows for A/B testing changes in prompt, model, or retrieval strategy.\n' +      '\n' +      '4. Debugging: LangSmith offers tracing and debugging information at each step of an LLM sequence, making it easier to identify and root-cause issues when things go wrong.\n' +      '\n' +      '5. Beta Testing and Production: LangSmith enables the addition of runs as examples to datasets, expanding test coverage on real-world scenarios. It also provides monitoring for application performance with respect to latency, cost, and feedback scores at the production stage.\n' +      '\n' +      'Overall, LangSmith provides comprehensive testing and monitoring capabilities for LLM applications.'  }*/

Adding in memory[​](#adding-in-memory "Direct link to Adding in memory")
------------------------------------------------------------------------

As mentioned earlier, this agent is stateless. This means it does not remember previous interactions. To give it memory we need to pass in previous `chat_history`.

**Note:** the input variable below needs to be called `chat_history` because of the prompt we are using. If we use a different prompt, we could change the variable name.

    const result3 = await agentExecutor.invoke({  input: "hi! my name is cob.",  chat_history: [],});console.log(result3);/*  {    input: 'hi! my name is cob.',    chat_history: [],    output: "Hello Cob! It's nice to meet you. How can I assist you today?"  }*/

    import { HumanMessage, AIMessage } from "@langchain/core/messages";const result4 = await agentExecutor.invoke({  input: "what's my name?",  chat_history: [    new HumanMessage("hi! my name is cob."),    new AIMessage("Hello Cob! How can I assist you today?"),  ],});console.log(result4);/*  {    input: "what's my name?",    chat_history: [      HumanMessage {        content: 'hi! my name is cob.',        additional_kwargs: {}      },      AIMessage {        content: 'Hello Cob! How can I assist you today?',        additional_kwargs: {}      }    ],    output: 'Your name is Cob. How can I assist you today, Cob?'  }*/

If we want to keep track of these messages automatically, we can wrap this in a RunnableWithMessageHistory. For more information on how to use this, see [this guide](/v0.2/docs/how_to/message_history/).

    import { ChatMessageHistory } from "langchain/stores/message/in_memory";import { RunnableWithMessageHistory } from "@langchain/core/runnables";const messageHistory = new ChatMessageHistory();const agentWithChatHistory = new RunnableWithMessageHistory({  runnable: agentExecutor,  // This is needed because in most real world scenarios, a session id is needed per user.  // It isn't really used here because we are using a simple in memory ChatMessageHistory.  getMessageHistory: (_sessionId) => messageHistory,  inputMessagesKey: "input",  historyMessagesKey: "chat_history",});const result5 = await agentWithChatHistory.invoke(  {    input: "hi! i'm cob",  },  {    // This is needed because in most real world scenarios, a session id is needed per user.    // It isn't really used here because we are using a simple in memory ChatMessageHistory.    configurable: {      sessionId: "foo",    },  });console.log(result5);/*  {    input: "hi! i'm cob",    chat_history: [      HumanMessage {        content: "hi! i'm cob",        additional_kwargs: {}      },      AIMessage {        content: 'Hello Cob! How can I assist you today?',        additional_kwargs: {}      }    ],    output: 'Hello Cob! How can I assist you today?'  }*/

    const result6 = await agentWithChatHistory.invoke(  {    input: "what's my name?",  },  {    // This is needed because in most real world scenarios, a session id is needed per user.    // It isn't really used here because we are using a simple in memory ChatMessageHistory.    configurable: {      sessionId: "foo",    },  });console.log(result6);/*  {    input: "what's my name?",    chat_history: [      HumanMessage {        content: "hi! i'm cob",        additional_kwargs: {}      },      AIMessage {        content: 'Hello Cob! How can I assist you today?',        additional_kwargs: {}      },      HumanMessage {        content: "what's my name?",        additional_kwargs: {}      },      AIMessage {        content: 'Your name is Cob. How can I assist you today, Cob?',        additional_kwargs: {}      }    ],    output: 'Your name is Cob. How can I assist you today, Cob?'  }*/

Conclusion[​](#conclusion "Direct link to Conclusion")
------------------------------------------------------

That’s a wrap! In this quick start we covered how to create a simple agent. Agents are a complex topic, and there’s lot to learn! Head back to the [main agent page](/v0.2/docs/how_to/agent_executor/) to find more resources on conceptual guides, different types of agents, how to create custom tools, and more!

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

Build a Chatbot

](/v0.2/docs/tutorials/chatbot)[

Next

Build an Extraction Chain

](/v0.2/docs/tutorials/extraction)

*   [Concepts](#concepts)
*   [Setup: LangSmith](#setup-langsmith)
*   [Define tools](#define-tools)
    *   [Tavily](#tavily)
    *   [Retriever](#retriever)
    *   [Tools](#tools)
*   [Create the agent](#create-the-agent)
*   [Run the agent](#run-the-agent)
*   [Adding in memory](#adding-in-memory)
*   [Conclusion](#conclusion)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.