!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to do retrieval

On this page

How to do retrieval
===================

Prerequisites

This guide assumes familiarity with the following:

*   [Chatbots](/v0.2/docs/tutorials/chatbot)
*   [Retrieval-augmented generation](/v0.2/docs/tutorials/rag)

Retrieval is a common technique chatbots use to augment their responses with data outside a chat model’s training data. This section will cover how to implement retrieval in the context of chatbots, but it’s worth noting that retrieval is a very subtle and deep topic.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

You’ll need to install a few packages, and set any LLM API keys:

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai cheerio

    yarn add @langchain/openai cheerio

    pnpm add @langchain/openai cheerio

Let’s also set up a chat model that we’ll use for the below examples.

### Pick your chat model:

*   OpenAI
*   Anthropic
*   FireworksAI
*   MistralAI
*   Groq
*   VertexAI

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai 

    yarn add @langchain/openai 

    pnpm add @langchain/openai 

#### Add environment variables

    OPENAI_API_KEY=your-api-key

#### Instantiate the model

    import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({  model: "gpt-3.5-turbo",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/anthropic 

    yarn add @langchain/anthropic 

    pnpm add @langchain/anthropic 

#### Add environment variables

    ANTHROPIC_API_KEY=your-api-key

#### Instantiate the model

    import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({  model: "claude-3-sonnet-20240229",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/community 

    yarn add @langchain/community 

    pnpm add @langchain/community 

#### Add environment variables

    FIREWORKS_API_KEY=your-api-key

#### Instantiate the model

    import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({  model: "accounts/fireworks/models/firefunction-v1",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/mistralai 

    yarn add @langchain/mistralai 

    pnpm add @langchain/mistralai 

#### Add environment variables

    MISTRAL_API_KEY=your-api-key

#### Instantiate the model

    import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({  model: "mistral-large-latest",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/groq 

    yarn add @langchain/groq 

    pnpm add @langchain/groq 

#### Add environment variables

    GROQ_API_KEY=your-api-key

#### Instantiate the model

    import { ChatGroq } from "@langchain/groq";const llm = new ChatGroq({  model: "mixtral-8x7b-32768",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/google-vertexai 

    yarn add @langchain/google-vertexai 

    pnpm add @langchain/google-vertexai 

#### Add environment variables

    GOOGLE_APPLICATION_CREDENTIALS=credentials.json

#### Instantiate the model

    import { ChatVertexAI } from "@langchain/google-vertexai";const llm = new ChatVertexAI({  model: "gemini-1.5-pro",  temperature: 0});

Creating a retriever[​](#creating-a-retriever "Direct link to Creating a retriever")
------------------------------------------------------------------------------------

We’ll use [the LangSmith documentation](https://docs.smith.langchain.com) as source material and store the content in a vectorstore for later retrieval. Note that this example will gloss over some of the specifics around parsing and storing a data source - you can see more [in-depth documentation on creating retrieval systems here](/v0.2/docs/how_to/#qa-with-rag).

Let’s use a document loader to pull text from the docs:

    import "cheerio";import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";const loader = new CheerioWebBaseLoader(  "https://docs.smith.langchain.com/user_guide");const rawDocs = await loader.load();rawDocs[0].pageContent.length;

    36687

Next, we split it into smaller chunks that the LLM’s context window can handle and store it in a vector database:

    import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";const textSplitter = new RecursiveCharacterTextSplitter({  chunkSize: 500,  chunkOverlap: 0,});const allSplits = await textSplitter.splitDocuments(rawDocs);

Then we embed and store those chunks in a vector database:

    import { OpenAIEmbeddings } from "@langchain/openai";import { MemoryVectorStore } from "langchain/vectorstores/memory";const vectorstore = await MemoryVectorStore.fromDocuments(  allSplits,  new OpenAIEmbeddings());

And finally, let’s create a retriever from our initialized vectorstore:

    const retriever = vectorstore.asRetriever(4);const docs = await retriever.invoke("how can langsmith help with testing?");console.log(docs);

    [  Document {    pageContent: "These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L"... 294 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 7, to: 7 } }    }  },  Document {    pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 6, to: 6 } }    }  },  Document {    pageContent: "will help in curation of test cases that can help track regressions/improvements and development of "... 393 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 11, to: 11 } }    }  },  Document {    pageContent: "that time period — this is especially handy for debugging production issues.LangSmith also allows fo"... 396 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 11, to: 11 } }    }  }]

We can see that invoking the retriever above results in some parts of the LangSmith docs that contain information about testing that our chatbot can use as context when answering questions. And now we’ve got a retriever that can return related data from the LangSmith docs!

Document chains[​](#document-chains "Direct link to Document chains")
---------------------------------------------------------------------

Now that we have a retriever that can return LangChain docs, let’s create a chain that can use them as context to answer questions. We’ll use a `createStuffDocumentsChain` helper function to “stuff” all of the input documents into the prompt. It will also handle formatting the docs as strings.

In addition to a chat model, the function also expects a prompt that has a `context` variable, as well as a placeholder for chat history messages named `messages`. We’ll create an appropriate prompt and pass it as shown below:

    import { createStuffDocumentsChain } from "langchain/chains/combine_documents";import {  ChatPromptTemplate,  MessagesPlaceholder,} from "@langchain/core/prompts";const SYSTEM_TEMPLATE = `Answer the user's questions based on the below context. If the context doesn't contain any relevant information to the question, don't make something up and just say "I don't know":<context>{context}</context>`;const questionAnsweringPrompt = ChatPromptTemplate.fromMessages([  ["system", SYSTEM_TEMPLATE],  new MessagesPlaceholder("messages"),]);const documentChain = await createStuffDocumentsChain({  llm,  prompt: questionAnsweringPrompt,});

We can invoke this `documentChain` by itself to answer questions. Let’s use the docs we retrieved above and the same question, `how can langsmith help with testing?`:

    import { HumanMessage, AIMessage } from "@langchain/core/messages";await documentChain.invoke({  messages: [new HumanMessage("Can LangSmith help test my LLM applications?")],  context: docs,});

    "Yes, LangSmith can help test your LLM applications. It allows developers to create datasets, which a"... 229 more characters

Looks good! For comparison, we can try it with no context docs and compare the result:

    await documentChain.invoke({  messages: [new HumanMessage("Can LangSmith help test my LLM applications?")],  context: [],});

    "I don't know."

We can see that the LLM does not return any results.

Retrieval chains[​](#retrieval-chains "Direct link to Retrieval chains")
------------------------------------------------------------------------

Let’s combine this document chain with the retriever. Here’s one way this can look:

    import type { BaseMessage } from "@langchain/core/messages";import {  RunnablePassthrough,  RunnableSequence,} from "@langchain/core/runnables";const parseRetrieverInput = (params: { messages: BaseMessage[] }) => {  return params.messages[params.messages.length - 1].content;};const retrievalChain = RunnablePassthrough.assign({  context: RunnableSequence.from([parseRetrieverInput, retriever]),}).assign({  answer: documentChain,});

Given a list of input messages, we extract the content of the last message in the list and pass that to the retriever to fetch some documents. Then, we pass those documents as context to our document chain to generate a final response.

Invoking this chain combines both steps outlined above:

    await retrievalChain.invoke({  messages: [new HumanMessage("Can LangSmith help test my LLM applications?")],});

    {  messages: [    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Can LangSmith help test my LLM applications?",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Can LangSmith help test my LLM applications?",      name: undefined,      additional_kwargs: {},      response_metadata: {}    }  ],  context: [    Document {      pageContent: "These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L"... 294 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "this guide, we’ll highlight the breadth of workflows LangSmith supports and how they fit into each s"... 343 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "The ability to rapidly understand how the model is performing — and debug where it is failing — is i"... 138 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    }  ],  answer: "Yes, LangSmith can help test your LLM applications. It allows developers to create datasets, which a"... 297 more characters}

Looks good!

Query transformation[​](#query-transformation "Direct link to Query transformation")
------------------------------------------------------------------------------------

Our retrieval chain is capable of answering questions about LangSmith, but there’s a problem - chatbots interact with users conversationally, and therefore have to deal with followup questions.

The chain in its current form will struggle with this. Consider a followup question to our original question like `Tell me more!`. If we invoke our retriever with that query directly, we get documents irrelevant to LLM application testing:

    await retriever.invoke("Tell me more!");

    [  Document {    pageContent: "Oftentimes, changes in the prompt, retrieval strategy, or model choice can have huge implications in"... 40 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 8, to: 8 } }    }  },  Document {    pageContent: "This allows you to quickly test out different prompts and models. You can open the playground from a"... 37 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 10, to: 10 } }    }  },  Document {    pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 6, to: 6 } }    }  },  Document {    pageContent: "together, making it easier to track the performance of and annotate your application across multiple"... 244 more characters,    metadata: {      source: "https://docs.smith.langchain.com/user_guide",      loc: { lines: { from: 11, to: 11 } }    }  }]

This is because the retriever has no innate concept of state, and will only pull documents most similar to the query given. To solve this, we can transform the query into a standalone query without any external references an LLM.

Here’s an example:

    const queryTransformPrompt = ChatPromptTemplate.fromMessages([  new MessagesPlaceholder("messages"),  [    "user",    "Given the above conversation, generate a search query to look up in order to get information relevant to the conversation. Only respond with the query, nothing else.",  ],]);const queryTransformationChain = queryTransformPrompt.pipe(llm);await queryTransformationChain.invoke({  messages: [    new HumanMessage("Can LangSmith help test my LLM applications?"),    new AIMessage(      "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs. Additionally, LangSmith can be used to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise."    ),    new HumanMessage("Tell me more!"),  ],});

    AIMessage {  lc_serializable: true,  lc_kwargs: {    content: '"LangSmith LLM application testing and evaluation features"',    tool_calls: [],    invalid_tool_calls: [],    additional_kwargs: { function_call: undefined, tool_calls: undefined },    response_metadata: {}  },  lc_namespace: [ "langchain_core", "messages" ],  content: '"LangSmith LLM application testing and evaluation features"',  name: undefined,  additional_kwargs: { function_call: undefined, tool_calls: undefined },  response_metadata: {    tokenUsage: { completionTokens: 11, promptTokens: 144, totalTokens: 155 },    finish_reason: "stop"  },  tool_calls: [],  invalid_tool_calls: []}

Awesome! That transformed query would pull up context documents related to LLM application testing.

Let’s add this to our retrieval chain. We can wrap our retriever as follows:

    import { RunnableBranch } from "@langchain/core/runnables";import { StringOutputParser } from "@langchain/core/output_parsers";const queryTransformingRetrieverChain = RunnableBranch.from([  [    (params: { messages: BaseMessage[] }) => params.messages.length === 1,    RunnableSequence.from([parseRetrieverInput, retriever]),  ],  queryTransformPrompt.pipe(llm).pipe(new StringOutputParser()).pipe(retriever),]).withConfig({ runName: "chat_retriever_chain" });

Then, we can use this query transformation chain to make our retrieval chain better able to handle such followup questions:

    const conversationalRetrievalChain = RunnablePassthrough.assign({  context: queryTransformingRetrieverChain,}).assign({  answer: documentChain,});

Awesome! Let’s invoke this new chain with the same inputs as earlier:

    await conversationalRetrievalChain.invoke({  messages: [new HumanMessage("Can LangSmith help test my LLM applications?")],});

    {  messages: [    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Can LangSmith help test my LLM applications?",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Can LangSmith help test my LLM applications?",      name: undefined,      additional_kwargs: {},      response_metadata: {}    }  ],  context: [    Document {      pageContent: "These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L"... 294 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "this guide, we’ll highlight the breadth of workflows LangSmith supports and how they fit into each s"... 343 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "The ability to rapidly understand how the model is performing — and debug where it is failing — is i"... 138 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    }  ],  answer: "Yes, LangSmith can help test your LLM applications. It allows developers to create datasets, which a"... 297 more characters}

    await conversationalRetrievalChain.invoke({  messages: [    new HumanMessage("Can LangSmith help test my LLM applications?"),    new AIMessage(      "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs. Additionally, LangSmith can be used to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise."    ),    new HumanMessage("Tell me more!"),  ],});

    {  messages: [    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Can LangSmith help test my LLM applications?",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Can LangSmith help test my LLM applications?",      name: undefined,      additional_kwargs: {},      response_metadata: {}    },    AIMessage {      lc_serializable: true,      lc_kwargs: {        content: "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examp"... 317 more characters,        tool_calls: [],        invalid_tool_calls: [],        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examp"... 317 more characters,      name: undefined,      additional_kwargs: {},      response_metadata: {},      tool_calls: [],      invalid_tool_calls: []    },    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Tell me more!",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Tell me more!",      name: undefined,      additional_kwargs: {},      response_metadata: {}    }  ],  context: [    Document {      pageContent: "These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L"... 294 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "this guide, we’ll highlight the breadth of workflows LangSmith supports and how they fit into each s"... 343 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "will help in curation of test cases that can help track regressions/improvements and development of "... 393 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    }  ],  answer: "LangSmith supports a variety of workflows to aid in the development of your applications, from creat"... 607 more characters}

You can check out [this LangSmith trace](https://smith.langchain.com/public/dc4d6bd4-fea5-45df-be94-06ad18882ae9/r) to see the internal query transformation step for yourself.

Streaming[​](#streaming "Direct link to Streaming")
---------------------------------------------------

Because this chain is constructed with LCEL, you can use familiar methods like `.stream()` with it:

    const stream = await conversationalRetrievalChain.stream({  messages: [    new HumanMessage("Can LangSmith help test my LLM applications?"),    new AIMessage(      "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs. Additionally, LangSmith can be used to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise."    ),    new HumanMessage("Tell me more!"),  ],});for await (const chunk of stream) {  console.log(chunk);}

    {  messages: [    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Can LangSmith help test my LLM applications?",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Can LangSmith help test my LLM applications?",      name: undefined,      additional_kwargs: {},      response_metadata: {}    },    AIMessage {      lc_serializable: true,      lc_kwargs: {        content: "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examp"... 317 more characters,        tool_calls: [],        invalid_tool_calls: [],        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examp"... 317 more characters,      name: undefined,      additional_kwargs: {},      response_metadata: {},      tool_calls: [],      invalid_tool_calls: []    },    HumanMessage {      lc_serializable: true,      lc_kwargs: {        content: "Tell me more!",        additional_kwargs: {},        response_metadata: {}      },      lc_namespace: [ "langchain_core", "messages" ],      content: "Tell me more!",      name: undefined,      additional_kwargs: {},      response_metadata: {}    }  ]}{  context: [    Document {      pageContent: "These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L"... 294 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "We provide native rendering of chat messages, functions, and retrieve documents.Initial Test Set​Whi"... 347 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "this guide, we’ll highlight the breadth of workflows LangSmith supports and how they fit into each s"... 343 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    },    Document {      pageContent: "will help in curation of test cases that can help track regressions/improvements and development of "... 393 more characters,      metadata: {        source: "https://docs.smith.langchain.com/user_guide",        loc: { lines: [Object] }      }    }  ]}{ answer: "" }{ answer: "Lang" }{ answer: "Smith" }{ answer: " offers" }{ answer: " a" }{ answer: " comprehensive" }{ answer: " suite" }{ answer: " of" }{ answer: " tools" }{ answer: " and" }{ answer: " workflows" }{ answer: " to" }{ answer: " support" }{ answer: " the" }{ answer: " development" }{ answer: " and" }{ answer: " testing" }{ answer: " of" }{ answer: " L" }{ answer: "LM" }{ answer: " applications" }{ answer: "." }{ answer: " Here" }{ answer: " are" }{ answer: " some" }{ answer: " key" }{ answer: " features" }{ answer: " and" }{ answer: " functionalities" }{ answer: ":\n\n" }{ answer: "1" }{ answer: "." }{ answer: " **" }{ answer: "Test" }{ answer: " Case" }{ answer: " Management" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Bulk" }{ answer: " Upload" }{ answer: " and" }{ answer: " Creation" }{ answer: "**" }{ answer: ":" }{ answer: " You" }{ answer: " can" }{ answer: " upload" }{ answer: " test" }{ answer: " cases" }{ answer: " in" }{ answer: " bulk" }{ answer: "," }{ answer: " create" }{ answer: " them" }{ answer: " on" }{ answer: " the" }{ answer: " fly" }{ answer: "," }{ answer: " or" }{ answer: " export" }{ answer: " them" }{ answer: " from" }{ answer: " application" }{ answer: " traces" }{ answer: ".\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Datas" }{ answer: "ets" }{ answer: "**" }{ answer: ":" }{ answer: " Lang" }{ answer: "Smith" }{ answer: " allows" }{ answer: " you" }{ answer: " to" }{ answer: " create" }{ answer: " datasets" }{ answer: "," }{ answer: " which" }{ answer: " are" }{ answer: " collections" }{ answer: " of" }{ answer: " inputs" }{ answer: " and" }{ answer: " reference" }{ answer: " outputs" }{ answer: "." }{ answer: " These" }{ answer: " datasets" }{ answer: " can" }{ answer: " be" }{ answer: " used" }{ answer: " to" }{ answer: " run" }{ answer: " tests" }{ answer: " on" }{ answer: " your" }{ answer: " L" }{ answer: "LM" }{ answer: " applications" }{ answer: ".\n\n" }{ answer: "2" }{ answer: "." }{ answer: " **" }{ answer: "Custom" }{ answer: " Evalu" }{ answer: "ations" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "LL" }{ answer: "M" }{ answer: " and" }{ answer: " He" }{ answer: "uristic" }{ answer: " Based" }{ answer: "**" }{ answer: ":" }{ answer: " You" }{ answer: " can" }{ answer: " run" }{ answer: " custom" }{ answer: " evaluations" }{ answer: " using" }{ answer: " both" }{ answer: " L" }{ answer: "LM" }{ answer: "-based" }{ answer: " and" }{ answer: " heuristic" }{ answer: "-based" }{ answer: " methods" }{ answer: " to" }{ answer: " score" }{ answer: " test" }{ answer: " results" }{ answer: ".\n\n" }{ answer: "3" }{ answer: "." }{ answer: " **" }{ answer: "Comparison" }{ answer: " View" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Pro" }{ answer: "tot" }{ answer: "yp" }{ answer: "ing" }{ answer: " and" }{ answer: " Regression" }{ answer: " Tracking" }{ answer: "**" }{ answer: ":" }{ answer: " When" }{ answer: " prot" }{ answer: "otyping" }{ answer: " different" }{ answer: " versions" }{ answer: " of" }{ answer: " your" }{ answer: " applications" }{ answer: "," }{ answer: " Lang" }{ answer: "Smith" }{ answer: " provides" }{ answer: " a" }{ answer: " comparison" }{ answer: " view" }{ answer: " to" }{ answer: " see" }{ answer: " if" }{ answer: " there" }{ answer: " have" }{ answer: " been" }{ answer: " any" }{ answer: " regress" }{ answer: "ions" }{ answer: " with" }{ answer: " respect" }{ answer: " to" }{ answer: " your" }{ answer: " initial" }{ answer: " test" }{ answer: " cases" }{ answer: ".\n\n" }{ answer: "4" }{ answer: "." }{ answer: " **" }{ answer: "Native" }{ answer: " Rendering" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Chat" }{ answer: " Messages" }{ answer: "," }{ answer: " Functions" }{ answer: "," }{ answer: " and" }{ answer: " Documents" }{ answer: "**" }{ answer: ":" }{ answer: " Lang" }{ answer: "Smith" }{ answer: " provides" }{ answer: " native" }{ answer: " rendering" }{ answer: " of" }{ answer: " chat" }{ answer: " messages" }{ answer: "," }{ answer: " functions" }{ answer: "," }{ answer: " and" }{ answer: " retrieved" }{ answer: " documents" }{ answer: "," }{ answer: " making" }{ answer: " it" }{ answer: " easier" }{ answer: " to" }{ answer: " visualize" }{ answer: " and" }{ answer: " understand" }{ answer: " the" }{ answer: " outputs" }{ answer: ".\n\n" }{ answer: "5" }{ answer: "." }{ answer: " **" }{ answer: "Pro" }{ answer: "tot" }{ answer: "yp" }{ answer: "ing" }{ answer: " Support" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Quick" }{ answer: " Experiment" }{ answer: "ation" }{ answer: "**" }{ answer: ":" }{ answer: " The" }{ answer: " platform" }{ answer: " supports" }{ answer: " quick" }{ answer: " experimentation" }{ answer: " with" }{ answer: " different" }{ answer: " prompts" }{ answer: "," }{ answer: " model" }{ answer: " types" }{ answer: "," }{ answer: " retrieval" }{ answer: " strategies" }{ answer: "," }{ answer: " and" }{ answer: " other" }{ answer: " parameters" }{ answer: ".\n\n" }{ answer: "6" }{ answer: "." }{ answer: " **" }{ answer: "Feedback" }{ answer: " Capture" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Human" }{ answer: " Feedback" }{ answer: "**" }{ answer: ":" }{ answer: " When" }{ answer: " launching" }{ answer: " your" }{ answer: " application" }{ answer: " to" }{ answer: " an" }{ answer: " initial" }{ answer: " set" }{ answer: " of" }{ answer: " users" }{ answer: "," }{ answer: " Lang" }{ answer: "Smith" }{ answer: " allows" }{ answer: " you" }{ answer: " to" }{ answer: " gather" }{ answer: " human" }{ answer: " feedback" }{ answer: " on" }{ answer: " the" }{ answer: " responses" }{ answer: "." }{ answer: " This" }{ answer: " helps" }{ answer: " identify" }{ answer: " interesting" }{ answer: " runs" }{ answer: " and" }{ answer: " highlight" }{ answer: " edge" }{ answer: " cases" }{ answer: " causing" }{ answer: " problematic" }{ answer: " responses" }{ answer: ".\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Feedback" }{ answer: " Scores" }{ answer: "**" }{ answer: ":" }{ answer: " You" }{ answer: " can" }{ answer: " attach" }{ answer: " feedback" }{ answer: " scores" }{ answer: " to" }{ answer: " logged" }{ answer: " traces" }{ answer: "," }{ answer: " often" }{ answer: " integrated" }{ answer: " into" }{ answer: " the" }{ answer: " system" }{ answer: ".\n\n" }{ answer: "7" }{ answer: "." }{ answer: " **" }{ answer: "Monitoring" }{ answer: " and" }{ answer: " Troubles" }{ answer: "hooting" }{ answer: "**" }{ answer: ":\n" }{ answer: "  " }{ answer: " -" }{ answer: " **" }{ answer: "Logging" }{ answer: " and" }{ answer: " Visualization" }{ answer: "**" }{ answer: ":" }{ answer: " Lang" }{ answer: "Smith" }{ answer: " logs" }{ answer: " all" }{ answer: " traces" }{ answer: "," }{ answer: " visual" }{ answer: "izes" }{ answer: " latency" }{ answer: " and" }{ answer: " token" }{ answer: " usage" }{ answer: " statistics" }{ answer: "," }{ answer: " and" }{ answer: " helps" }{ answer: " troubleshoot" }{ answer: " specific" }{ answer: " issues" }{ answer: " as" }{ answer: " they" }{ answer: " arise" }{ answer: ".\n\n" }{ answer: "Overall" }{ answer: "," }{ answer: " Lang" }{ answer: "Smith" }{ answer: " is" }{ answer: " designed" }{ answer: " to" }{ answer: " support" }{ answer: " the" }{ answer: " entire" }{ answer: " lifecycle" }{ answer: " of" }{ answer: " L" }{ answer: "LM" }{ answer: " application" }{ answer: " development" }{ answer: "," }{ answer: " from" }{ answer: " initial" }{ answer: " prot" }{ answer: "otyping" }{ answer: " to" }{ answer: " deployment" }{ answer: " and" }{ answer: " ongoing" }{ answer: " monitoring" }{ answer: "," }{ answer: " making" }{ answer: " it" }{ answer: " a" }{ answer: " powerful" }{ answer: " tool" }{ answer: " for" }{ answer: " developers" }{ answer: " looking" }{ answer: " to" }{ answer: " build" }{ answer: " and" }{ answer: " maintain" }{ answer: " high" }{ answer: "-quality" }{ answer: " L" }{ answer: "LM" }{ answer: " applications" }{ answer: "." }{ answer: "" }

Next steps[​](#next-steps "Direct link to Next steps")
------------------------------------------------------

You’ve now learned some techniques for adding personal data as context to your chatbots.

This guide only scratches the surface of retrieval techniques. For more on different ways of ingesting, preparing, and retrieving the most relevant data, check out our [how to guides on retrieval](/v0.2/docs/how_to/#retrievers).

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to manage memory

](/v0.2/docs/how_to/chatbots_memory)[

Next

How to use tools

](/v0.2/docs/how_to/chatbots_tools)

*   [Setup](#setup)
*   [Creating a retriever](#creating-a-retriever)
*   [Document chains](#document-chains)
*   [Retrieval chains](#retrieval-chains)
*   [Query transformation](#query-transformation)
*   [Streaming](#streaming)
*   [Next steps](#next-steps)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.