!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [Tutorials](/v0.2/docs/tutorials/)
*   Build a Local RAG Application

On this page

Build a Local RAG Application
=============================

The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), [GPT4All](https://github.com/nomic-ai/gpt4all), and [llamafile](https://github.com/Mozilla-Ocho/llamafile) underscore the importance of running LLMs locally.

LangChain has integrations with many open-source LLMs that can be run locally.

For example, here we show how to run `OllamaEmbeddings` or `LLaMA2` locally (e.g., on your laptop) using local embeddings and a local LLM.

Document Loading[​](#document-loading "Direct link to Document Loading")
------------------------------------------------------------------------

First, install packages needed for local embeddings and vector storage.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

### Dependencies[​](#dependencies "Direct link to Dependencies")

We’ll use the following packages:

    npm install --save langchain @langchain/community cheerio

### LangSmith[​](#langsmith "Direct link to LangSmith")

Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com/).

Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:

    export LANGCHAIN_TRACING_V2=trueexport LANGCHAIN_API_KEY=YOUR_KEY

### Initial setup[​](#initial-setup "Direct link to Initial setup")

Load and split an example document.

We’ll use a blog post on agents as an example.

    import "cheerio";import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";

    const loader = new CheerioWebBaseLoader(  "https://lilianweng.github.io/posts/2023-06-23-agent/");const docs = await loader.load();const textSplitter = new RecursiveCharacterTextSplitter({  chunkSize: 500,  chunkOverlap: 0,});const allSplits = await textSplitter.splitDocuments(docs);console.log(allSplits.length);

    146

Next, we’ll use `OllamaEmbeddings` for our local embeddings. Follow [these instructions](https://github.com/ollama/ollama) to set up and run a local Ollama instance.

    import { OllamaEmbeddings } from "@langchain/community/embeddings/ollama";import { MemoryVectorStore } from "langchain/vectorstores/memory";const embeddings = new OllamaEmbeddings();const vectorStore = await MemoryVectorStore.fromDocuments(  allSplits,  embeddings);

Test similarity search is working with our local embeddings.

    const question = "What are the approaches to Task Decomposition?";const docs = await vectorStore.similaritySearch(question);console.log(docs.length);

    4

Model[​](#model "Direct link to Model")
---------------------------------------

### LLaMA2[​](#llama2 "Direct link to LLaMA2")

For local LLMs we’ll use also use `ollama`.

    import { ChatOllama } from "@langchain/community/chat_models/ollama";const ollamaLlm = new ChatOllama({  baseUrl: "http://localhost:11434", // Default value  model: "llama2", // Default value});

    const response = await ollamaLlm.invoke(  "Simulate a rap battle between Stephen Colbert and John Oliver");console.log(response.content);

    [The stage is set for a fierce rap battle between two of the funniest men on television. Stephen Colbert and John Oliver are standing face to face, each with their own microphone and confident smirk on their face.]Stephen Colbert:Yo, John Oliver, I heard you've been talking smackAbout my show and my satire, saying it's all fakeBut let me tell you something, brother, I'm the real dealI've been making fun of politicians for years, with no concealJohn Oliver:Oh, Stephen, you think you're so clever and smartBut your jokes are stale and your delivery's a work of artYou're just a pale imitation of the real deal, Jon StewartI'm the one who's really making waves, while you're just a little birdStephen Colbert:Well, John, I may not be as loud as you, but I'm smarterMy satire is more subtle, and it goes right over their headsI'm the one who's been exposing the truth for yearsWhile you're just a British interloper, trying to steal the cheersJohn Oliver:Oh, Stephen, you may have your fans, but I've got the brainsMy show is more than just slapstick and silly jokes, it's got depth and gainsI'm the one who's really making a difference, while you're just a clownMy satire is more than just a joke, it's a call to action, and I've got the crown[The crowd cheers and chants as the two comedians continue their rap battle.]Stephen Colbert:You may have your fans, John, but I'm the king of satireI've been making fun of politicians for years, and I'm still standing tallMy jokes are clever and smart, while yours are just plain dumbI'm the one who's really in control, and you're just a pretender to the throne.John Oliver:Oh, Stephen, you may have your moment in the sunBut I'm the one who's really shining bright, and my star is just beginning to riseMy satire is more than just a joke, it's a call to action, and I've got the powerI'm the one who's really making a difference, and you're just a fleeting flower.[The crowd continues to cheer and chant as the two comedians continue their rap battle.]

See the LangSmith trace [here](https://smith.langchain.com/public/31c178b5-4bea-4105-88c3-7ec95325c817/r)

Using in a chain[​](#using-in-a-chain "Direct link to Using in a chain")
------------------------------------------------------------------------

We can create a summarization chain with either model by passing in the retrieved docs and a simple prompt.

It formats the prompt template using the input key values provided and passes the formatted string to `LLama-V2`, or another specified LLM.

    import { RunnableSequence } from "@langchain/core/runnables";import { StringOutputParser } from "@langchain/core/output_parsers";import { PromptTemplate } from "@langchain/core/prompts";import { createStuffDocumentsChain } from "langchain/chains/combine_documents";const prompt = PromptTemplate.fromTemplate(  "Summarize the main themes in these retrieved docs: {context}");const chain = await createStuffDocumentsChain({  llm: ollamaLlm,  outputParser: new StringOutputParser(),  prompt,});

    const question = "What are the approaches to Task Decomposition?";const docs = await vectorStore.similaritySearch(question);await chain.invoke({  context: docs,});

    "The main themes retrieved from the provided documents are:\n" +  "\n" +  "1. Sensory Memory: The ability to retain"... 1117 more characters

See the LangSmith trace [here](https://smith.langchain.com/public/47cf6c2a-3d86-4f2b-9a51-ee4663b19152/r)

Q&A[​](#qa "Direct link to Q&A")
--------------------------------

We can also use the LangChain Prompt Hub to store and fetch prompts that are model-specific.

Let’s try with a default RAG prompt, [here](https://smith.langchain.com/hub/rlm/rag-prompt).

    import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";const ragPrompt = await pull<ChatPromptTemplate>("rlm/rag-prompt");const chain = await createStuffDocumentsChain({  llm: ollamaLlm,  outputParser: new StringOutputParser(),  prompt: ragPrompt,});

Let’s see what this prompt actually looks like:

    console.log(  ragPrompt.promptMessages.map((msg) => msg.prompt.template).join("\n"));

    You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.Question: {question}Context: {context}Answer:

    await chain.invoke({ context: docs, question });

    "Task decomposition is a crucial step in breaking down complex problems into manageable parts for eff"... 1095 more characters

See the LangSmith trace [here](https://smith.langchain.com/public/dd3a189b-53a1-4f31-9766-244cd04ad1f7/r)

Q&A with retrieval[​](#qa-with-retrieval "Direct link to Q&A with retrieval")
-----------------------------------------------------------------------------

Instead of manually passing in docs, we can automatically retrieve them from our vector store based on the user question.

This will use a QA default prompt and will retrieve from the vectorDB.

    import {  RunnablePassthrough,  RunnableSequence,} from "@langchain/core/runnables";import { formatDocumentsAsString } from "langchain/util/document";const retriever = vectorStore.asRetriever();const qaChain = RunnableSequence.from([  {    context: (input: { question: string }, callbacks) => {      const retrieverAndFormatter = retriever.pipe(formatDocumentsAsString);      return retrieverAndFormatter.invoke(input.question, callbacks);    },    question: new RunnablePassthrough(),  },  ragPrompt,  ollamaLlm,  new StringOutputParser(),]);await qaChain.invoke({ question });

    "Based on the context provided, I understand that you are asking me to answer a question related to m"... 948 more characters

See the LangSmith trace [here](https://smith.langchain.com/public/440e65ee-0301-42cf-afc9-f09cfb52cf64/r)

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

Tagging

](/v0.2/docs/tutorials/classification)[

Next

Conversational RAG

](/v0.2/docs/tutorials/qa_chat_history)

*   [Document Loading](#document-loading)
*   [Setup](#setup)
    *   [Dependencies](#dependencies)
    *   [LangSmith](#langsmith)
    *   [Initial setup](#initial-setup)
*   [Model](#model)
    *   [LLaMA2](#llama2)
*   [Using in a chain](#using-in-a-chain)
*   [Q&A](#qa)
*   [Q&A with retrieval](#qa-with-retrieval)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.