!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to cache model responses

On this page

How to cache model responses
============================

LangChain provides an optional caching layer for LLMs. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

tip

See [this section for general instructions on installing integration packages](/v0.2/docs/how_to/installation#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/openai

    yarn add @langchain/openai

    pnpm add @langchain/openai

    import { OpenAI } from "@langchain/openai";const model = new OpenAI({  model: "gpt-3.5-turbo-instruct",  cache: true,});

In Memory Cache[​](#in-memory-cache "Direct link to In Memory Cache")
---------------------------------------------------------------------

The default cache is stored in-memory. This means that if you restart your application, the cache will be cleared.

    console.time();// The first time, it is not yet in cache, so it should take longerconst res = await model.invoke("Tell me a long joke");console.log(res);console.timeEnd();/*  A man walks into a bar and sees a jar filled with money on the counter. Curious, he asks the bartender about it.  The bartender explains, "We have a challenge for our customers. If you can complete three tasks, you win all the money in the jar."  Intrigued, the man asks what the tasks are.  The bartender replies, "First, you have to drink a whole bottle of tequila without making a face. Second, there's a pitbull out back with a sore tooth. You have to pull it out. And third, there's an old lady upstairs who has never had an orgasm. You have to give her one."  The man thinks for a moment and then confidently says, "I'll do it."  He grabs the bottle of tequila and downs it in one gulp, without flinching. He then heads to the back and after a few minutes of struggling, emerges with the pitbull's tooth in hand.  The bar erupts in cheers and the bartender leads the man upstairs to the old lady's room. After a few minutes, the man walks out with a big smile on his face and the old lady is giggling with delight.  The bartender hands the man the jar of money and asks, "How  default: 4.187s*/

    console.time();// The second time it is, so it goes fasterconst res2 = await model.invoke("Tell me a joke");console.log(res2);console.timeEnd();/*  A man walks into a bar and sees a jar filled with money on the counter. Curious, he asks the bartender about it.  The bartender explains, "We have a challenge for our customers. If you can complete three tasks, you win all the money in the jar."  Intrigued, the man asks what the tasks are.  The bartender replies, "First, you have to drink a whole bottle of tequila without making a face. Second, there's a pitbull out back with a sore tooth. You have to pull it out. And third, there's an old lady upstairs who has never had an orgasm. You have to give her one."  The man thinks for a moment and then confidently says, "I'll do it."  He grabs the bottle of tequila and downs it in one gulp, without flinching. He then heads to the back and after a few minutes of struggling, emerges with the pitbull's tooth in hand.  The bar erupts in cheers and the bartender leads the man upstairs to the old lady's room. After a few minutes, the man walks out with a big smile on his face and the old lady is giggling with delight.  The bartender hands the man the jar of money and asks, "How  default: 175.74ms*/

Caching with Momento[​](#caching-with-momento "Direct link to Caching with Momento")
------------------------------------------------------------------------------------

LangChain also provides a Momento-based cache. [Momento](https://gomomento.com) is a distributed, serverless cache that requires zero setup or infrastructure maintenance. Given Momento's compatibility with Node.js, browser, and edge environments, ensure you install the relevant package.

To install for **Node.js**:

*   npm
*   Yarn
*   pnpm

    npm install @gomomento/sdk

    yarn add @gomomento/sdk

    pnpm add @gomomento/sdk

To install for **browser/edge workers**:

*   npm
*   Yarn
*   pnpm

    npm install @gomomento/sdk-web

    yarn add @gomomento/sdk-web

    pnpm add @gomomento/sdk-web

Next you'll need to sign up and create an API key. Once you've done that, pass a `cache` option when you instantiate the LLM like this:

    import { OpenAI } from "@langchain/openai";import {  CacheClient,  Configurations,  CredentialProvider,} from "@gomomento/sdk";import { MomentoCache } from "@langchain/community/caches/momento";// See https://github.com/momentohq/client-sdk-javascript for connection optionsconst client = new CacheClient({  configuration: Configurations.Laptop.v1(),  credentialProvider: CredentialProvider.fromEnvironmentVariable({    environmentVariableName: "MOMENTO_API_KEY",  }),  defaultTtlSeconds: 60 * 60 * 24,});const cache = await MomentoCache.fromProps({  client,  cacheName: "langchain",});const model = new OpenAI({ cache });

#### API Reference:

*   [OpenAI](https://v02.api.js.langchain.com/classes/langchain_openai.OpenAI.html) from `@langchain/openai`
*   [MomentoCache](https://v02.api.js.langchain.com/classes/langchain_community_caches_momento.MomentoCache.html) from `@langchain/community/caches/momento`

Caching with Redis[​](#caching-with-redis "Direct link to Caching with Redis")
------------------------------------------------------------------------------

LangChain also provides a Redis-based cache. This is useful if you want to share the cache across multiple processes or servers. To use it, you'll need to install the `redis` package:

*   npm
*   Yarn
*   pnpm

    npm install ioredis

    yarn add ioredis

    pnpm add ioredis

Then, you can pass a `cache` option when you instantiate the LLM. For example:

    import { OpenAI } from "@langchain/openai";import { RedisCache } from "@langchain/community/caches/ioredis";import { Redis } from "ioredis";// See https://github.com/redis/ioredis for connection optionsconst client = new Redis({});const cache = new RedisCache(client);const model = new OpenAI({ cache });

Caching with Upstash Redis[​](#caching-with-upstash-redis "Direct link to Caching with Upstash Redis")
------------------------------------------------------------------------------------------------------

LangChain provides an Upstash Redis-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Upstash Redis client uses HTTP and supports edge environments. To use it, you'll need to install the `@upstash/redis` package:

*   npm
*   Yarn
*   pnpm

    npm install @upstash/redis

    yarn add @upstash/redis

    pnpm add @upstash/redis

You'll also need an [Upstash account](https://docs.upstash.com/redis#create-account) and a [Redis database](https://docs.upstash.com/redis#create-a-database) to connect to. Once you've done that, retrieve your REST URL and REST token.

Then, you can pass a `cache` option when you instantiate the LLM. For example:

    import { OpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// See https://docs.upstash.com/redis/howto/connectwithupstashredis#quick-start for connection optionsconst cache = new UpstashRedisCache({  config: {    url: "UPSTASH_REDIS_REST_URL",    token: "UPSTASH_REDIS_REST_TOKEN",  },});const model = new OpenAI({ cache });

#### API Reference:

*   [OpenAI](https://v02.api.js.langchain.com/classes/langchain_openai.OpenAI.html) from `@langchain/openai`
*   [UpstashRedisCache](https://v02.api.js.langchain.com/classes/langchain_community_caches_upstash_redis.UpstashRedisCache.html) from `@langchain/community/caches/upstash_redis`

You can also directly pass in a previously created [@upstash/redis](https://docs.upstash.com/redis/sdks/javascriptsdk/overview) client instance:

    import { Redis } from "@upstash/redis";import https from "https";import { OpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// const client = new Redis({//   url: process.env.UPSTASH_REDIS_REST_URL!,//   token: process.env.UPSTASH_REDIS_REST_TOKEN!,//   agent: new https.Agent({ keepAlive: true }),// });// Or simply call Redis.fromEnv() to automatically load the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables.const client = Redis.fromEnv({  agent: new https.Agent({ keepAlive: true }),});const cache = new UpstashRedisCache({ client });const model = new OpenAI({ cache });

#### API Reference:

*   [OpenAI](https://v02.api.js.langchain.com/classes/langchain_openai.OpenAI.html) from `@langchain/openai`
*   [UpstashRedisCache](https://v02.api.js.langchain.com/classes/langchain_community_caches_upstash_redis.UpstashRedisCache.html) from `@langchain/community/caches/upstash_redis`

Caching with Cloudflare KV[​](#caching-with-cloudflare-kv "Direct link to Caching with Cloudflare KV")
------------------------------------------------------------------------------------------------------

info

This integration is only supported in Cloudflare Workers.

If you're deploying your project as a Cloudflare Worker, you can use LangChain's Cloudflare KV-powered LLM cache.

For information on how to set up KV in Cloudflare, see [the official documentation](https://developers.cloudflare.com/kv/).

**Note:** If you are using TypeScript, you may need to install types if they aren't already present:

*   npm
*   Yarn
*   pnpm

    npm install -S @cloudflare/workers-types

    yarn add @cloudflare/workers-types

    pnpm add @cloudflare/workers-types

    import type { KVNamespace } from "@cloudflare/workers-types";import { OpenAI } from "@langchain/openai";import { CloudflareKVCache } from "@langchain/cloudflare";export interface Env {  KV_NAMESPACE: KVNamespace;  OPENAI_API_KEY: string;}export default {  async fetch(_request: Request, env: Env) {    try {      const cache = new CloudflareKVCache(env.KV_NAMESPACE);      const model = new OpenAI({        cache,        model: "gpt-3.5-turbo-instruct",        apiKey: env.OPENAI_API_KEY,      });      const response = await model.invoke("How are you today?");      return new Response(JSON.stringify(response), {        headers: { "content-type": "application/json" },      });    } catch (err: any) {      console.log(err.message);      return new Response(err.message, { status: 500 });    }  },};

#### API Reference:

*   [OpenAI](https://v02.api.js.langchain.com/classes/langchain_openai.OpenAI.html) from `@langchain/openai`
*   [CloudflareKVCache](https://v02.api.js.langchain.com/classes/langchain_cloudflare.CloudflareKVCache.html) from `@langchain/cloudflare`

Caching on the File System[​](#caching-on-the-file-system "Direct link to Caching on the File System")
------------------------------------------------------------------------------------------------------

danger

This cache is not recommended for production use. It is only intended for local development.

LangChain provides a simple file system cache. By default the cache is stored a temporary directory, but you can specify a custom directory if you want.

    const cache = await LocalFileCache.create();

Next steps[​](#next-steps "Direct link to Next steps")
------------------------------------------------------

You've now learned how to cache model responses to save time and money.

Next, check out the other how-to guides on LLMs, like [how to create your own custom LLM class](/v0.2/docs/how_to/custom_llm).

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to use few shot examples in chat models

](/v0.2/docs/how_to/few_shot_examples_chat)[

Next

How to cache chat model responses

](/v0.2/docs/how_to/chat_model_caching)

*   [In Memory Cache](#in-memory-cache)
*   [Caching with Momento](#caching-with-momento)
*   [Caching with Redis](#caching-with-redis)
*   [Caching with Upstash Redis](#caching-with-upstash-redis)
*   [Caching with Cloudflare KV](#caching-with-cloudflare-kv)
*   [Caching on the File System](#caching-on-the-file-system)
*   [Next steps](#next-steps)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.