!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Get started](/v0.1/docs/get_started/)
    
    *   [Introduction](/v0.1/docs/get_started/introduction/)
    *   [Installation](/v0.1/docs/get_started/installation/)
    *   [Quickstart](/v0.1/docs/get_started/quickstart/)
*   [LangChain Expression Language](/v0.1/docs/expression_language/)
    
    *   [Get started](/v0.1/docs/expression_language/get_started/)
    *   [Why use LCEL?](/v0.1/docs/expression_language/why/)
    *   [Interface](/v0.1/docs/expression_language/interface/)
    *   [Streaming](/v0.1/docs/expression_language/streaming/)
    *   [How to](/v0.1/docs/expression_language/how_to/routing/)
        
    *   [Cookbook](/v0.1/docs/expression_language/cookbook/)
        
    *   [LangChain Expression Language (LCEL)](/v0.1/docs/expression_language/)
*   [Modules](/v0.1/docs/modules/)
    
    *   [Model I/O](/v0.1/docs/modules/model_io/)
        
        *   [Quickstart](/v0.1/docs/modules/model_io/quick_start/)
        *   [Concepts](/v0.1/docs/modules/model_io/concepts/)
        *   [Prompts](/v0.1/docs/modules/model_io/prompts/)
            
        *   [LLMs](/v0.1/docs/modules/model_io/llms/)
            
        *   [Chat Models](/v0.1/docs/modules/model_io/chat/)
            
            *   [Quick Start](/v0.1/docs/modules/model_io/chat/quick_start/)
            *   [Streaming](/v0.1/docs/modules/model_io/chat/streaming/)
            *   [Caching](/v0.1/docs/modules/model_io/chat/caching/)
            *   [Custom chat models](/v0.1/docs/modules/model_io/chat/custom_chat/)
            *   [Tracking token usage](/v0.1/docs/modules/model_io/chat/token_usage_tracking/)
            *   [Cancelling requests](/v0.1/docs/modules/model_io/chat/cancelling_requests/)
            *   [Dealing with API Errors](/v0.1/docs/modules/model_io/chat/dealing_with_api_errors/)
            *   [Dealing with rate limits](/v0.1/docs/modules/model_io/chat/dealing_with_rate_limits/)
            *   [Tool/function calling](/v0.1/docs/modules/model_io/chat/function_calling/)
            *   [Response metadata](/v0.1/docs/modules/model_io/chat/response_metadata/)
            *   [Subscribing to events](/v0.1/docs/modules/model_io/chat/subscribing_events/)
            *   [Adding a timeout](/v0.1/docs/modules/model_io/chat/timeouts/)
        *   [Model I/O](/v0.1/docs/modules/model_io/)
        *   [Output Parsers](/v0.1/docs/modules/model_io/output_parsers/)
            
    *   [Retrieval](/v0.1/docs/modules/data_connection/)
        
    *   [Chains](/v0.1/docs/modules/chains/)
    *   [Agents](/v0.1/docs/modules/agents/)
        
    *   [More](/v0.1/docs/modules/memory/)
        
*   [Security](/v0.1/docs/security/)
*   [Guides](/v0.1/docs/guides/)
    
*   [Ecosystem](/v0.1/docs/ecosystem/)
    
*   [LangGraph](/v0.1/docs/langgraph/)
*   * * *
    

*   [](/v0.1/)
*   [Modules](/v0.1/docs/modules/)
*   [Model I/O](/v0.1/docs/modules/model_io/)
*   [Chat Models](/v0.1/docs/modules/model_io/chat/)
*   Caching

On this page

Caching
=======

LangChain provides an optional caching layer for chat models. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

tip

We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.

    import { ChatOpenAI } from "@langchain/openai";// To make the caching really obvious, lets use a slower model.const model = new ChatOpenAI({  model: "gpt-4",  cache: true,});

In Memory Cache[​](#in-memory-cache "Direct link to In Memory Cache")
---------------------------------------------------------------------

The default cache is stored in-memory. This means that if you restart your application, the cache will be cleared.

    console.time();// The first time, it is not yet in cache, so it should take longerconst res = await model.invoke("Tell me a joke!");console.log(res);console.timeEnd();/*  AIMessage {    lc_serializable: true,    lc_kwargs: {      content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",      additional_kwargs: { function_call: undefined, tool_calls: undefined }    },    lc_namespace: [ 'langchain_core', 'messages' ],    content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",    name: undefined,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  }  default: 2.224s*/

    console.time();// The second time it is, so it goes fasterconst res2 = await model.invoke("Tell me a joke!");console.log(res2);console.timeEnd();/*  AIMessage {    lc_serializable: true,    lc_kwargs: {      content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",      additional_kwargs: { function_call: undefined, tool_calls: undefined }    },    lc_namespace: [ 'langchain_core', 'messages' ],    content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",    name: undefined,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  }  default: 181.98ms*/

Caching with Momento[​](#caching-with-momento "Direct link to Caching with Momento")
------------------------------------------------------------------------------------

LangChain also provides a Momento-based cache. [Momento](https://gomomento.com) is a distributed, serverless cache that requires zero setup or infrastructure maintenance. To use it, you'll need to install the `@gomomento/sdk` package:

*   npm
*   Yarn
*   pnpm

    npm install @gomomento/sdk

    yarn add @gomomento/sdk

    pnpm add @gomomento/sdk

Next you'll need to sign up and create an API key. Once you've done that, pass a `cache` option when you instantiate the LLM like this:

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/openai

    yarn add @langchain/openai

    pnpm add @langchain/openai

    import { ChatOpenAI } from "@langchain/openai";import {  CacheClient,  Configurations,  CredentialProvider,} from "@gomomento/sdk";import { MomentoCache } from "@langchain/community/caches/momento";// See https://github.com/momentohq/client-sdk-javascript for connection optionsconst client = new CacheClient({  configuration: Configurations.Laptop.v1(),  credentialProvider: CredentialProvider.fromEnvironmentVariable({    environmentVariableName: "MOMENTO_API_KEY",  }),  defaultTtlSeconds: 60 * 60 * 24,});const cache = await MomentoCache.fromProps({  client,  cacheName: "langchain",});const model = new ChatOpenAI({ cache });

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [MomentoCache](https://api.js.langchain.com/classes/langchain_community_caches_momento.MomentoCache.html) from `@langchain/community/caches/momento`

Caching with Redis[​](#caching-with-redis "Direct link to Caching with Redis")
------------------------------------------------------------------------------

LangChain also provides a Redis-based cache. This is useful if you want to share the cache across multiple processes or servers. To use it, you'll need to install the `redis` package:

*   npm
*   Yarn
*   pnpm

    npm install ioredis

    yarn add ioredis

    pnpm add ioredis

Then, you can pass a `cache` option when you instantiate the LLM. For example:

    import { ChatOpenAI } from "@langchain/openai";import { Redis } from "ioredis";import { RedisCache } from "@langchain/community/caches/ioredis";const client = new Redis("redis://localhost:6379");const cache = new RedisCache(client, {  ttl: 60, // Optional key expiration value});const model = new ChatOpenAI({ cache });const response1 = await model.invoke("Do something random!");console.log(response1);/*  AIMessage {    content: "Sure! I'll generate a random number for you: 37",    additional_kwargs: {}  }*/const response2 = await model.invoke("Do something random!");console.log(response2);/*  AIMessage {    content: "Sure! I'll generate a random number for you: 37",    additional_kwargs: {}  }*/await client.disconnect();

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [RedisCache](https://api.js.langchain.com/classes/langchain_community_caches_ioredis.RedisCache.html) from `@langchain/community/caches/ioredis`

Caching with Upstash Redis[​](#caching-with-upstash-redis "Direct link to Caching with Upstash Redis")
------------------------------------------------------------------------------------------------------

LangChain provides an Upstash Redis-based cache. Like the Redis-based cache, this cache is useful if you want to share the cache across multiple processes or servers. The Upstash Redis client uses HTTP and supports edge environments. To use it, you'll need to install the `@upstash/redis` package:

*   npm
*   Yarn
*   pnpm

    npm install @upstash/redis

    yarn add @upstash/redis

    pnpm add @upstash/redis

You'll also need an [Upstash account](https://docs.upstash.com/redis#create-account) and a [Redis database](https://docs.upstash.com/redis#create-a-database) to connect to. Once you've done that, retrieve your REST URL and REST token.

Then, you can pass a `cache` option when you instantiate the LLM. For example:

    import { ChatOpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// See https://docs.upstash.com/redis/howto/connectwithupstashredis#quick-start for connection optionsconst cache = new UpstashRedisCache({  config: {    url: "UPSTASH_REDIS_REST_URL",    token: "UPSTASH_REDIS_REST_TOKEN",  },});const model = new ChatOpenAI({ cache });

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [UpstashRedisCache](https://api.js.langchain.com/classes/langchain_community_caches_upstash_redis.UpstashRedisCache.html) from `@langchain/community/caches/upstash_redis`

You can also directly pass in a previously created [@upstash/redis](https://docs.upstash.com/redis/sdks/javascriptsdk/overview) client instance:

    import { Redis } from "@upstash/redis";import https from "https";import { ChatOpenAI } from "@langchain/openai";import { UpstashRedisCache } from "@langchain/community/caches/upstash_redis";// const client = new Redis({//   url: process.env.UPSTASH_REDIS_REST_URL!,//   token: process.env.UPSTASH_REDIS_REST_TOKEN!,//   agent: new https.Agent({ keepAlive: true }),// });// Or simply call Redis.fromEnv() to automatically load the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables.const client = Redis.fromEnv({  agent: new https.Agent({ keepAlive: true }),});const cache = new UpstashRedisCache({ client });const model = new ChatOpenAI({ cache });

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [UpstashRedisCache](https://api.js.langchain.com/classes/langchain_community_caches_upstash_redis.UpstashRedisCache.html) from `@langchain/community/caches/upstash_redis`

Caching with Cloudflare KV[​](#caching-with-cloudflare-kv "Direct link to Caching with Cloudflare KV")
------------------------------------------------------------------------------------------------------

info

This integration is only supported in Cloudflare Workers.

If you're deploying your project as a Cloudflare Worker, you can use LangChain's Cloudflare KV-powered LLM cache.

For information on how to set up KV in Cloudflare, see [the official documentation](https://developers.cloudflare.com/kv/).

**Note:** If you are using TypeScript, you may need to install types if they aren't already present:

*   npm
*   Yarn
*   pnpm

    npm install -S @cloudflare/workers-types

    yarn add @cloudflare/workers-types

    pnpm add @cloudflare/workers-types

    import type { KVNamespace } from "@cloudflare/workers-types";import { ChatOpenAI } from "@langchain/openai";import { CloudflareKVCache } from "@langchain/cloudflare";export interface Env {  KV_NAMESPACE: KVNamespace;  OPENAI_API_KEY: string;}export default {  async fetch(_request: Request, env: Env) {    try {      const cache = new CloudflareKVCache(env.KV_NAMESPACE);      const model = new ChatOpenAI({        cache,        model: "gpt-3.5-turbo",        apiKey: env.OPENAI_API_KEY,      });      const response = await model.invoke("How are you today?");      return new Response(JSON.stringify(response), {        headers: { "content-type": "application/json" },      });    } catch (err: any) {      console.log(err.message);      return new Response(err.message, { status: 500 });    }  },};

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [CloudflareKVCache](https://api.js.langchain.com/classes/langchain_cloudflare.CloudflareKVCache.html) from `@langchain/cloudflare`

Caching on the File System[​](#caching-on-the-file-system "Direct link to Caching on the File System")
------------------------------------------------------------------------------------------------------

danger

This cache is not recommended for production use. It is only intended for local development.

LangChain provides a simple file system cache. By default the cache is stored a temporary directory, but you can specify a custom directory if you want.

    const cache = await LocalFileCache.create();

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Streaming

](/v0.1/docs/modules/model_io/chat/streaming/)[

Next

Custom chat models

](/v0.1/docs/modules/model_io/chat/custom_chat/)

*   [In Memory Cache](#in-memory-cache)
*   [Caching with Momento](#caching-with-momento)
*   [Caching with Redis](#caching-with-redis)
*   [Caching with Upstash Redis](#caching-with-upstash-redis)
*   [Caching with Cloudflare KV](#caching-with-cloudflare-kv)
*   [Caching on the File System](#caching-on-the-file-system)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.