*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to cache chat model responses

On this page

How to cache chat model responses
=================================

Prerequisites

This guide assumes familiarity with the following concepts:

*   [Chat models](/v0.2/docs/concepts/#chat-models)
*   [LLMs](/v0.2/docs/concepts/#llms)

LangChain provides an optional caching layer for chat models. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

    import { ChatOpenAI } from "@langchain/openai";// To make the caching really obvious, lets use a slower model.const model = new ChatOpenAI({  model: "gpt-4",  cache: true,});

In Memory Cache[​](#in-memory-cache "Direct link to In Memory Cache")
---------------------------------------------------------------------

The default cache is stored in-memory. This means that if you restart your application, the cache will be cleared.

    console.time();// The first time, it is not yet in cache, so it should take longerconst res = await model.invoke("Tell me a joke!");console.log(res);console.timeEnd();/*  AIMessage {    lc_serializable: true,    lc_kwargs: {      content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",      additional_kwargs: { function_call: undefined, tool_calls: undefined }    },    lc_namespace: [ 'langchain_core', 'messages' ],    content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",    name: undefined,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  }  default: 2.224s*/

    console.time();// The second time it is, so it goes fasterconst res2 = await model.invoke("Tell me a joke!");console.log(res2);console.timeEnd();/*  AIMessage {    lc_serializable: true,    lc_kwargs: {      content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",      additional_kwargs: { function_call: undefined, tool_calls: undefined }    },    lc_namespace: [ 'langchain_core', 'messages' ],    content: "Why don't scientists trust atoms?\n\nBecause they make up everything!",    name: undefined,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  }  default: 181.98ms*/

Caching with Redis[​](#caching-with-redis "Direct link to Caching with Redis")
------------------------------------------------------------------------------

LangChain also provides a Redis-based cache. This is useful if you want to share the cache across multiple processes or servers. To use it, you'll need to install the `redis` package:

*   npm
*   Yarn
*   pnpm

    npm install ioredis @langchain/community

    yarn add ioredis @langchain/community

    pnpm add ioredis @langchain/community

Then, you can pass a `cache` option when you instantiate the LLM. For example:

    import { ChatOpenAI } from "@langchain/openai";import { Redis } from "ioredis";import { RedisCache } from "@langchain/community/caches/ioredis";const client = new Redis("redis://localhost:6379");const cache = new RedisCache(client, {  ttl: 60, // Optional key expiration value});const model = new ChatOpenAI({ cache });const response1 = await model.invoke("Do something random!");console.log(response1);/*  AIMessage {    content: "Sure! I'll generate a random number for you: 37",    additional_kwargs: {}  }*/const response2 = await model.invoke("Do something random!");console.log(response2);/*  AIMessage {    content: "Sure! I'll generate a random number for you: 37",    additional_kwargs: {}  }*/await client.disconnect();

#### API Reference:

*   [ChatOpenAI](https://v02.api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [RedisCache](https://v02.api.js.langchain.com/classes/langchain_community_caches_ioredis.RedisCache.html) from `@langchain/community/caches/ioredis`

Caching on the File System[​](#caching-on-the-file-system "Direct link to Caching on the File System")
------------------------------------------------------------------------------------------------------

danger

This cache is not recommended for production use. It is only intended for local development.

LangChain provides a simple file system cache. By default the cache is stored a temporary directory, but you can specify a custom directory if you want.

    const cache = await LocalFileCache.create();

Next steps[​](#next-steps "Direct link to Next steps")
------------------------------------------------------

You've now learned how to cache model responses to save time and money.

Next, check out the other how-to guides on chat models, like [how to get a model to return structured output](/v0.2/docs/how_to/structured_output) or [how to create your own custom chat model](/v0.2/docs/how_to/custom_chat).

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to cache model responses

](/v0.2/docs/how_to/llm_caching)[

Next

How to create a custom LLM class

](/v0.2/docs/how_to/custom_llm)

*   [In Memory Cache](#in-memory-cache)
*   [Caching with Redis](#caching-with-redis)
*   [Caching on the File System](#caching-on-the-file-system)
*   [Next steps](#next-steps)