Package: llm-elasticsearch-cache
llm-elasticsearch-cacheA caching layer for LLMs that exploits Elasticsearch, fully compatible with LangChain caching, both for chat and embeddings models.Installpipinstallllm-elasticsearch-cacheChat cache usageThe LangChain cache can be used similarly to theother cache integrations.Basic examplefromlangchain.globalsimportset_llm_cachefromllmescache.langchainimportElasticsearchCachefromelasticsearchimportElasticsearches_client=Elasticsearch(hosts="http://localhost:9200")set_llm_cache(ElasticsearchCache(es_client=es_client,es_index="llm-chat-cache",metadata={"project":"my_chatgpt_project"}))Thees_indexparameter can also take aliases. This allows to use theILM: Manage the index lifecyclethat we suggest to consider for managing retention and controlling cache growth.Look at the class docstring for all parameters.Index the generated textThe cached data won't be searchable by default.
The developer can customize the building of the Elasticsearch document in order to add indexed text fields,
where to put, for example, the text generated by the LLM.This can be done by subclassing end overriding methods.
The new cache class can be applied also to a pre-existing cache index:fromllmescache.langchainimportElasticsearchCachefromelasticsearchimportElasticsearchfromlangchain_core.cachesimportRETURN_VAL_TYPEfromtypingimportAny,Dict,Listfromlangchain.globalsimportset_llm_cacheimportjsonclassSearchableElasticsearchCache(ElasticsearchCache):@propertydefmapping(self)->Dict[str,Any]:mapping=super().mappingmapping["mappings"]["properties"]["parsed_llm_output"]={"type":"text","analyzer":"english"}returnmappingdefbuild_document(self,prompt:str,llm_string:str,return_val:RETURN_VAL_TYPE)->Dict[str,Any]:body=super().build_document(prompt,llm_string,return_val)body["parsed_llm_output"]=self._parse_output(body["llm_output"])returnbody@staticmethoddef_parse_output(data:List[str])->List[str]:return[json.loads(output)["kwargs"]["message"]["kwargs"]["content"]foroutputindata]es_client=Elasticsearch(hosts="http://localhost:9200")set_llm_cache(SearchableElasticsearchCache(es_client=es_client,es_index="llm-chat-cache"))Embeddings cache usageCaching embeddings is obtained by using theCacheBackedEmbeddings,
in a slightly different way than the official documentation.fromllmescache.langchainimportElasticsearchStorefromelasticsearchimportElasticsearchfromlangchain.embeddingsimportCacheBackedEmbeddingsfromlangchain_openaiimportOpenAIEmbeddingses_client=Elasticsearch(hosts="http://localhost:9200")underlying_embeddings=OpenAIEmbeddings(model="text-embedding-3-small")store=ElasticsearchStore(es_client=es_client,es_index="llm-embeddings-cache",namespace=underlying_embeddings.model,metadata={"project":"my_llm_project"})cached_embeddings=CacheBackedEmbeddings(underlying_embeddings,store)Similarly to the chat cache, one can subclassElasticsearchStorein order to index vectors for search.fromllmescache.langchainimportElasticsearchStorefromtypingimportAny,Dict,ListclassSearchableElasticsearchStore(ElasticsearchStore):@propertydefmapping(self)->Dict[str,Any]:mapping=super().mappingmapping["mappings"]["properties"]["vector"]={"type":"dense_vector","dims":1536,"index":True,"similarity":"dot_product"}returnmappingdefbuild_document(self,llm_input:str,vector:List[float])->Dict[str,Any]:body=super().build_document(llm_input,vector)body["vector"]=vectorreturnbody