Tony20100/langchain_full huggingface dataset:
llm_string (str) – 
Return type
None
async adelete_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → None[source]¶
A wrapper around adelete with the LLM being passed.
In case the llm.invoke(prompt) calls have a stop param, you should
pass it here
Parameters
prompt (str) – 
llm (LLM) – 
stop (Optional[List[str]]) – 
Return type
None
async alookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
A cache implementation is expected to generate a key from the 2-tuple
of prompt and llm_string (e.g., by concatenating them with a delimiter).
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
Returns
On a cache miss, return None. On a cache hit, return the cached value.
The cached value is a list of Generations (or subclasses).
Return type
Optional[Sequence[Generation]]
async aupdate(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string.
The prompt and llm_string are used to generate a key for the cache.
The key should match that of the look up method.
Parameters
The key should match that of the look up method.
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
return_val (Sequence[Generation]) – The value to be cached. The value is a list of Generations
(or subclasses).
Return type
None
clear(**kwargs: Any) → None[source]¶
Clear cache that can take additional keyword arguments.
Parameters
kwargs (Any) – 
Return type
None
delete(prompt: str, llm_string: str) → None[source]¶
Evict from cache if there’s an entry.
Parameters
prompt (str) – 
llm_string (str) – 
Return type
None
delete_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → None[source]¶
A wrapper around delete with the LLM being passed.
In case the llm.invoke(prompt) calls have a stop param, you should
pass it here
Parameters
prompt (str) – 
llm (LLM) – 
stop (Optional[List[str]]) – 
Return type
None
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
A cache implementation is expected to generate a key from the 2-tuple
of prompt and llm_string (e.g., by concatenating them with a delimiter).
Parameters
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
Returns
On a cache miss, return None. On a cache hit, return the cached value.
The cached value is a list of Generations (or subclasses).
Return type
Optional[Sequence[Generation]]
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string.
The prompt and llm_string are used to generate a key for the cache.
The key should match that of the look up method.
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
return_val (Sequence[Generation]) – The value to be cached. The value is a list of Generations
(or subclasses).
Return type
None
langchain_community.cache.MomentoCache¶
class langchain_community.cache.MomentoCache(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]¶
Cache that uses Momento as a backend. See https://gomomento.com/
Instantiate a prompt cache using Momento as a backend.
Note: to instantiate the cache client passed to MomentoCache,
you must have a Momento account. See https://gomomento.com/.
Parameters
cache_client (CacheClient) – The Momento cache client.
cache_name (str) – The name of the cache to use to store the data.
ttl (Optional[timedelta], optional) – The time to live for the cache items.
Defaults to None, ie use the client default TTL.
ensure_cache_exists (bool, optional) – Create the cache if it doesn’t
exist. Defaults to True.
Raises
ImportError – Momento python package is not installed.
TypeError – cache_client is not of type momento.CacheClientObject
ValueError – ttl is non-null and non-negative
Methods
__init__(cache_client, cache_name, *[, ttl, ...])
Instantiate a prompt cache using Momento as a backend.
aclear(**kwargs)
Clear cache that can take additional keyword arguments.
alookup(prompt, llm_string)
Look up based on prompt and llm_string.
aupdate(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
clear(**kwargs)
Clear the cache.
from_client_params(cache_name, ttl, *[, ...])
Construct cache from CacheClient parameters.
lookup(prompt, llm_string)
Lookup llm generations in cache by prompt and associated model and settings.
Lookup llm generations in cache by prompt and associated model and settings.
update(prompt, llm_string, return_val)
Store llm generations in cache.
__init__(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]¶
Instantiate a prompt cache using Momento as a backend.
Note: to instantiate the cache client passed to MomentoCache,
you must have a Momento account. See https://gomomento.com/.
Parameters
cache_client (CacheClient) – The Momento cache client.
cache_name (str) – The name of the cache to use to store the data.
ttl (Optional[timedelta], optional) – The time to live for the cache items.
Defaults to None, ie use the client default TTL.
ensure_cache_exists (bool, optional) – Create the cache if it doesn’t
exist. Defaults to True.
Raises
ImportError – Momento python package is not installed.
TypeError – cache_client is not of type momento.CacheClientObject
ValueError – ttl is non-null and non-negative
async aclear(**kwargs: Any) → None¶
Clear cache that can take additional keyword arguments.
Parameters
kwargs (Any) – 
Return type
None
async alookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]¶
Look up based on prompt and llm_string.
A cache implementation is expected to generate a key from the 2-tuple
of prompt and llm_string (e.g., by concatenating them with a delimiter).
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
Returns
On a cache miss, return None. On a cache hit, return the cached value.
The cached value is a list of Generations (or subclasses).
Return type
Optional[Sequence[Generation]]
async aupdate(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None¶
Update cache based on prompt and llm_string.
The prompt and llm_string are used to generate a key for the cache.
The key should match that of the look up method.
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
return_val (Sequence[Generation]) – The value to be cached. The value is a list of Generations
(or subclasses).
Return type
None
clear(**kwargs: Any) → None[source]¶
Clear the cache.
Raises
SdkException – Momento service or network error
Parameters
kwargs (Any) – 
Return type
None
Parameters
kwargs (Any) – 
Return type
None
classmethod from_client_params(cache_name: str, ttl: timedelta, *, configuration: Optional[momento.config.Configuration] = None, api_key: Optional[str] = None, auth_token: Optional[str] = None, **kwargs: Any) → MomentoCache[source]¶
Construct cache from CacheClient parameters.
Parameters
cache_name (str) – 
ttl (timedelta) – 
configuration (Optional[momento.config.Configuration]) – 
api_key (Optional[str]) – 
auth_token (Optional[str]) – 
kwargs (Any) – 
Return type
MomentoCache
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Lookup llm generations in cache by prompt and associated model and settings.
Parameters
prompt (str) – The prompt run through the language model.
llm_string (str) – The language model version and settings.
Raises
SdkException – Momento service or network error
Returns
A list of language model generations.
Return type
Optional[RETURN_VAL_TYPE]
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Store llm generations in cache.
Parameters
prompt (str) – The prompt run through the language model.
llm_string (str) – The language model string.
return_val (RETURN_VAL_TYPE) – A list of language model generations.
Raises
SdkException – Momento service or network error
Exception – Unexpected response
Return type
None
Examples using MomentoCache¶
Model caches
Momento
langchain_elasticsearch.cache.ElasticsearchEmbeddingsCache¶
class langchain_elasticsearch.cache.ElasticsearchEmbeddingsCache(index_name: str, store_input: bool = True, metadata: Optional[Dict[str, Any]] = None, namespace: Optional[str] = None, maximum_duplicates_allowed: int = 1, *, es_url: Optional[str] = None, es_cloud_id: Optional[str] = None, es_user: Optional[str] = None, es_api_key: Optional[str] = None, es_password: Optional[str] = None, es_params: Optional[Dict[str, Any]] = None)[source]¶
An Elasticsearch store for caching embeddings.
Initialize the Elasticsearch cache store by specifying the index/alias
to use and determining which additional information (like input, input
parameters, and any other metadata) should be stored in the cache.
Provide a namespace to organize the cache.
Parameters
index_name (str) – The name of the index or the alias to use for the cache.
If they do not exist an index is created,
according to the default mapping defined by the mapping property.
store_input (bool) – Whether to store the input in the cache.
Default to True.
metadata (Optional[dict]) – Additional metadata to store in the cache,
for filtering purposes. This must be JSON serializable in an
Elasticsearch document. Default to None.
namespace (Optional[str]) – A namespace to use for the cache.
maximum_duplicates_allowed (int) – Defines the maximum number of duplicate
keys permitted. Must be used in scenarios where the same key appears
across multiple indices that share the same alias. Default to 1.
es_url (Optional[str]) – URL of the Elasticsearch instance to connect to.
es_cloud_id (Optional[str]) – Cloud ID of the Elasticsearch instance to connect to.
es_cloud_id (Optional[str]) – Cloud ID of the Elasticsearch instance to connect to.
es_user (Optional[str]) – Username to use when connecting to Elasticsearch.
es_password (Optional[str]) – Password to use when connecting to Elasticsearch.
es_api_key (Optional[str]) – API key to use when connecting to Elasticsearch.
es_params (Optional[Dict[str, Any]]) – Other parameters for the Elasticsearch client.
Attributes
mapping
Get the default mapping for the index.
Methods
__init__(index_name[, store_input, ...])
Initialize the Elasticsearch cache store by specifying the index/alias to use and determining which additional information (like input, input parameters, and any other metadata) should be stored in the cache.
amdelete(keys)
Delete the given keys and their associated values.
amget(keys)
Get the values associated with the given keys.
amset(key_value_pairs)
Set the values for the given keys.
ayield_keys(*[, prefix])
Get an iterator over keys that match the given prefix.
build_document(text_input, vector)
Build the Elasticsearch document for storing a single embedding
decode_vector(data)
Decode the base64 string to vector data as bytes.
encode_vector(data)
Encode the vector data as bytes to as a base64 string.
mdelete(keys)
Delete the given keys and their associated values.
mget(keys)
Get the values associated with the given keys.
mset(key_value_pairs)
Set the values for the given keys.
yield_keys(*[, prefix])
Get an iterator over keys that match the given prefix.
yield_keys(*[, prefix])
Get an iterator over keys that match the given prefix.
__init__(index_name: str, store_input: bool = True, metadata: Optional[Dict[str, Any]] = None, namespace: Optional[str] = None, maximum_duplicates_allowed: int = 1, *, es_url: Optional[str] = None, es_cloud_id: Optional[str] = None, es_user: Optional[str] = None, es_api_key: Optional[str] = None, es_password: Optional[str] = None, es_params: Optional[Dict[str, Any]] = None)[source]¶
Initialize the Elasticsearch cache store by specifying the index/alias
to use and determining which additional information (like input, input
parameters, and any other metadata) should be stored in the cache.
Provide a namespace to organize the cache.
Parameters
index_name (str) – The name of the index or the alias to use for the cache.
If they do not exist an index is created,
according to the default mapping defined by the mapping property.
store_input (bool) – Whether to store the input in the cache.
Default to True.
metadata (Optional[dict]) – Additional metadata to store in the cache,
for filtering purposes. This must be JSON serializable in an
Elasticsearch document. Default to None.
namespace (Optional[str]) – A namespace to use for the cache.
maximum_duplicates_allowed (int) – Defines the maximum number of duplicate
keys permitted. Must be used in scenarios where the same key appears
across multiple indices that share the same alias. Default to 1.
es_url (Optional[str]) – URL of the Elasticsearch instance to connect to.
es_cloud_id (Optional[str]) – Cloud ID of the Elasticsearch instance to connect to.
es_user (Optional[str]) – Username to use when connecting to Elasticsearch.
es_user (Optional[str]) – Username to use when connecting to Elasticsearch.
es_password (Optional[str]) – Password to use when connecting to Elasticsearch.
es_api_key (Optional[str]) – API key to use when connecting to Elasticsearch.
es_params (Optional[Dict[str, Any]]) – Other parameters for the Elasticsearch client.
async amdelete(keys: Sequence[K]) → None¶
Delete the given keys and their associated values.
Parameters
keys (Sequence[K]) – A sequence of keys to delete.
Return type
None
async amget(keys: Sequence[K]) → List[Optional[V]]¶
Get the values associated with the given keys.
Parameters
keys (Sequence[K]) – A sequence of keys.
Returns
A sequence of optional values associated with the keys.
If a key is not found, the corresponding value will be None.
Return type
List[Optional[V]]
async amset(key_value_pairs: Sequence[Tuple[K, V]]) → None¶
Set the values for the given keys.
Parameters
key_value_pairs (Sequence[Tuple[K, V]]) – A sequence of key-value pairs.
Return type
None
async ayield_keys(*, prefix: Optional[str] = None) → Union[AsyncIterator[K], AsyncIterator[str]]¶
Get an iterator over keys that match the given prefix.
Parameters
prefix (str) – The prefix to match.
Returns
An iterator over keys that match the given prefix.
This method is allowed to return an iterator over either K or str
depending on what makes more sense for the given store.
Return type
Iterator[K | str]
build_document(text_input: str, vector: bytes) → Dict[str, Any][source]¶
Build the Elasticsearch document for storing a single embedding
Parameters
text_input (str) – 
vector (bytes) – 
Return type
Dict[str, Any]
vector (bytes) – 
Return type
Dict[str, Any]
static decode_vector(data: str) → bytes[source]¶
Decode the base64 string to vector data as bytes.
Parameters
data (str) – 
Return type
bytes
static encode_vector(data: bytes) → str[source]¶
Encode the vector data as bytes to as a base64 string.
Parameters
data (bytes) – 
Return type
str
mdelete(keys: Sequence[str]) → None[source]¶
Delete the given keys and their associated values.
Parameters
keys (Sequence[str]) – 
Return type
None
mget(keys: Sequence[str]) → List[Optional[bytes]][source]¶
Get the values associated with the given keys.
Parameters
keys (Sequence[str]) – 
Return type
List[Optional[bytes]]
mset(key_value_pairs: Sequence[Tuple[str, bytes]]) → None[source]¶
Set the values for the given keys.
Parameters
key_value_pairs (Sequence[Tuple[str, bytes]]) – 
Return type
None
yield_keys(*, prefix: Optional[str] = None) → Iterator[str][source]¶
Get an iterator over keys that match the given prefix.
Parameters
prefix (Optional[str]) – 
Return type
Iterator[str]
langchain_community.cache.SQLiteCache¶
class langchain_community.cache.SQLiteCache(database_path: str = '.langchain.db')[source]¶
Cache that uses SQLite as a backend.
Initialize by creating the engine and all tables.
Methods
__init__([database_path])
Initialize by creating the engine and all tables.
aclear(**kwargs)
Clear cache that can take additional keyword arguments.
alookup(prompt, llm_string)
Look up based on prompt and llm_string.
aupdate(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update based on prompt and llm_string.
Parameters
database_path (str) – 
__init__(database_path: str = '.langchain.db')[source]¶
Initialize by creating the engine and all tables.
Parameters
database_path (str) – 
async aclear(**kwargs: Any) → None¶
Clear cache that can take additional keyword arguments.
Parameters
kwargs (Any) – 
Return type
None
async alookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]¶
Look up based on prompt and llm_string.
A cache implementation is expected to generate a key from the 2-tuple
of prompt and llm_string (e.g., by concatenating them with a delimiter).
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
Returns
On a cache miss, return None. On a cache hit, return the cached value.
The cached value is a list of Generations (or subclasses).
Return type
Optional[Sequence[Generation]]
async aupdate(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None¶
Update cache based on prompt and llm_string.
The prompt and llm_string are used to generate a key for the cache.
The key should match that of the look up method.
Parameters
prompt (str) – a string representation of the prompt.
In the case of a Chat model, the prompt is a non-trivial
serialization of the prompt into the language model.
llm_string (str) – A string representation of the LLM configuration.
This is used to capture the invocation parameters of the LLM
(e.g., model name, temperature, stop tokens, max tokens, etc.).
These invocation parameters are serialized into a string
representation.
return_val (Sequence[Generation]) – The value to be cached. The value is a list of Generations
(or subclasses).
Return type
None
clear(**kwargs: Any) → None¶
Clear cache.
Parameters
kwargs (Any) – 
Return type
None
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]¶
Look up based on prompt and llm_string.
Parameters
prompt (str) – 
llm_string (str) – 
Return type
Optional[Sequence[Generation]]
llm_string (str) – 
Return type
Optional[Sequence[Generation]]
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None¶
Update based on prompt and llm_string.
Parameters
prompt (str) – 
llm_string (str) – 
return_val (Sequence[Generation]) – 
Return type
None
Examples using SQLiteCache¶
DSPy
How to cache LLM responses
How to cache chat model responses
Model caches
langchain_community.cache.AsyncRedisCache¶
class langchain_community.cache.AsyncRedisCache(redis_: Any, *, ttl: Optional[int] = None)[source]¶
Cache that uses Redis as a backend. Allows to use an
async redis.asyncio.Redis client.
Initialize an instance of AsyncRedisCache.
This method initializes an object with Redis caching capabilities.
It takes a redis_ parameter, which should be an instance of a Redis
client class (redis.asyncio.Redis), allowing the object
to interact with a Redis server for caching purposes.
Parameters
redis (Any) – An instance of a Redis client class
(redis.asyncio.Redis) to be used for caching.
This allows the object to communicate with a
Redis server for caching operations.
ttl (int, optional) – Time-to-live (TTL) for cached items in seconds.
If provided, it sets the time duration for how long cached
items will remain valid. If not provided, cached items will not
have an automatic expiration.
redis_ (Any) – 
Methods
__init__(redis_, *[, ttl])
Initialize an instance of AsyncRedisCache.
aclear(**kwargs)
Clear cache.
alookup(prompt, llm_string)
Look up based on prompt and llm_string.
aupdate(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
__init__(redis_: Any, *, ttl: Optional[int] = None)[source]¶
Initialize an instance of AsyncRedisCache.
This method initializes an object with Redis caching capabilities.
Initialize an instance of AsyncRedisCache.
This method initializes an object with Redis caching capabilities.
It takes a redis_ parameter, which should be an instance of a Redis
client class (redis.asyncio.Redis), allowing the object
to interact with a Redis server for caching purposes.
Parameters
redis (Any) – An instance of a Redis client class
(redis.asyncio.Redis) to be used for caching.
This allows the object to communicate with a
Redis server for caching operations.
ttl (int, optional) – Time-to-live (TTL) for cached items in seconds.
If provided, it sets the time duration for how long cached
items will remain valid. If not provided, cached items will not
have an automatic expiration.
redis_ (Any) – 
async aclear(**kwargs: Any) → None[source]¶
Clear cache. If asynchronous is True, flush asynchronously.
Async version.
Parameters
kwargs (Any) – 
Return type
None
async alookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string. Async version.
Parameters
prompt (str) – 
llm_string (str) – 
Return type
Optional[Sequence[Generation]]
async aupdate(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string. Async version.
Parameters
prompt (str) – 
llm_string (str) – 
return_val (Sequence[Generation]) – 
Return type
None
clear(**kwargs: Any) → None[source]¶
Clear cache. If asynchronous is True, flush asynchronously.
Parameters
kwargs (Any) – 
Return type
None