viruPVS/langchain-docs huggingface dataset:
langchain_community.cache.SQLAlchemyMd5Cache¶
class langchain_community.cache.SQLAlchemyMd5Cache(engine: ~sqlalchemy.engine.base.Engine, cache_schema: ~typing.Type[~langchain_community.cache.FullMd5LLMCache] = <class 'langchain_community.cache.FullMd5LLMCache'>)[source]¶
Cache that uses SQAlchemy as a backend.
Initialize by creating all tables.
Methods
__init__(engine[, cache_schema])
Initialize by creating all tables.
clear(**kwargs)
Clear cache.
get_md5(input_string)
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update based on prompt and llm_string.
__init__(engine: ~sqlalchemy.engine.base.Engine, cache_schema: ~typing.Type[~langchain_community.cache.FullMd5LLMCache] = <class 'langchain_community.cache.FullMd5LLMCache'>)[source]¶
Initialize by creating all tables.
clear(**kwargs: Any) → None[source]¶
Clear cache.
static get_md5(input_string: str) → str[source]¶
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update based on prompt and llm_string.
langchain_community.cache.RedisSemanticCache¶
class langchain_community.cache.RedisSemanticCache(redis_url: str, embedding: Embeddings, score_threshold: float = 0.2)[source]¶
Cache that uses Redis as a vector-store backend.
Initialize by passing in the init GPTCache func
Parameters
redis_url (str) – URL to connect to Redis.
embedding (Embedding) – Embedding provider for semantic encoding and search.
score_threshold (float, 0.2) – 
Example:
from langchain_community.globals import set_llm_cache
from langchain_community.cache import RedisSemanticCache
from langchain_community.embeddings import OpenAIEmbeddings
set_llm_cache(RedisSemanticCache(
    redis_url="redis://localhost:6379",
    embedding=OpenAIEmbeddings()
))
Attributes
DEFAULT_SCHEMA
Methods
__init__(redis_url, embedding[, score_threshold])
Initialize by passing in the init GPTCache func
clear(**kwargs)
Clear semantic cache for a given llm_string.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
__init__(redis_url: str, embedding: Embeddings, score_threshold: float = 0.2)[source]¶
Initialize by passing in the init GPTCache func
Parameters
redis_url (str) – URL to connect to Redis.
embedding (Embedding) – Embedding provider for semantic encoding and search.
score_threshold (float, 0.2) – 
Example:
from langchain_community.globals import set_llm_cache
from langchain_community.cache import RedisSemanticCache
from langchain_community.embeddings import OpenAIEmbeddings
set_llm_cache(RedisSemanticCache(
set_llm_cache(RedisSemanticCache(
    redis_url="redis://localhost:6379",
    embedding=OpenAIEmbeddings()
))
clear(**kwargs: Any) → None[source]¶
Clear semantic cache for a given llm_string.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string.
Examples using RedisSemanticCache¶
Redis
LLM Caching integrations
langchain_community.cache.SQLiteCache¶
class langchain_community.cache.SQLiteCache(database_path: str = '.langchain.db')[source]¶
Cache that uses SQLite as a backend.
Initialize by creating the engine and all tables.
Methods
__init__([database_path])
Initialize by creating the engine and all tables.
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update based on prompt and llm_string.
__init__(database_path: str = '.langchain.db')[source]¶
Initialize by creating the engine and all tables.
clear(**kwargs: Any) → None¶
Clear cache.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]¶
Look up based on prompt and llm_string.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None¶
Update based on prompt and llm_string.
Examples using SQLiteCache¶
LLM Caching integrations
langchain_community.cache.CassandraSemanticCache¶
class langchain_community.cache.CassandraSemanticCache(session: Optional[CassandraSession], keyspace: Optional[str], embedding: Embeddings, table_name: str = 'langchain_llm_semantic_cache', distance_metric: str = 'dot', score_threshold: float = 0.85, ttl_seconds: Optional[int] = None, skip_provisioning: bool = False)[source]¶
Cache that uses Cassandra as a vector-store backend for semantic
(i.e. similarity-based) lookup.
It uses a single (vector) Cassandra table and stores, in principle,
cached values from several LLMs, so the LLM’s llm_string is part
of the rows’ primary keys.
The similarity is based on one of several distance metrics (default: “dot”).
If choosing another metric, the default threshold is to be re-tuned accordingly.
Initialize the cache with all relevant parameters.
:param session: an open Cassandra session
:type session: cassandra.cluster.Session
:param keyspace: the keyspace to use for storing the cache
:type keyspace: str
:param embedding: Embedding provider for semantic
encoding and search.
Parameters
table_name (str) – name of the Cassandra (vector) table
to use as cache
distance_metric (str, 'dot') – which measure to adopt for
similarity searches
score_threshold (optional float) – numeric value to use as
cutoff for the similarity searches
ttl_seconds (optional int) – time-to-live for cache entries
(default: None, i.e. forever)
The default score threshold is tuned to the default metric.
Tune it carefully yourself if switching to another distance metric.
Methods
__init__(session, keyspace, embedding[, ...])
Methods
__init__(session, keyspace, embedding[, ...])
Initialize the cache with all relevant parameters. :param session: an open Cassandra session :type session: cassandra.cluster.Session :param keyspace: the keyspace to use for storing the cache :type keyspace: str :param embedding: Embedding provider for semantic                   encoding and search. :type embedding: Embedding :param table_name: name of the Cassandra (vector) table                    to use as cache :type table_name: str :param distance_metric: which measure to adopt for                         similarity searches :type distance_metric: str, 'dot' :param score_threshold: numeric value to use as                         cutoff for the similarity searches :type score_threshold: optional float :param ttl_seconds: time-to-live for cache entries                     (default: None, i.e. forever) :type ttl_seconds: optional int.
clear(**kwargs)
Clear the whole semantic cache.
delete_by_document_id(document_id)
Given this is a "similarity search" cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
lookup_with_id(prompt, llm_string)
Look up based on prompt and llm_string.
lookup_with_id_through_llm(prompt, llm[, stop])
update(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
__init__(session: Optional[CassandraSession], keyspace: Optional[str], embedding: Embeddings, table_name: str = 'langchain_llm_semantic_cache', distance_metric: str = 'dot', score_threshold: float = 0.85, ttl_seconds: Optional[int] = None, skip_provisioning: bool = False)[source]¶
Initialize the cache with all relevant parameters.
:param session: an open Cassandra session
:type session: cassandra.cluster.Session
:param keyspace: the keyspace to use for storing the cache
:type keyspace: str
:param embedding: Embedding provider for semantic
encoding and search.
Parameters
table_name (str) – name of the Cassandra (vector) table
to use as cache
distance_metric (str, 'dot') – which measure to adopt for
similarity searches
score_threshold (optional float) – numeric value to use as
cutoff for the similarity searches
ttl_seconds (optional int) – time-to-live for cache entries
(default: None, i.e. forever)
The default score threshold is tuned to the default metric.
Tune it carefully yourself if switching to another distance metric.
clear(**kwargs: Any) → None[source]¶
Clear the whole semantic cache.
delete_by_document_id(document_id: str) → None[source]¶
Given this is a “similarity search” cache, an invalidation pattern
that makes sense is first a lookup to get an ID, and then deleting
with that ID. This is for the second step.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
lookup_with_id(prompt: str, llm_string: str) → Optional[Tuple[str, Sequence[Generation]]][source]¶
Look up based on prompt and llm_string.
If there are hits, return (document_id, cached_entry)
lookup_with_id_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → Optional[Tuple[str, Sequence[Generation]]][source]¶
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
langchain_community.cache.GPTCache¶
class langchain_community.cache.GPTCache(init_func: Optional[Union[Callable[[Any, str], None], Callable[[Any], None]]] = None)[source]¶
Cache that uses GPTCache as a backend.
Initialize by passing in init function (default: None).
Parameters
init_func (Optional[Callable[[Any], None]]) – init GPTCache function
(default – None)
Example:
.. code-block:: python
# Initialize GPTCache with a custom init function
import gptcache
from gptcache.processor.pre import get_prompt
from gptcache.manager.factory import get_data_manager
from langchain_community.globals import set_llm_cache
# Avoid multiple caches using the same file,
causing different llm model caches to affect each other
def init_gptcache(cache_obj: gptcache.Cache, llm str):
cache_obj.init(pre_embedding_func=get_prompt,
data_manager=manager_factory(
manager=”map”,
data_dir=f”map_cache_{llm}”
),
)
set_llm_cache(GPTCache(init_gptcache))
Methods
__init__([init_func])
Initialize by passing in init function (default: None).
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up the cache data.
update(prompt, llm_string, return_val)
Update cache.
__init__(init_func: Optional[Union[Callable[[Any, str], None], Callable[[Any], None]]] = None)[source]¶
Initialize by passing in init function (default: None).
Parameters
init_func (Optional[Callable[[Any], None]]) – init GPTCache function
(default – None)
Example:
.. code-block:: python
# Initialize GPTCache with a custom init function
import gptcache
# Initialize GPTCache with a custom init function
import gptcache
from gptcache.processor.pre import get_prompt
from gptcache.manager.factory import get_data_manager
from langchain_community.globals import set_llm_cache
# Avoid multiple caches using the same file,
causing different llm model caches to affect each other
def init_gptcache(cache_obj: gptcache.Cache, llm str):
cache_obj.init(pre_embedding_func=get_prompt,
data_manager=manager_factory(
manager=”map”,
data_dir=f”map_cache_{llm}”
),
)
set_llm_cache(GPTCache(init_gptcache))
clear(**kwargs: Any) → None[source]¶
Clear cache.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up the cache data.
First, retrieve the corresponding cache object using the llm_string parameter,
and then retrieve the data from the cache based on the prompt.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache.
First, retrieve the corresponding cache object using the llm_string parameter,
and then store the prompt and return_val in the cache object.
Examples using GPTCache¶
LLM Caching integrations
langchain_community.cache.AstraDBSemanticCache¶
class langchain_community.cache.AstraDBSemanticCache(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)[source]¶
Cache that uses Astra DB as a vector-store backend for semantic
(i.e. similarity-based) lookup.
It uses a single (vector) collection and can store
cached values from several LLMs, so the LLM’s ‘llm_string’ is stored
in the document metadata.
You can choose the preferred similarity (or use the API default) –
remember the threshold might require metric-dependend tuning.
Initialize the cache with all relevant parameters.
:param collection_name: name of the Astra DB collection to create/use.
:type collection_name: str
:param token: API token for Astra DB usage.
:type token: Optional[str]
:param api_endpoint: full URL to the API endpoint,
such as “https://<DB-ID>-us-east1.apps.astra.datastax.com”.
Parameters
astra_db_client (Optional[Any]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AstraDB’ instance.
namespace (Optional[str]) – namespace (aka keyspace) where the
collection is created. Defaults to the database’s “default namespace”.
embedding (Embedding) – Embedding provider for semantic
encoding and search.
metric – the function to use for evaluating similarity of text embeddings.
Defaults to ‘cosine’ (alternatives: ‘euclidean’, ‘dot_product’)
similarity_threshold (float, optional) – the minimum similarity
similarity_threshold (float, optional) – the minimum similarity
for accepting a (semantic-search) match.
The default score threshold is tuned to the default metric.
Tune it carefully yourself if switching to another distance metric.
Methods
__init__(*[, collection_name, token, ...])
Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint,                      such as "https://<DB-ID>-us-east1.apps.astra.datastax.com". :type api_endpoint: Optional[str] :param astra_db_client: alternative to token+api_endpoint,                         you can pass an already-created 'astrapy.db.AstraDB' instance. :type astra_db_client: Optional[Any] :param namespace: namespace (aka keyspace) where the                   collection is created. Defaults to the database's "default namespace". :type namespace: Optional[str] :param embedding: Embedding provider for semantic                   encoding and search. :type embedding: Embedding :param metric: the function to use for evaluating similarity of text embeddings. Defaults to 'cosine' (alternatives: 'euclidean', 'dot_product') :param similarity_threshold: the minimum similarity                              for accepting a (semantic-search) match. :type similarity_threshold: float, optional.
clear(**kwargs)
Clear the whole semantic cache.
delete_by_document_id(document_id)
Given this is a "similarity search" cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
lookup_with_id(prompt, llm_string)
lookup_with_id(prompt, llm_string)
Look up based on prompt and llm_string.
lookup_with_id_through_llm(prompt, llm[, stop])
update(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
__init__(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)[source]¶
Initialize the cache with all relevant parameters.
:param collection_name: name of the Astra DB collection to create/use.
:type collection_name: str
:param token: API token for Astra DB usage.
:type token: Optional[str]
:param api_endpoint: full URL to the API endpoint,
such as “https://<DB-ID>-us-east1.apps.astra.datastax.com”.
Parameters
astra_db_client (Optional[Any]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AstraDB’ instance.
namespace (Optional[str]) – namespace (aka keyspace) where the
collection is created. Defaults to the database’s “default namespace”.
embedding (Embedding) – Embedding provider for semantic
encoding and search.
metric – the function to use for evaluating similarity of text embeddings.
Defaults to ‘cosine’ (alternatives: ‘euclidean’, ‘dot_product’)
similarity_threshold (float, optional) – the minimum similarity
for accepting a (semantic-search) match.
The default score threshold is tuned to the default metric.
Tune it carefully yourself if switching to another distance metric.
Tune it carefully yourself if switching to another distance metric.
clear(**kwargs: Any) → None[source]¶
Clear the whole semantic cache.
delete_by_document_id(document_id: str) → None[source]¶
Given this is a “similarity search” cache, an invalidation pattern
that makes sense is first a lookup to get an ID, and then deleting
with that ID. This is for the second step.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
lookup_with_id(prompt: str, llm_string: str) → Optional[Tuple[str, Sequence[Generation]]][source]¶
Look up based on prompt and llm_string.
If there are hits, return (document_id, cached_entry) for the top hit
lookup_with_id_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → Optional[Tuple[str, Sequence[Generation]]][source]¶
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string.
langchain_community.cache.FullLLMCache¶
class langchain_community.cache.FullLLMCache(**kwargs)[source]¶
SQLite table for full LLM Cache (all generations).
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and
values in kwargs.
Only keys that are present as
attributes of the instance’s class are allowed. These could be,
for example, any mapped columns or relationships.
Attributes
idx
llm
metadata
prompt
registry
response
Methods
__init__(**kwargs)
A simple constructor that allows initialization from kwargs.
__init__(**kwargs)¶
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and
values in kwargs.
Only keys that are present as
attributes of the instance’s class are allowed. These could be,
for example, any mapped columns or relationships.
langchain_community.cache.UpstashRedisCache¶
class langchain_community.cache.UpstashRedisCache(redis_: Any, *, ttl: Optional[int] = None)[source]¶
Cache that uses Upstash Redis as a backend.
Initialize an instance of UpstashRedisCache.
This method initializes an object with Upstash Redis caching capabilities.
It takes a redis_ parameter, which should be an instance of an Upstash Redis
client class, allowing the object to interact with Upstash Redis
server for caching purposes.
Parameters
redis – An instance of Upstash Redis client class
(e.g., Redis) used for caching.
This allows the object to communicate with
Redis server for caching operations on.
ttl (int, optional) – Time-to-live (TTL) for cached items in seconds.
If provided, it sets the time duration for how long cached
items will remain valid. If not provided, cached items will not
have an automatic expiration.
Methods
__init__(redis_, *[, ttl])
Initialize an instance of UpstashRedisCache.
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update cache based on prompt and llm_string.
__init__(redis_: Any, *, ttl: Optional[int] = None)[source]¶
Initialize an instance of UpstashRedisCache.
This method initializes an object with Upstash Redis caching capabilities.
It takes a redis_ parameter, which should be an instance of an Upstash Redis
client class, allowing the object to interact with Upstash Redis
server for caching purposes.
Parameters
redis – An instance of Upstash Redis client class
(e.g., Redis) used for caching.
This allows the object to communicate with
Redis server for caching operations on.
This allows the object to communicate with
Redis server for caching operations on.
ttl (int, optional) – Time-to-live (TTL) for cached items in seconds.
If provided, it sets the time duration for how long cached
items will remain valid. If not provided, cached items will not
have an automatic expiration.
clear(**kwargs: Any) → None[source]¶
Clear cache. If asynchronous is True, flush asynchronously.
This flushes the whole db.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update cache based on prompt and llm_string.
langchain_community.cache.SQLAlchemyCache¶
class langchain_community.cache.SQLAlchemyCache(engine: ~sqlalchemy.engine.base.Engine, cache_schema: ~typing.Type[~langchain_community.cache.FullLLMCache] = <class 'langchain_community.cache.FullLLMCache'>)[source]¶
Cache that uses SQAlchemy as a backend.
Initialize by creating all tables.
Methods
__init__(engine[, cache_schema])
Initialize by creating all tables.
clear(**kwargs)
Clear cache.
lookup(prompt, llm_string)
Look up based on prompt and llm_string.
update(prompt, llm_string, return_val)
Update based on prompt and llm_string.
__init__(engine: ~sqlalchemy.engine.base.Engine, cache_schema: ~typing.Type[~langchain_community.cache.FullLLMCache] = <class 'langchain_community.cache.FullLLMCache'>)[source]¶
Initialize by creating all tables.
clear(**kwargs: Any) → None[source]¶
Clear cache.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Update based on prompt and llm_string.
Examples using SQLAlchemyCache¶
LLM Caching integrations
langchain_community.cache.MomentoCache¶
class langchain_community.cache.MomentoCache(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]¶
Cache that uses Momento as a backend. See https://gomomento.com/
Instantiate a prompt cache using Momento as a backend.
Note: to instantiate the cache client passed to MomentoCache,
you must have a Momento account. See https://gomomento.com/.
Parameters
cache_client (CacheClient) – The Momento cache client.
cache_name (str) – The name of the cache to use to store the data.
ttl (Optional[timedelta], optional) – The time to live for the cache items.
Defaults to None, ie use the client default TTL.
ensure_cache_exists (bool, optional) – Create the cache if it doesn’t
exist. Defaults to True.
Raises
ImportError – Momento python package is not installed.
TypeError – cache_client is not of type momento.CacheClientObject
ValueError – ttl is non-null and non-negative
Methods
__init__(cache_client, cache_name, *[, ttl, ...])
Instantiate a prompt cache using Momento as a backend.
clear(**kwargs)
Clear the cache.
from_client_params(cache_name, ttl, *[, ...])
Construct cache from CacheClient parameters.
lookup(prompt, llm_string)
Lookup llm generations in cache by prompt and associated model and settings.
update(prompt, llm_string, return_val)
Store llm generations in cache.
__init__(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]¶