viruPVS/langchain-docs huggingface dataset:
Performs similarity search from a embeddings vector.
Parameters
query_embedding – Embeddings vector to search for.
k – Number of results to return.
custom_query – Use this custom query instead default query (kwargs)
kwargs – other vector store specific parameters
Returns
List of ids from adding the texts into the vectorstore.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with distance.
langchain_community.vectorstores.starrocks.has_mul_sub_str¶
langchain_community.vectorstores.starrocks.has_mul_sub_str(s: str, *args: Any) → bool[source]¶
Check if a string has multiple substrings.
:param s: The string to check
:param *args: The substrings to check for in the string
Returns
True if all substrings are present in the string, False otherwise
Return type
bool
langchain_community.vectorstores.redis.filters.RedisFilterOperator¶
class langchain_community.vectorstores.redis.filters.RedisFilterOperator(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
RedisFilterOperator enumerator is used to create RedisFilterExpressions.
EQ = 1¶
NE = 2¶
LT = 3¶
GT = 4¶
LE = 5¶
GE = 6¶
OR = 7¶
AND = 8¶
LIKE = 9¶
IN = 10¶
langchain_community.vectorstores.analyticdb.AnalyticDB¶
class langchain_community.vectorstores.analyticdb.AnalyticDB(connection_string: str, embedding_function: Embeddings, embedding_dimension: int = 1536, collection_name: str = 'langchain_document', pre_delete_collection: bool = False, logger: Optional[Logger] = None, engine_args: Optional[dict] = None)[source]¶
AnalyticDB (distributed PostgreSQL) vector store.
AnalyticDB is a distributed full postgresql syntax cloud-native database.
- connection_string is a postgres connection string.
- embedding_function any embedding function implementing
langchain.embeddings.base.Embeddings interface.
collection_name is the name of the collection to use. (default: langchain)
NOTE: This is not the name of the table, but the name of the collection.The tables will be created when initializing the store (if not exists)
So, make sure the user has the right permissions to create tables.
pre_delete_collection if True, will delete the collection if it exists.(default: False)
- Useful for testing.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(connection_string, embedding_function)
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, batch_size])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
connection_string_from_db_params(driver, ...)
Return connection string from database parameters.
create_collection()
create_table_if_not_exists()
delete([ids])
Delete by vector IDs.
delete_collection()
from_documents(documents, embedding[, ...])
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Return VectorStore initialized from texts and embeddings.
get_connection_string(kwargs)
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Run similarity search with AnalyticDB with distance.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, filter])
Return docs most similar to query.
similarity_search_with_score_by_vector(embedding)
__init__(connection_string: str, embedding_function: Embeddings, embedding_dimension: int = 1536, collection_name: str = 'langchain_document', pre_delete_collection: bool = False, logger: Optional[Logger] = None, engine_args: Optional[dict] = None) → None[source]¶
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, batch_size: int = 500, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
kwargs – vectorstore specific parameters
Returns
List of ids from adding the texts into the vectorstore.
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
classmethod connection_string_from_db_params(driver: str, host: str, port: int, database: str, user: str, password: str) → str[source]¶
Return connection string from database parameters.
create_collection() → None[source]¶
create_table_if_not_exists() → None[source]¶
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Delete by vector IDs.
Parameters
ids – List of ids to delete.
delete_collection() → None[source]¶
classmethod from_documents(documents: List[Document], embedding: Embeddings, embedding_dimension: int = 1536, collection_name: str = 'langchain_document', ids: Optional[List[str]] = None, pre_delete_collection: bool = False, engine_args: Optional[dict] = None, **kwargs: Any) → AnalyticDB[source]¶
Return VectorStore initialized from documents and embeddings.
Postgres Connection string is required
Either pass it as a parameter
or set the PG_CONNECTION_STRING environment variable.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, embedding_dimension: int = 1536, collection_name: str = 'langchain_document', ids: Optional[List[str]] = None, pre_delete_collection: bool = False, engine_args: Optional[dict] = None, **kwargs: Any) → AnalyticDB[source]¶
Return VectorStore initialized from texts and embeddings.
Postgres Connection string is required
Either pass it as a parameter
or set the PG_CONNECTION_STRING environment variable.
Either pass it as a parameter
or set the PG_CONNECTION_STRING environment variable.
classmethod get_connection_string(kwargs: Dict[str, Any]) → str[source]¶
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Run similarity search with AnalyticDB with distance.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents most similar to the query.
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[dict] = None) → List[Tuple[Document, float]][source]¶
Return docs most similar to query.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents most similar to the query and score for each
similarity_search_with_score_by_vector(embedding: List[float], k: int = 4, filter: Optional[dict] = None) → List[Tuple[Document, float]][source]¶
Examples using AnalyticDB¶
AnalyticDB
langchain_core.vectorstores.VectorStoreRetriever¶
class langchain_core.vectorstores.VectorStoreRetriever[source]¶
Bases: BaseRetriever
Base Retriever class for VectorStore.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param metadata: Optional[Dict[str, Any]] = None¶
Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param search_kwargs: dict [Optional]¶
Keyword arguments to pass to the search function.
param search_type: str = 'similarity'¶
Type of search to perform. Defaults to “similarity”.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param vectorstore: langchain_core.vectorstores.VectorStore [Required]¶
VectorStore to use for retrieval.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str][source]¶
Add documents to vectorstore.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
add_documents(documents: List[Document], **kwargs: Any) → List[str][source]¶
Add documents to vectorstore.
async aget_relevant_documents(query: str, *, callbacks: Callbacks = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, **kwargs: Any) → List[Document]¶
Asynchronously get documents relevant to a query.
:param query: string to find relevant documents for
:param callbacks: Callback manager or list of callbacks
:param tags: Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Parameters
metadata – Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Returns
List of relevant documents
async ainvoke(input: str, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → List[Document]¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
async astream(input: Input, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → AsyncIterator[Output]¶
Default implementation of astream, which calls ainvoke.
Subclasses should override this method if they support streaming output.
between retries
stop_after_attempt – The maximum number of attempts to make before giving up
Returns
A new Runnable that retries the original runnable on exceptions.
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
property InputType: Type[langchain_core.runnables.utils.Input]¶
The type of input this runnable accepts specified as a type annotation.
property OutputType: Type[langchain_core.runnables.utils.Output]¶
The type of output this runnable produces specified as a type annotation.
allowed_search_types: ClassVar[Collection[str]] = ('similarity', 'similarity_score_threshold', 'mmr')¶
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
langchain_community.vectorstores.astradb.AstraDB¶
class langchain_community.vectorstores.astradb.AstraDB(*, embedding: Embeddings, collection_name: str, token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, metric: Optional[str] = None, batch_size: Optional[int] = None, bulk_insert_batch_concurrency: Optional[int] = None, bulk_insert_overwrite_concurrency: Optional[int] = None, bulk_delete_concurrency: Optional[int] = None, pre_delete_collection: bool = False)[source]¶
Wrapper around DataStax Astra DB for vector-store workloads.
To use it, you need a recent installation of the astrapy library
and an Astra DB cloud database.
For quickstart and details, visit:docs.datastax.com/en/astra/home/astra.html
Example
      from langchain_community.vectorstores import AstraDB
      from langchain_community.embeddings.openai import OpenAIEmbeddings
      embeddings = OpenAIEmbeddings()
      vectorstore = AstraDB(
        embedding=embeddings,
        collection_name="my_store",
        token="AstraCS:...",
        api_endpoint="https://<DB-ID>-us-east1.apps.astra.datastax.com"
      )
      vectorstore.add_texts(["Giraffes", "All good here"])
      results = vectorstore.similarity_search("Everything's ok", k=1)
Constructor Args (only keyword-arguments accepted):
    embedding (Embeddings): embedding function to use.
    collection_name (str): name of the Astra DB collection to create/use.
    token (Optional[str]): API token for Astra DB usage.
    api_endpoint (Optional[str]): full URL to the API endpoint,
api_endpoint (Optional[str]): full URL to the API endpoint,
        such as "https://<DB-ID>-us-east1.apps.astra.datastax.com".
    astra_db_client (Optional[Any]): *alternative to token+api_endpoint*,
        you can pass an already-created 'astrapy.db.AstraDB' instance.
    namespace (Optional[str]): namespace (aka keyspace) where the
        collection is created. Defaults to the database's "default namespace".
    metric (Optional[str]): similarity function to use out of those
        available in Astra DB. If left out, it will use Astra DB API's
        defaults (i.e. "cosine" - but, for performance reasons,
        "dot_product" is suggested if embeddings are normalized to one).
Advanced arguments (coming with sensible defaults):
    batch_size (Optional[int]): Size of batches for bulk insertions.
    bulk_insert_batch_concurrency (Optional[int]): Number of threads
        to insert batches concurrently.
    bulk_insert_overwrite_concurrency (Optional[int]): Number of
        threads in a batch to insert pre-existing entries.
    bulk_delete_concurrency (Optional[int]): Number of threads
        (for deleting multiple rows concurrently).
    pre_delete_collection (Optional[bool]): whether to delete the collection
        before creating it. If False and the collection already exists,
        the collection will be used as is.
A note on concurrency: as a rule of thumb, on a typical client machine
it is suggested to keep the quantity
    bulk_insert_batch_concurrency * bulk_insert_overwrite_concurrency
much below 1000 to avoid exhausting the client multithreading/networking
resources. The hardcoded defaults are somewhat conservative to meet
most machines' specs, but a sensible choice to test may be:
    bulk_insert_batch_concurrency = 80
bulk_insert_batch_concurrency = 80
    bulk_insert_overwrite_concurrency = 10
A bit of experimentation is required to nail the best results here,
depending on both the machine/network specs and the expected workload
(specifically, how often a write is an update of an existing id).
Remember you can pass concurrency settings to individual calls to
add_texts and add_documents as well.
Create an AstraDB vector store object. See class docstring for help.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(*, embedding, collection_name[, ...])
Create an AstraDB vector store object.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, ...])
Run texts through the embeddings and add them to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
clear()
Empty the collection of all its stored entries.
delete([ids, concurrency])
Delete by vector ids.
delete_by_document_id(document_id)
Remove a single document from the store, given its document_id (str).
delete_collection()
Completely delete the collection from the database (as opposed to 'clear()', which empties it only).
from_documents(documents, embedding, **kwargs)
Create an Astra DB vectorstore from a document list.
from_texts(texts, embedding[, metadatas, ids])
Create an Astra DB vectorstore from raw texts.
max_marginal_relevance_search(query[, k, ...])