viruPVS/langchain-docs huggingface dataset:
namespace (str) : The instance data will be partitioned based on the “namespace”
field,If the namespace is enabled, you need to specify the namespace field
name during initialization, Otherwise, the queries cannot be executed
correctly.
embedding_field_separator(str): Delimiter specified for writing vectorfield data, default is comma.
output_fields: Specify the field list returned when invoking OpenSearch,by default it is the value list of the field mapping field.
Methods
__init__(endpoint, instance_id, username, ...)
__init__(endpoint: str, instance_id: str, username: str, password: str, table_name: str, field_name_mapping: Dict[str, str], protocol: str = 'http', namespace: str = '', embedding_field_separator: str = ',', output_fields: Optional[List[str]] = None) → None[source]¶
Examples using AlibabaCloudOpenSearchSettings¶
Alibaba Cloud OpenSearch
langchain_community.vectorstores.vectara.VectaraRetriever¶
class langchain_community.vectorstores.vectara.VectaraRetriever[source]¶
Bases: VectorStoreRetriever
Retriever class for Vectara.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param metadata: Optional[Dict[str, Any]] = None¶
Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param search_kwargs: dict [Optional]¶
Search params.
k: Number of Documents to return. Defaults to 5.
lambda_val: lexical match parameter for hybrid search.
filter: Dictionary of argument(s) to filter on metadata. For example a
filter can be “doc.rating > 3.0 and part.lang = ‘deu’”} see
https://docs.vectara.com/docs/search-apis/sql/filter-overview
for more details.
n_sentence_context: number of sentences before/after the matching segment to add
param search_type: str = 'similarity'¶
Type of search to perform. Defaults to “similarity”.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param vectorstore: Vectara [Required]¶
Vectara vectorstore.
param vectorstore: Vectara [Required]¶
Vectara vectorstore.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Add documents to vectorstore.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Add documents to vectorstore.
add_texts(texts: List[str], metadatas: Optional[List[dict]] = None, doc_metadata: Optional[dict] = None) → None[source]¶
Add text to the Vectara vectorstore.
Parameters
texts (List[str]) – The text
metadatas (List[dict]) – Metadata dicts, must line up with existing store
async aget_relevant_documents(query: str, *, callbacks: Callbacks = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, **kwargs: Any) → List[Document]¶
Asynchronously get documents relevant to a query.
:param query: string to find relevant documents for
:param callbacks: Callback manager or list of callbacks
:param tags: Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Parameters
and passed as arguments to the handlers defined in callbacks.
Parameters
metadata – Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Returns
List of relevant documents
async ainvoke(input: str, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → List[Document]¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
async astream(input: Input, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → AsyncIterator[Output]¶
Default implementation of astream, which calls ainvoke.
Subclasses should override this method if they support streaming output.
async astream_log(input: Any, config: Optional[RunnableConfig] = None, *, diff: bool = True, with_streamed_output_list: bool = True, include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Optional[Any]) → Union[AsyncIterator[RunLogPatch], AsyncIterator[RunLog]]¶
Stream all output from a runnable, as reported to the callback system.
This includes all inner runs of LLMs, Retrievers, Tools, etc.
Output is streamed as Log objects, which include a list of
jsonpatch ops that describe how the state of the run has changed in each
step, and the final state of the run.
To mark a field as configurable, see the configurable_fields
and configurable_alternatives methods.
Parameters
include – A list of fields to include in the config schema.
Returns
A pydantic model that can be used to validate config.
configurable_alternatives(which: ConfigurableField, *, default_key: str = 'default', prefix_keys: bool = False, **kwargs: Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) → RunnableSerializable[Input, Output]¶
configurable_fields(**kwargs: Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) → RunnableSerializable[Input, Output]¶
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
property OutputType: Type[langchain_core.runnables.utils.Output]¶
The type of output this runnable produces specified as a type annotation.
allowed_search_types: ClassVar[Collection[str]] = ('similarity', 'similarity_score_threshold', 'mmr')¶
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VectaraRetriever¶
Chat Over Documents with Vectara
langchain_community.vectorstores.pgembedding.PGEmbedding¶
class langchain_community.vectorstores.pgembedding.PGEmbedding(connection_string: str, embedding_function: Embeddings, collection_name: str = 'langchain', collection_metadata: Optional[dict] = None, pre_delete_collection: bool = False, logger: Optional[Logger] = None)[source]¶
Postgres with the pg_embedding extension as a vector store.
pg_embedding uses sequential scan by default. but you can create a HNSW index
using the create_hnsw_index method.
- connection_string is a postgres connection string.
- embedding_function any embedding function implementing
langchain.embeddings.base.Embeddings interface.
collection_name is the name of the collection to use. (default: langchain)
NOTE: This is not the name of the table, but the name of the collection.The tables will be created when initializing the store (if not exists)
So, make sure the user has the right permissions to create tables.
distance_strategy is the distance strategy to use. (default: EUCLIDEAN)
EUCLIDEAN is the euclidean distance.
pre_delete_collection if True, will delete the collection if it exists.(default: False)
- Useful for testing.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(connection_string, embedding_function)
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_embeddings(texts, embeddings, metadatas, ...)
add_texts(texts[, metadatas, ids])
add_texts(texts[, metadatas, ids])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
connect()
create_collection()
create_hnsw_extension()
create_hnsw_index([max_elements, dims, m, ...])
create_tables_if_not_exists()
delete([ids])
Delete by vector ID or other criteria.
delete_collection()
drop_tables()
from_documents(documents, embedding[, ...])
Return VectorStore initialized from documents and embeddings.
from_embeddings(text_embeddings, embedding)
from_existing_index(embedding[, ...])
from_texts(texts, embedding[, metadatas, ...])
from_texts(texts, embedding[, metadatas, ...])
Return VectorStore initialized from texts and embeddings.
get_collection(session)
get_connection_string(kwargs)
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, filter])
Run similarity search with distance.
similarity_search_with_score_by_vector(embedding)
__init__(connection_string: str, embedding_function: Embeddings, collection_name: str = 'langchain', collection_metadata: Optional[dict] = None, pre_delete_collection: bool = False, logger: Optional[Logger] = None) → None[source]¶
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_embeddings(texts: List[str], embeddings: List[List[float]], metadatas: List[dict], ids: List[str], **kwargs: Any) → None[source]¶
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
kwargs – vectorstore specific parameters
Returns
List of ids from adding the texts into the vectorstore.
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
connect() → Connection[source]¶
create_collection() → None[source]¶
create_hnsw_extension() → None[source]¶
create_hnsw_index(max_elements: int = 10000, dims: int = 1536, m: int = 8, ef_construction: int = 16, ef_search: int = 16) → None[source]¶
create_tables_if_not_exists() → None[source]¶
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
delete_collection() → None[source]¶
drop_tables() → None[source]¶
classmethod from_documents(documents: List[Document], embedding: Embeddings, collection_name: str = 'langchain', ids: Optional[List[str]] = None, pre_delete_collection: bool = False, **kwargs: Any) → PGEmbedding[source]¶
Return VectorStore initialized from documents and embeddings.
Return VectorStore initialized from documents and embeddings.
classmethod from_embeddings(text_embeddings: List[Tuple[str, List[float]]], embedding: Embeddings, metadatas: Optional[List[dict]] = None, collection_name: str = 'langchain', ids: Optional[List[str]] = None, pre_delete_collection: bool = False, **kwargs: Any) → PGEmbedding[source]¶
classmethod from_existing_index(embedding: Embeddings, collection_name: str = 'langchain', pre_delete_collection: bool = False, **kwargs: Any) → PGEmbedding[source]¶
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, collection_name: str = 'langchain', ids: Optional[List[str]] = None, pre_delete_collection: bool = False, **kwargs: Any) → PGEmbedding[source]¶
Return VectorStore initialized from texts and embeddings.
get_collection(session: Session) → Optional[CollectionStore][source]¶
classmethod get_connection_string(kwargs: Dict[str, Any]) → str[source]¶
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[dict] = None) → List[Tuple[Document, float]][source]¶
Run similarity search with distance.
similarity_search_with_score_by_vector(embedding: List[float], k: int = 4, filter: Optional[dict] = None) → List[Tuple[Document, float]][source]¶
Examples using PGEmbedding¶
Postgres Embedding
langchain_community.vectorstores.redis.filters.RedisFilter¶
class langchain_community.vectorstores.redis.filters.RedisFilter[source]¶
Collection of RedisFilterFields.
Methods
__init__()
num(field)
tag(field)
text(field)
__init__()¶
static num(field: str) → RedisNum[source]¶
static tag(field: str) → RedisTag[source]¶
static text(field: str) → RedisText[source]¶
Examples using RedisFilter¶
Redis
langchain_community.vectorstores.milvus.Milvus¶
class langchain_community.vectorstores.milvus.Milvus(embedding_function: Embeddings, collection_name: str = 'LangChainCollection', connection_args: Optional[dict[str, Any]] = None, consistency_level: str = 'Session', index_params: Optional[dict] = None, search_params: Optional[dict] = None, drop_old: Optional[bool] = False, *, primary_field: str = 'pk', text_field: str = 'text', vector_field: str = 'vector')[source]¶
Milvus vector store.
You need to install pymilvus and run Milvus.
See the following documentation for how to run a Milvus instance:
https://milvus.io/docs/install_standalone-docker.md
If looking for a hosted Milvus, take a look at this documentation:
https://zilliz.com/cloud and make use of the Zilliz vectorstore found in
this project.
IF USING L2/IP metric, IT IS HIGHLY SUGGESTED TO NORMALIZE YOUR DATA.
Parameters
embedding_function (Embeddings) – Function used to embed the text.
collection_name (str) – Which Milvus collection to use. Defaults to
“LangChainCollection”.
connection_args (Optional[dict[str, any]]) – The connection args used for
this class comes in the form of a dict.
consistency_level (str) – The consistency level to use for a collection.
Defaults to “Session”.
index_params (Optional[dict]) – Which index params to use. Defaults to
HNSW/AUTOINDEX depending on service.
search_params (Optional[dict]) – Which search params to use. Defaults to
default of index.
drop_old (Optional[bool]) – Whether to drop the current collection. Defaults
drop_old (Optional[bool]) – Whether to drop the current collection. Defaults
to False.
primary_field (str) – Name of the primary key field. Defaults to “pk”.
text_field (str) – Name of the text field. Defaults to “text”.
vector_field (str) – Name of the vector field. Defaults to “vector”.
The connection args used for this class comes in the form of a dict,
here are a few of the options:
address (str): The actual address of Milvusinstance. Example address: “localhost:19530”
uri (str): The uri of Milvus instance. Example uri:“http://randomwebsite:19530”,
“tcp:foobarsite:19530”,
“https://ok.s3.south.com:19530”.
host (str): The host of Milvus instance. Default at “localhost”,PyMilvus will fill in the default host if only port is provided.
port (str/int): The port of Milvus instance. Default at 19530, PyMilvuswill fill in the default port if only host is provided.
user (str): Use which user to connect to Milvus instance. If user andpassword are provided, we will add related header in every RPC call.
password (str): Required when user is provided. The passwordcorresponding to the user.
secure (bool): Default is false. If set to true, tls will be enabled.
client_key_path (str): If use tls two-way authentication, need to
write the client.key path.
client_pem_path (str): If use tls two-way authentication, need towrite the client.pem path.
ca_pem_path (str): If use tls two-way authentication, need to writethe ca.pem path.