Tony20100/langchain_full huggingface dataset:
langchain_community.vectorstores.ecloud_vector_search.EcloudESVectorStore¶
class langchain_community.vectorstores.ecloud_vector_search.EcloudESVectorStore(index_name: str, es_url: str, user: Optional[str] = None, password: Optional[str] = None, embedding: Optional[Embeddings] = None, **kwargs: Optional[dict])[source]¶
ecloud Elasticsearch vector store.
Example
from langchain.vectorstores import EcloudESVectorStore
from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = EcloudESVectorStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_url="http://localhost:9200"
)
Parameters
index_name (str) – Name of the Elasticsearch index to create.
es_url (str) – URL of the ecloud Elasticsearch instance to connect to.
user (Optional[str]) – Username to use when connecting to Elasticsearch.
password (Optional[str]) – Password to use when connecting to Elasticsearch.
embedding (Optional[Embeddings]) – 
kwargs (Optional[dict]) – 
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(index_name, es_url[, user, ...])
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
delete([ids])
Delete documents from the index.
es_client(*[, es_url, username, password])
from_documents(documents[, embedding])
Construct EcloudESVectorStore wrapper from documents.
from_texts(texts[, embedding, metadatas])
Construct EcloudESVectorStore wrapper from raw documents.
get_dense_specific_model_similarity_params(...)
get_sparse_specific_model_similarity_params(...)
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Return documents most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query, k[, filter])
Return documents most similar to query, along with scores.
__init__(index_name: str, es_url: str, user: Optional[str] = None, password: Optional[str] = None, embedding: Optional[Embeddings] = None, **kwargs: Optional[dict]) → None[source]¶
Parameters
index_name (str) – 
es_url (str) – 
user (Optional[str]) – 
password (Optional[str]) – 
embedding (Optional[Embeddings]) – 
kwargs (Optional[dict]) – 
Return type
None
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
kwargs (Any) – 
Returns
List of Documents most similar to the query.
Return type
List[Document]
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Delete documents from the index.
Delete documents from the index.
Parameters
ids (Optional[List[str]]) – List of ids of documents to delete
kwargs (Any) – 
Return type
Optional[bool]
static es_client(*, es_url: Optional[str] = None, username: Optional[str] = None, password: Optional[str] = None, **kwargs: Optional[dict]) → Elasticsearch[source]¶
Parameters
es_url (Optional[str]) – 
username (Optional[str]) – 
password (Optional[str]) – 
kwargs (Optional[dict]) – 
Return type
Elasticsearch
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, **kwargs: Any) → EcloudESVectorStore[source]¶
Construct EcloudESVectorStore wrapper from documents.
Parameters
documents (List[Document]) – List of documents to add to the Elasticsearch index.
embedding (Optional[Embeddings]) – Embedding function to use to embed the texts.
Do not provide if using a strategy
that doesn’t require inference.
kwargs (Any) – create index key words arguments
Return type
EcloudESVectorStore
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[Dict[str, Any]]] = None, **kwargs: Any) → EcloudESVectorStore[source]¶
Construct EcloudESVectorStore wrapper from raw documents.
Parameters
texts (List[str]) – List of texts to add to the Elasticsearch index.
embedding (Optional[Embeddings]) – Embedding function to use to embed the texts.
metadatas (Optional[List[Dict[str, Any]]]) – Optional list of metadatas associated with the texts.
index_name – Name of the Elasticsearch index to create.
kwargs (Any) – create index key words arguments
Return type
kwargs (Any) – create index key words arguments
Return type
EcloudESVectorStore
static get_dense_specific_model_similarity_params(search_params: Dict[str, Any]) → Dict[source]¶
Parameters
search_params (Dict[str, Any]) – 
Return type
Dict
static get_sparse_specific_model_similarity_params(search_params: Dict[str, Any]) → Dict[source]¶
Parameters
search_params (Dict[str, Any]) – 
Return type
Dict
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 4, filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return documents most similar to query.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
filter (Optional[dict]) – Array of Elasticsearch filter clauses to apply to the query.
kwargs (Any) – 
Returns
List of Documents most similar to the query,
in descending order of similarity.
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(query: str, k: int, filter: Optional[dict] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return documents most similar to query, along with scores.
Parameters
query (str) – Text to look up documents similar to.
size – Number of Documents to return. Defaults to 4.
filter (Optional[dict]) – Array of Elasticsearch filter clauses to apply to the query.
k (int) – 
kwargs (Any) – 
Returns
List of Documents most similar to the query and score for each
Return type
List[Tuple[Document, float]]
Examples using EcloudESVectorStore¶
China Mobile ECloud ElasticSearch VectorSearch
langchain_elasticsearch.vectorstores.BM25RetrievalStrategy¶
class langchain_elasticsearch.vectorstores.BM25RetrievalStrategy(k1: Optional[float] = None, b: Optional[float] = None)[source]¶
[Deprecated] Retrieval strategy using the native BM25 algorithm of Elasticsearch.
Notes
Deprecated since version 0.2.0: Use BM25Strategy instead.
Methods
__init__([k1, b])
before_index_setup(client, text_field, ...)
Executes before the index is created.
index(dims_length, vector_query_field, ...)
Executes when the index is created.
query(query_vector, query, k, fetch_k, ...)
Executes when a search is performed on the store.
require_inference()
Returns whether or not the strategy requires inference to be performed on the text before it is added to the index.
Parameters
k1 (Optional[float]) – 
b (Optional[float]) – 
__init__(k1: Optional[float] = None, b: Optional[float] = None)[source]¶
Parameters
k1 (Optional[float]) – 
b (Optional[float]) – 
before_index_setup(client: Elasticsearch, text_field: str, vector_query_field: str) → None¶
Executes before the index is created. Used for setting up
any required Elasticsearch resources like a pipeline.
Parameters
client (Elasticsearch) – The Elasticsearch client.
text_field (str) – The field containing the text data in the index.
vector_query_field (str) – The field containing the vector
representations in the index.
Return type
None
index(dims_length: Optional[int], vector_query_field: str, text_field: str, similarity: Optional[DistanceStrategy]) → Dict[source]¶
Executes when the index is created.
Executes when the index is created.
Parameters
dims_length (Optional[int]) – Numeric length of the embedding vectors,
or None if not using vector-based query.
vector_query_field (str) – The field containing the vector
representations in the index.
text_field (str) – The field containing the text data in the index.
similarity (Optional[DistanceStrategy]) – The similarity strategy to use,
or None if not using one.
Returns
The Elasticsearch settings and mappings for the strategy.
Return type
Dict
query(query_vector: Optional[List[float]], query: Optional[str], k: int, fetch_k: int, vector_query_field: str, text_field: str, filter: List[dict], similarity: Optional[DistanceStrategy]) → Dict[source]¶
Executes when a search is performed on the store.
Parameters
query_vector (Optional[List[float]]) – The query vector,
or None if not using vector-based query.
query (Optional[str]) – The text query, or None if not using text-based query.
k (int) – The total number of results to retrieve.
fetch_k (int) – The number of results to fetch initially.
vector_query_field (str) – The field containing the vector
representations in the index.
text_field (str) – The field containing the text data in the index.
filter (List[dict]) – List of filter clauses to apply to the query.
similarity (Optional[DistanceStrategy]) – The similarity strategy to use, or None if not using one.
Returns
The Elasticsearch query body.
Return type
Dict
require_inference() → bool[source]¶
Returns whether or not the strategy requires inference
to be performed on the text before it is added to the index.
Returns
Whether or not the strategy requires inference
langchain_community.vectorstores.azure_cosmos_db.AzureCosmosDBVectorSearch¶
class langchain_community.vectorstores.azure_cosmos_db.AzureCosmosDBVectorSearch(collection: Collection[CosmosDBDocumentType], embedding: Embeddings, *, index_name: str = 'vectorSearchIndex', text_key: str = 'textContent', embedding_key: str = 'vectorContent')[source]¶
Azure Cosmos DB for MongoDB vCore vector store.
To use, you should have both:
- the pymongo python package installed
- a connection string associated with a MongoDB VCore Cluster
Example
. code-block:: python
from langchain_community.vectorstores import AzureCosmosDBVectorSearch
from langchain_community.embeddings.openai import OpenAIEmbeddings
from pymongo import MongoClient
mongo_client = MongoClient(“<YOUR-CONNECTION-STRING>”)
collection = mongo_client[“<db_name>”][“<collection_name>”]
embeddings = OpenAIEmbeddings()
vectorstore = AzureCosmosDBVectorSearch(collection, embeddings)
Constructor for AzureCosmosDBVectorSearch
Parameters
collection (Collection[CosmosDBDocumentType]) – MongoDB collection to add the texts to.
embedding (Embeddings) – Text embedding model to use.
index_name (str) – Name of the Atlas Search index.
text_key (str) – MongoDB field that will contain the text
for each document.
embedding_key (str) – MongoDB field that will contain the embedding
for each document.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(collection, embedding, *[, ...])
Constructor for AzureCosmosDBVectorSearch
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
create_index([num_lists, dimensions, ...])
Creates an index using the index name specified at
delete([ids])
Delete by vector ID or other criteria.
delete_document_by_id([document_id])
Removes a Specific Document by Id
delete_index()
Deletes the index specified during instance construction if it exists
from_connection_string(connection_string, ...)
Deletes the index specified during instance construction if it exists
from_connection_string(connection_string, ...)
Creates an Instance of AzureCosmosDBVectorSearch from a Connection String
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Return VectorStore initialized from texts and embeddings.
get_collection()
get_index_name()
Returns the index name
index_exists()
Verifies if the specified index name during instance
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, kind, ...])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Run similarity search with distance.
__init__(collection: Collection[CosmosDBDocumentType], embedding: Embeddings, *, index_name: str = 'vectorSearchIndex', text_key: str = 'textContent', embedding_key: str = 'vectorContent')[source]¶
Constructor for AzureCosmosDBVectorSearch
Parameters
collection (Collection[CosmosDBDocumentType]) – MongoDB collection to add the texts to.
embedding (Embeddings) – Text embedding model to use.
index_name (str) – Name of the Atlas Search index.
text_key (str) – MongoDB field that will contain the text
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
create_index(num_lists: int = 100, dimensions: int = 1536, similarity: CosmosDBSimilarityType = CosmosDBSimilarityType.COS, kind: str = 'vector-ivf', m: int = 16, ef_construction: int = 64) → dict[str, Any][source]¶
Creates an index using the index name specified atinstance construction
Setting the numLists parameter correctly is important for achievinggood accuracy and performance.
Since the vector store uses IVF as the indexing strategy,
you should create the index only after you
have loaded a large enough sample documents to ensure that the
centroids for the respective buckets are
have loaded a large enough sample documents to ensure that the
centroids for the respective buckets are
faily distributed.
We recommend that numLists is set to documentCount/1000 for upto 1 million documents
and to sqrt(documentCount) for more than 1 million documents.
As the number of items in your database grows, you should
tune numLists to be larger
in order to achieve good latency performance for vector search.
If you’re experimenting with a new scenario or creating a
small demo, you can start with numLists
set to 1 to perform a brute-force search across all vectors.
This should provide you with the most
accurate results from the vector search, however be aware that
the search speed and latency will be slow.
After your initial setup, you should go ahead and tune
the numLists parameter using the above guidance.
Parameters
kind (str) – Type of vector index to create.
Possible options are:
vector-ivf
vector-hnsw: available as a preview feature only,to enable visit https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/preview-features
num_lists (int) – This integer is the number of clusters that the
inverted file (IVF) index uses to group the vector data.
We recommend that numLists is set to documentCount/1000
for up to 1 million documents and to sqrt(documentCount)
for more than 1 million documents.
Using a numLists value of 1 is akin to performing
brute-force search, which has limited performance
dimensions (int) – Number of dimensions for vector similarity.
The maximum number of supported dimensions is 2000
similarity (CosmosDBSimilarityType) – Similarity metric to use with the IVF index.
Possible options are:
CosmosDBSimilarityType.COS (cosine distance),