Tony20100/langchain_full huggingface dataset:
k (int) – 
fetch_k (int) – 
filter (Optional[Dict[str, List]]) – 
kwargs (Any) – 
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(query: str, k: int = 3, fetch_k: int = 15, filter: Optional[Dict[str, List]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with VDMS with distance.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 3.
fetch_k (int) – Number of candidates to fetch for knn (>= k).
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
kwargs (Any) – 
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
similarity_search_with_score_by_vector(embedding: List[float], k: int = 3, fetch_k: int = 15, filter: Optional[Dict[str, List]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return docs most similar to embedding vector and similarity score.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 3.
fetch_k (int) – Number of candidates to fetch for knn (>= k).
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
kwargs (Any) – 
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
update_document(collection_name: str, document_id: str, document: Document) → None[source]¶
Update a document in the collection.
Parameters
document_id (str) – ID of the document to update.
document (Document) – Document to update.
collection_name (str) – 
Return type
None
update_documents(collection_name: str, ids: List[str], documents: List[Document]) → None[source]¶
Update a document in the collection.
Parameters
ids (List[str]) – List of ids of the document to update.
documents (List[Document]) – List of documents to update.
collection_name (str) – 
Return type
None
Examples using VDMS¶
Intel’s Visual Data Management System (VDMS)
VDMS
langchain_community.vectorstores.neo4j_vector.SearchType¶
class langchain_community.vectorstores.neo4j_vector.SearchType(value)[source]¶
Enumerator of the Distance strategies.
VECTOR = 'vector'¶
HYBRID = 'hybrid'¶
Examples using SearchType¶
Docugami
How to retrieve using multiple vectors per document
Zep Open Source
langchain_community.vectorstores.opensearch_vector_search.OpenSearchVectorSearch¶
class langchain_community.vectorstores.opensearch_vector_search.OpenSearchVectorSearch(opensearch_url: str, index_name: str, embedding_function: Embeddings, **kwargs: Any)[source]¶
Amazon OpenSearch Vector Engine vector store.
Example
from langchain_community.vectorstores import OpenSearchVectorSearch
opensearch_vector_search = OpenSearchVectorSearch(
    "http://localhost:9200",
    "embeddings",
    embedding_function
)
Initialize with necessary components.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(opensearch_url, index_name, ...)
Initialize with necessary components.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas, ids, bulk_size])
Asynchronously run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_embeddings(text_embeddings[, metadatas, ...])
Add the given texts and embeddings to the vectorstore.
add_texts(texts[, metadatas, ids, bulk_size])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Asynchronously delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_embeddings(embeddings, texts, embedding)
Asynchronously construct OpenSearchVectorSearch wrapper from pre-vectorized embeddings.
afrom_texts(texts, embedding[, metadatas, ...])
Asynchronously construct OpenSearchVectorSearch wrapper from raw texts.
Asynchronously construct OpenSearchVectorSearch wrapper from raw texts.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
create_index(dimension[, index_name])
Create a new Index with given arguments
delete([ids, refresh_indices])
Delete documents from the Opensearch index.
delete_index([index_name])
Deletes a given index from vectorstore.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_embeddings(embeddings, texts, embedding)
Construct OpenSearchVectorSearch wrapper from pre-vectorized embeddings.
from_texts(texts, embedding[, metadatas, ...])
Construct OpenSearchVectorSearch wrapper from raw texts.
index_exists([index_name])
If given index present in vectorstore, returns True else False.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, score_threshold])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to the embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Return docs and it's scores most similar to query.
similarity_search_with_score_by_vector(embedding)
Return docs and it's scores most similar to the embedding vector.
Parameters
opensearch_url (str) – 
index_name (str) – 
embedding_function (Embeddings) – 
kwargs (Any) – 
__init__(opensearch_url: str, index_name: str, embedding_function: Embeddings, **kwargs: Any)[source]¶
Initialize with necessary components.
Parameters
opensearch_url (str) – 
index_name (str) – 
embedding_function (Embeddings) – 
kwargs (Any) – 
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, bulk_size: int = 500, **kwargs: Any) → List[str][source]¶
Asynchronously run more texts through the embeddings
and add to the vectorstore.
Parameters
texts (Iterable[str]) – 
metadatas (Optional[List[dict]]) – 
ids (Optional[List[str]]) – 
bulk_size (int) – 
kwargs (Any) – 
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_embeddings(text_embeddings: Iterable[Tuple[str, List[float]]], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, bulk_size: int = 500, **kwargs: Any) → List[str][source]¶
Add the given texts and embeddings to the vectorstore.
Parameters
text_embeddings (Iterable[Tuple[str, List[float]]]) – Iterable pairs of string and embedding to
add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
ids (Optional[List[str]]) – Optional list of ids to associate with the texts.
bulk_size (int) – Bulk API request count; Default: 500
kwargs (Any) – 
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
Optional Args:vector_field: Document field embeddings are stored in. Defaults to
“vector_field”.
text_field: Document field the text of the document is stored in. Defaults
to “text”.
to “text”.
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, bulk_size: int = 500, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
ids (Optional[List[str]]) – Optional list of ids to associate with the texts.
bulk_size (int) – Bulk API request count; Default: 500
kwargs (Any) – 
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
Optional Args:vector_field: Document field embeddings are stored in. Defaults to
“vector_field”.
text_field: Document field the text of the document is stored in. Defaults
to “text”.
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Asynchronously delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
kwargs (Any) – 
Return type
VST
async classmethod afrom_embeddings(embeddings: List[List[float]], texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, bulk_size: int = 500, ids: Optional[List[str]] = None, **kwargs: Any) → OpenSearchVectorSearch[source]¶
Asynchronously construct OpenSearchVectorSearch wrapper from pre-vectorized
embeddings.
Example
from langchain_community.vectorstores import OpenSearchVectorSearch
from langchain_community.embeddings import OpenAIEmbeddings
embedder = OpenAIEmbeddings()
embeddings = await embedder.aembed_documents(["foo", "bar"])
opensearch_vector_search =
    await OpenSearchVectorSearch.afrom_embeddings(
        embeddings,
        texts,
        embedder,
        opensearch_url="http://localhost:9200"
)
OpenSearch by default supports Approximate Search powered by nmslib, faiss
and lucene engines recommended for large datasets. Also supports brute force
search through Script Scoring and Painless Scripting.
Optional Args:vector_field: Document field embeddings are stored in. Defaults to
“vector_field”.
text_field: Document field the text of the document is stored in. Defaults
to “text”.
Optional Keyword Args for Approximate Search:engine: “nmslib”, “faiss”, “lucene”; default: “nmslib”
space_type: “l2”, “l1”, “cosinesimil”, “linf”, “innerproduct”; default: “l2”
ef_search: Size of the dynamic list used during k-NN searches. Higher values
lead to more accurate but slower searches; default: 512
ef_construction: Size of the dynamic list used during k-NN graph creation.
ef_construction: Size of the dynamic list used during k-NN graph creation.
Higher values lead to more accurate graph but slower indexing speed;
default: 512
m: Number of bidirectional links created for each new element. Large impact
on memory consumption. Between 2 and 100; default: 16
Keyword Args for Script Scoring or Painless Scripting:is_appx_search: False
Parameters
embeddings (List[List[float]]) – 
texts (List[str]) – 
embedding (Embeddings) – 
metadatas (Optional[List[dict]]) – 
bulk_size (int) – 
ids (Optional[List[str]]) – 
kwargs (Any) – 
Return type
OpenSearchVectorSearch
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, bulk_size: int = 500, ids: Optional[List[str]] = None, **kwargs: Any) → OpenSearchVectorSearch[source]¶
Asynchronously construct OpenSearchVectorSearch wrapper from raw texts.
Example
from langchain_community.vectorstores import OpenSearchVectorSearch
from langchain_community.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
opensearch_vector_search = await OpenSearchVectorSearch.afrom_texts(
    texts,
    embeddings,
    opensearch_url="http://localhost:9200"
)
OpenSearch by default supports Approximate Search powered by nmslib, faiss
and lucene engines recommended for large datasets. Also supports brute force
search through Script Scoring and Painless Scripting.
Optional Args:vector_field: Document field embeddings are stored in. Defaults to
“vector_field”.
text_field: Document field the text of the document is stored in. Defaults
to “text”.
to “text”.
Optional Keyword Args for Approximate Search:engine: “nmslib”, “faiss”, “lucene”; default: “nmslib”
space_type: “l2”, “l1”, “cosinesimil”, “linf”, “innerproduct”; default: “l2”
ef_search: Size of the dynamic list used during k-NN searches. Higher values
lead to more accurate but slower searches; default: 512
ef_construction: Size of the dynamic list used during k-NN graph creation.
Higher values lead to more accurate graph but slower indexing speed;
default: 512
m: Number of bidirectional links created for each new element. Large impact
on memory consumption. Between 2 and 100; default: 16
Keyword Args for Script Scoring or Painless Scripting:is_appx_search: False
Parameters
texts (List[str]) – 
embedding (Embeddings) – 
metadatas (Optional[List[dict]]) – 
bulk_size (int) – 
ids (Optional[List[str]]) – 
kwargs (Any) – 
Return type
OpenSearchVectorSearch
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
kwargs (Any) – 
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text.
Parameters
query (str) – Input text.
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query.
Return type
List[Document]
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
create_index(dimension: int, index_name: Optional[str] = '0051e0ef3afd48b487409bb7a990c543', **kwargs: Any) → Optional[str][source]¶
Create a new Index with given arguments
Parameters
dimension (int) – 
index_name (Optional[str]) – 
kwargs (Any) – 
Return type
Optional[str]
delete(ids: Optional[List[str]] = None, refresh_indices: Optional[bool] = True, **kwargs: Any) → Optional[bool][source]¶
Delete documents from the Opensearch index.
Parameters
ids (Optional[List[str]]) – List of ids of documents to delete.
refresh_indices (Optional[bool]) – Whether to refresh the index
after deleting documents. Defaults to True.
kwargs (Any) – 
Return type
Optional[bool]
delete_index(index_name: Optional[str] = None) → Optional[bool][source]¶
Deletes a given index from vectorstore.
Parameters
index_name (Optional[str]) – 
Return type
Optional[bool]
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
classmethod from_embeddings(embeddings: List[List[float]], texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, bulk_size: int = 500, ids: Optional[List[str]] = None, **kwargs: Any) → OpenSearchVectorSearch[source]¶
Construct OpenSearchVectorSearch wrapper from pre-vectorized embeddings.
Example
from langchain_community.vectorstores import OpenSearchVectorSearch
from langchain_community.embeddings import OpenAIEmbeddings
embedder = OpenAIEmbeddings()
embeddings = embedder.embed_documents(["foo", "bar"])
opensearch_vector_search = OpenSearchVectorSearch.from_embeddings(
    embeddings,
    texts,
    embedder,
    opensearch_url="http://localhost:9200"
)
OpenSearch by default supports Approximate Search powered by nmslib, faiss
and lucene engines recommended for large datasets. Also supports brute force
search through Script Scoring and Painless Scripting.
Optional Args:vector_field: Document field embeddings are stored in. Defaults to
“vector_field”.
text_field: Document field the text of the document is stored in. Defaults
to “text”.
Optional Keyword Args for Approximate Search:engine: “nmslib”, “faiss”, “lucene”; default: “nmslib”
space_type: “l2”, “l1”, “cosinesimil”, “linf”, “innerproduct”; default: “l2”
ef_search: Size of the dynamic list used during k-NN searches. Higher values
lead to more accurate but slower searches; default: 512