Tony20100/langchain_full huggingface dataset:
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Retrieve {k} contexts with for a given query
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
Parameters
db (Any) – 
__init__(db: Any) → None[source]¶
Parameters
db (Any) – 
Return type
None
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
abstract add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
abstract classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
insert(documents: List[Dict[str, Any]])[source]¶
Inserts documents into the VectorStore and return the corresponding Sources.
Parameters
documents (List[Dict[str, Any]]) – A list of dictionaries that
VectorStores. (represent documents to be inserted to the) – 
format (The document dictionaries must be in the following) – 
{"document_type" – “DOCUMENT_TYPE”, **kwargs} where “DOCUMENT_TYPE”
following (is one of the) – 
"PDF" – 
"CSV" – 
"DOCX" – 
"URL" – 
"SentenceLevelPDF" – 
"SentenceLevelDOCX" – 
:param :
:param “Unstructured”:
:param “InMemoryText”.:
:param The kwargs for each document type are shown below:
:param “InMemoryText”.:
:param The kwargs for each document type are shown below:
:param class PDF: document_type: Literal[“PDF”]
path: str
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
version: str = “v1”
chunk_size: int = 100
stride: int = 40
emphasize_first_words: int = 0
ignore_header_footer: bool = True
ignore_nonstandard_orientation: bool = True
Parameters
CSV (class) – document_type: Literal[“CSV”]
path: str
id_column: Optional[str] = None
strong_columns: Optional[List[str]] = None
weak_columns: Optional[List[str]] = None
reference_columns: Optional[List[str]] = None
save_extra_info: bool = True
metadata: Optional[dict[str, Any]] = None
has_offset: bool = False
on_disk: bool = False
DOCX (class) – document_type: Literal[“DOCX”]
path: str
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
URL (class) – document_type: Literal[“URL”]
url: str
save_extra_info: bool = True
title_is_strong: bool = False
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
SentenceLevelPDF (class) – document_type: Literal[“SentenceLevelPDF”]
path: str
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
SentenceLevelDOCX (class) – document_type: Literal[“SentenceLevelDOCX”]
path: str
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
Unstructured (class) – document_type: Literal[“Unstructured”]
path: str
save_extra_info: bool = True
metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
InMemoryText (class) – document_type: Literal[“InMemoryText”]
name: str
texts: list[str]
metadatas: Optional[list[dict[str, Any]]] = None
global_metadata: Optional[dict[str, Any]] = None
on_disk: bool = False
"path" (For Document types with the arg) – 
that (ensure) – 
machine. (the path exists on your local) – 
documents (List[Dict[str, Any]]) – 
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
remove_documents(source_ids: List[str])[source]¶
Deletes documents from the VectorStore using source ids.
Parameters
files (List[str]) – A list of source ids to delete from the VectorStore.
source_ids (List[str]) – 
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
Return type
List[Document]
similarity_search(query: str, k: int = 10, **kwargs: Any) → List[Document][source]¶
Retrieve {k} contexts with for a given query
Parameters
query (str) – Query to submit to the model
k (int) – The max number of context results to retrieve. Defaults to 10.
kwargs (Any) – 
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
langchain_community.vectorstores.milvus.Milvus¶
class langchain_community.vectorstores.milvus.Milvus(embedding_function: Embeddings, collection_name: str = 'LangChainCollection', collection_description: str = '', collection_properties: Optional[dict[str, Any]] = None, connection_args: Optional[dict[str, Any]] = None, consistency_level: str = 'Session', index_params: Optional[dict] = None, search_params: Optional[dict] = None, drop_old: Optional[bool] = False, auto_id: bool = False, *, primary_field: str = 'pk', text_field: str = 'text', vector_field: str = 'vector', metadata_field: Optional[str] = None, partition_key_field: Optional[str] = None, partition_names: Optional[list] = None, replica_number: int = 1, timeout: Optional[float] = None, num_shards: Optional[int] = None)[source]¶
[Deprecated] Milvus vector store.
You need to install pymilvus and run Milvus.
See the following documentation for how to run a Milvus instance:
https://milvus.io/docs/install_standalone-docker.md
If looking for a hosted Milvus, take a look at this documentation:
https://zilliz.com/cloud and make use of the Zilliz vectorstore found in
this project.
IF USING L2/IP metric, IT IS HIGHLY SUGGESTED TO NORMALIZE YOUR DATA.
Parameters
embedding_function (Embeddings) – Function used to embed the text.
collection_name (str) – Which Milvus collection to use. Defaults to
“LangChainCollection”.
collection_description (str) – The description of the collection. Defaults to
“”.
collection_properties (Optional[dict[str, any]]) – The collection properties.
“”.
collection_properties (Optional[dict[str, any]]) – The collection properties.
Defaults to None.
If set, will override collection existing properties.
For example: {“collection.ttl.seconds”: 60}.
connection_args (Optional[dict[str, any]]) – The connection args used for
this class comes in the form of a dict.
consistency_level (str) – The consistency level to use for a collection.
Defaults to “Session”.
index_params (Optional[dict]) – Which index params to use. Defaults to
HNSW/AUTOINDEX depending on service.
search_params (Optional[dict]) – Which search params to use. Defaults to
default of index.
drop_old (Optional[bool]) – Whether to drop the current collection. Defaults
to False.
auto_id (bool) – Whether to enable auto id for primary key. Defaults to False.
If False, you needs to provide text ids (string less than 65535 bytes).
If True, Milvus will generate unique integers as primary keys.
primary_field (str) – Name of the primary key field. Defaults to “pk”.
text_field (str) – Name of the text field. Defaults to “text”.
vector_field (str) – Name of the vector field. Defaults to “vector”.
metadata_field (str) – Name of the metadata field. Defaults to None.
When metadata_field is specified,
the document’s metadata will store as json.
partition_key_field (Optional[str]) – 
partition_names (Optional[list]) – 
replica_number (int) – 
timeout (Optional[float]) – 
num_shards (Optional[int]) – 
The connection args used for this class comes in the form of a dict,
here are a few of the options:
here are a few of the options:
address (str): The actual address of Milvusinstance. Example address: “localhost:19530”
uri (str): The uri of Milvus instance. Example uri:“http://randomwebsite:19530”,
“tcp:foobarsite:19530”,
“https://ok.s3.south.com:19530”.
host (str): The host of Milvus instance. Default at “localhost”,PyMilvus will fill in the default host if only port is provided.
port (str/int): The port of Milvus instance. Default at 19530, PyMilvuswill fill in the default port if only host is provided.
user (str): Use which user to connect to Milvus instance. If user andpassword are provided, we will add related header in every RPC call.
password (str): Required when user is provided. The passwordcorresponding to the user.
secure (bool): Default is false. If set to true, tls will be enabled.
client_key_path (str): If use tls two-way authentication, need to
write the client.key path.
client_pem_path (str): If use tls two-way authentication, need towrite the client.pem path.
ca_pem_path (str): If use tls two-way authentication, need to writethe ca.pem path.
server_pem_path (str): If use tls one-way authentication, need towrite the server.pem path.
server_name (str): If use tls, need to write the common name.
Example
from langchain_community.vectorstores import Milvus
from langchain_community.embeddings import OpenAIEmbeddings
embedding = OpenAIEmbeddings()
# Connect to a milvus instance on localhost
milvus_store = Milvus(
embedding_function = Embeddings,
milvus_store = Milvus(
embedding_function = Embeddings,
collection_name = “LangChainCollection”,
drop_old = True,
auto_id = True
)
Raises
ValueError – If the pymilvus python package is not installed.
Parameters
embedding_function (Embeddings) – 
collection_name (str) – 
collection_description (str) – 
collection_properties (Optional[dict[str, Any]]) – 
connection_args (Optional[dict[str, Any]]) – 
consistency_level (str) – 
index_params (Optional[dict]) – 
search_params (Optional[dict]) – 
drop_old (Optional[bool]) – 
auto_id (bool) – 
primary_field (str) – 
text_field (str) – 
vector_field (str) – 
metadata_field (Optional[str]) – 
partition_key_field (Optional[str]) – 
partition_names (Optional[list]) – 
replica_number (int) – 
timeout (Optional[float]) – 
num_shards (Optional[int]) – 
Notes
Deprecated since version 0.2.0.
Initialize the Milvus vector store.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(embedding_function[, ...])
Initialize the Milvus vector store.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, timeout, ...])
Insert text data into Milvus.
adelete([ids])
Insert text data into Milvus.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
delete([ids, expr])
Delete by vector ID or boolean expression.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Create a Milvus collection, indexes it with HNSW, and insert data.
get_pks(expr, **kwargs)
Get primary keys with expression
max_marginal_relevance_search(query[, k, ...])
Perform a search and return results that are reordered by MMR.
max_marginal_relevance_search_by_vector(...)
Perform a search and return results that are reordered by MMR.
Perform a search and return results that are reordered by MMR.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, param, expr, ...])
Perform a similarity search against the query string.
similarity_search_by_vector(embedding[, k, ...])
Perform a similarity search against the query string.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Perform a search on a query string and return results with score.
similarity_search_with_score_by_vector(embedding)
Perform a search on a query string and return results with score.
upsert([ids, documents])
Update/Insert documents to the vectorstore.
__init__(embedding_function: Embeddings, collection_name: str = 'LangChainCollection', collection_description: str = '', collection_properties: Optional[dict[str, Any]] = None, connection_args: Optional[dict[str, Any]] = None, consistency_level: str = 'Session', index_params: Optional[dict] = None, search_params: Optional[dict] = None, drop_old: Optional[bool] = False, auto_id: bool = False, *, primary_field: str = 'pk', text_field: str = 'text', vector_field: str = 'vector', metadata_field: Optional[str] = None, partition_key_field: Optional[str] = None, partition_names: Optional[list] = None, replica_number: int = 1, timeout: Optional[float] = None, num_shards: Optional[int] = None)[source]¶
Initialize the Milvus vector store.
Parameters
embedding_function (Embeddings) – 
collection_name (str) –
Parameters
embedding_function (Embeddings) – 
collection_name (str) – 
collection_description (str) – 
collection_properties (Optional[dict[str, Any]]) – 
connection_args (Optional[dict[str, Any]]) – 
consistency_level (str) – 
index_params (Optional[dict]) – 
search_params (Optional[dict]) – 
drop_old (Optional[bool]) – 
auto_id (bool) – 
primary_field (str) – 
text_field (str) – 
vector_field (str) – 
metadata_field (Optional[str]) – 
partition_key_field (Optional[str]) – 
partition_names (Optional[list]) – 
replica_number (int) – 
timeout (Optional[float]) – 
num_shards (Optional[int]) – 
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, timeout: Optional[float] = None, batch_size: int = 1000, *, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Insert text data into Milvus.
Inserting data when the collection has not be made yet will result
in creating a new Collection. The data of the first entity decides
the schema of the new collection, the dim is extracted from the first
embedding and the columns are decided by the first metadata dict.
Metadata keys will need to be present for all inserted values. At
the moment there is no None equivalent in Milvus.
Parameters
texts (Iterable[str]) – The texts to embed, it is assumed
that they all fit in memory.
metadatas (Optional[List[dict]]) – Metadata dicts attached to each of
the texts. Defaults to None.
False. (should be less than 65535 bytes. Required and work when auto_id is) – 
timeout (Optional[float]) – Timeout for each batch insert. Defaults
to None.
batch_size (int, optional) – Batch size to use for insertion.
Defaults to 1000.
ids (Optional[List[str]]) – List of text ids. The length of each item
kwargs (Any) – 
Raises
MilvusException – Failure to add texts
Returns
The resulting keys for each inserted element.
Return type
List[str]