Tony20100/langchain_full huggingface dataset:
Delete by vector IDs.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
kwargs (Any) – 
Return type
None
delete_collection() → None[source]¶
Delete the collection.
Return type
None
encode_image(uri: str) → str[source]¶
Get base64 string from image URI.
Parameters
uri (str) – 
Return type
str
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a list of documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
documents (List[Document]) – List of documents to add to the vectorstore.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
client (Optional[chromadb.Client]) – 
kwargs (Any) – 
Returns
Chroma vectorstore.
Return type
Chroma
Returns
Chroma vectorstore.
Return type
Chroma
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a raw documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters
texts (List[str]) – List of texts to add to the collection.
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
client (Optional[chromadb.Client]) – 
kwargs (Any) – 
Returns
Chroma vectorstore.
Return type
Chroma
get(ids: Optional[OneOrMany[ID]] = None, where: Optional[Where] = None, limit: Optional[int] = None, offset: Optional[int] = None, where_document: Optional[WhereDocument] = None, include: Optional[List[str]] = None) → Dict[str, Any][source]¶
Gets the collection.
Parameters
Gets the collection.
Parameters
ids (Optional[OneOrMany[ID]]) – The ids of the embeddings to get. Optional.
where (Optional[Where]) – A Where type dict used to filter results by.
E.g. {“color” : “red”, “price”: 4.20}. Optional.
limit (Optional[int]) – The number of documents to return. Optional.
offset (Optional[int]) – The offset to start returning results from.
Useful for paging results with limit. Optional.
where_document (Optional[WhereDocument]) – A WhereDocument type dict used to filter by the documents.
E.g. {$contains: “hello”}. Optional.
include (Optional[List[str]]) – A list of what to include in the results.
Can contain “embeddings”, “metadatas”, “documents”.
Ids are always included.
Defaults to [“metadatas”, “documents”]. Optional.
Return type
Dict[str, Any]
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
where_document (Optional[Dict[str, str]]) – 
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
where_document (Optional[Dict[str, str]]) – 
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
persist() → None[source]¶
[Deprecated] Persist the collection.
This can be used to explicitly persist the data to disk.
It will also be called automatically when the object is destroyed.
It will also be called automatically when the object is destroyed.
Since Chroma 0.4.x the manual persistence method is no longer
supported as docs are automatically persisted.
Notes
Deprecated since version langchain-community==0.1.17: Since Chroma 0.4.x the manual persistence method is no longer supported as docs are automatically persisted.
Return type
None
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Run similarity search with Chroma.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
kwargs (Any) – 
Returns
List of documents most similar to the query text.
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
:param embedding: Embedding to look up documents similar to.
:type embedding: List[float]
:param embedding: Embedding to look up documents similar to.
:type embedding: List[float]
:param k: Number of Documents to return. Defaults to 4.
:type k: int
:param filter: Filter by metadata. Defaults to None.
:type filter: Optional[Dict[str, str]]
Returns
List of Documents most similar to the query vector.
Parameters
embedding (List[float]) – 
k (int) – 
filter (Optional[Dict[str, str]]) – 
where_document (Optional[Dict[str, str]]) – 
kwargs (Any) – 
Return type
List[Document]
similarity_search_by_vector_with_relevance_scores(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return docs most similar to embedding vector and similarity score.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
where_document (Optional[Dict[str, str]]) – 
kwargs (Any) – 
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with Chroma with distance.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
where_document (Optional[Dict[str, str]]) – 
kwargs (Any) – 
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
update_document(document_id: str, document: Document) → None[source]¶
Update a document in the collection.
Parameters
document_id (str) – ID of the document to update.
document (Document) – Document to update.
Return type
None
update_documents(ids: List[str], documents: List[Document]) → None[source]¶
Update a document in the collection.
Parameters
ids (List[str]) – List of ids of the document to update.
documents (List[Document]) – List of documents to update.
Return type
None
documents (List[Document]) – List of documents to update.
Return type
None
Examples using Chroma¶
Build a Local RAG Application
Build a Query Analysis System
Build a Retrieval Augmented Generation (RAG) App
Chroma
Confident
Conversational RAG
Docugami
Google Cloud Vertex AI Reranker
How deal with high cardinality categoricals when doing query analysis
How to add chat history
How to add retrieval to chatbots
How to create and query vector stores
How to do “self-querying” retrieval
How to get your RAG application to return sources
How to handle cases where no queries are generated
How to handle multiple queries when doing query analysis
How to handle multiple retrievers when doing query analysis
How to reorder retrieved results to mitigate the “lost in the middle” effect
How to retrieve using multiple vectors per document
How to select examples by similarity
How to stream results from your RAG application
How to use few shot examples
How to use few shot examples in chat models
How to use the MultiQueryRetriever
How to use the Parent Document Retriever
LOTR (Merger Retriever)
Psychic
RePhraseQuery
Vector stores and retrievers
langchain_community.vectorstores.singlestoredb.SingleStoreDB¶
class langchain_community.vectorstores.singlestoredb.SingleStoreDB(embedding: Embeddings, *, distance_strategy: DistanceStrategy = DistanceStrategy.DOT_PRODUCT, table_name: str = 'embeddings', content_field: str = 'content', metadata_field: str = 'metadata', vector_field: str = 'vector', id_field: str = 'id', use_vector_index: bool = False, vector_index_name: str = '', vector_index_options: Optional[dict] = None, vector_size: int = 1536, use_full_text_search: bool = False, pool_size: int = 5, max_overflow: int = 10, timeout: float = 30, **kwargs: Any)[source]¶
SingleStore DB vector store.
The prerequisite for using this class is the installation of the singlestoredb
Python package.
The SingleStoreDB vectorstore can be created by providing an embedding function and
the relevant parameters for the database connection, connection pool, and
optionally, the names of the table and the fields to use.
Initialize with necessary components.
Parameters
embedding (Embeddings) – A text embedding model.
distance_strategy (DistanceStrategy, optional) – Determines the strategy employed for calculating
the distance between vectors in the embedding space.
Defaults to DOT_PRODUCT.
Available options are:
- DOT_PRODUCT: Computes the scalar product of two vectors.
This is the default behavior
EUCLIDEAN_DISTANCE: Computes the Euclidean distance betweentwo vectors. This metric considers the geometric distance in
the vector space, and might be more suitable for embeddings
that rely on spatial relationships. This metric is not
compatible with the WEIGHTED_SUM search strategy.
table_name (str, optional) – Specifies the name of the table in use.
Defaults to “embeddings”.
Defaults to “embeddings”.
content_field (str, optional) – Specifies the field to store the content.
Defaults to “content”.
metadata_field (str, optional) – Specifies the field to store metadata.
Defaults to “metadata”.
vector_field (str, optional) – Specifies the field to store the vector.
Defaults to “vector”.
id_field (str, optional) – Specifies the field to store the id.
Defaults to “id”.
use_vector_index (bool, optional) – Toggles the use of a vector index.
Works only with SingleStoreDB 8.5 or later. Defaults to False.
If set to True, vector_size parameter is required to be set to
a proper value.
vector_index_name (str, optional) – Specifies the name of the vector index.
Defaults to empty. Will be ignored if use_vector_index is set to False.
vector_index_options (dict, optional) – Specifies the options for
the vector index. Defaults to {}.
Will be ignored if use_vector_index is set to False. The options are:
index_type (str, optional): Specifies the type of the index.
Defaults to IVF_PQFS.
For more options, please refer to the SingleStoreDB documentation:
https://docs.singlestore.com/cloud/reference/sql-reference/vector-functions/vector-indexing/
vector_size (int, optional) – Specifies the size of the vector.
Defaults to 1536. Required if use_vector_index is set to True.
Should be set to the same value as the size of the vectors
stored in the vector_field.
use_full_text_search (bool, optional) – Toggles the use a full-text index
on the document content. Defaults to False. If set to True, the table
will be created with a full-text index on the content field,
will be created with a full-text index on the content field,
and the simularity_search method will all using TEXT_ONLY,
FILTER_BY_TEXT, FILTER_BY_VECTOR, and WIGHTED_SUM search strategies.
If set to False, the simularity_search method will only allow
VECTOR_ONLY search strategy.
pool (Following arguments pertain to the connection) – 
pool_size (int, optional) – Determines the number of active connections in
the pool. Defaults to 5.
max_overflow (int, optional) – Determines the maximum number of connections
allowed beyond the pool_size. Defaults to 10.
timeout (float, optional) – Specifies the maximum wait time in seconds for
establishing a connection. Defaults to 30.
connection (database) – 
host (str, optional) – Specifies the hostname, IP address, or URL for the
database connection. The default scheme is “mysql”.
user (str, optional) – Database username.
password (str, optional) – Database password.
port (int, optional) – Database port. Defaults to 3306 for non-HTTP
connections, 80 for HTTP connections, and 443 for HTTPS connections.
database (str, optional) – Database name.
the (Additional optional arguments provide further customization over) – 
connection – 
pure_python (bool, optional) – Toggles the connector mode. If True,
operates in pure Python mode.
local_infile (bool, optional) – Allows local file uploads.
charset (str, optional) – Specifies the character set for string values.
ssl_key (str, optional) – Specifies the path of the file containing the SSL
key.
ssl_cert (str, optional) – Specifies the path of the file containing the SSL
certificate.
ssl_ca (str, optional) – Specifies the path of the file containing the SSL
certificate authority.
certificate authority.
ssl_cipher (str, optional) – Sets the SSL cipher list.
ssl_disabled (bool, optional) – Disables SSL usage.
ssl_verify_cert (bool, optional) – Verifies the server’s certificate.
Automatically enabled if ssl_ca is specified.
ssl_verify_identity (bool, optional) – Verifies the server’s identity.
conv (dict[int, Callable], optional) – A dictionary of data conversion
functions.
credential_type (str, optional) – Specifies the type of authentication to
use: auth.PASSWORD, auth.JWT, or auth.BROWSER_SSO.
autocommit (bool, optional) – Enables autocommits.
results_type (str, optional) – Determines the structure of the query results:
tuples, namedtuples, dicts.
results_format (str, optional) – Deprecated. This option has been renamed to
results_type.
kwargs (Any) – 
Examples
Basic Usage:
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import SingleStoreDB
vectorstore = SingleStoreDB(
    OpenAIEmbeddings(),
    host="https://user:password@127.0.0.1:3306/database"
)
Advanced Usage:
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import SingleStoreDB
vectorstore = SingleStoreDB(
    OpenAIEmbeddings(),
    distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,
    host="127.0.0.1",
    port=3306,
    user="user",
    password="password",
    database="db",
    table_name="my_custom_table",
    pool_size=10,
    timeout=60,
)
Using environment variables:
from langchain_openai import OpenAIEmbeddings
)
Using environment variables:
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import SingleStoreDB
os.environ['SINGLESTOREDB_URL'] = 'me:p455w0rd@s2-host.com/my_db'
vectorstore = SingleStoreDB(OpenAIEmbeddings())
Using vector index:
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import SingleStoreDB
os.environ['SINGLESTOREDB_URL'] = 'me:p455w0rd@s2-host.com/my_db'
vectorstore = SingleStoreDB(
    OpenAIEmbeddings(),
    use_vector_index=True,
)
Using full-text index:
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(embedding, *[, distance_strategy, ...])
Initialize with necessary components.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_images(uris[, metadatas, embeddings])
Run images through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, embeddings])
Add more texts to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
delete([ids])
Delete by vector ID or other criteria.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Create a SingleStoreDB vectorstore from raw documents. This is a user-friendly interface that:     1. Embeds documents. 2. Creates a new table for the embeddings in SingleStoreDB. 3. Adds the documents to the newly created table. This is intended to be a quick way to get started. :param texts: List of texts to add to the vectorstore. :type texts: List[str] :param embedding: A text embedding model. :type embedding: Embeddings :param metadatas: Optional list of metadatas. Defaults to None. :type metadatas: Optional[List[dict]], optional :param distance_strategy: Determines the strategy employed for calculating                           the distance between vectors in the embedding space. Defaults to DOT_PRODUCT. Available options are:                           - DOT_PRODUCT: Computes the scalar product of two vectors. This is the default behavior                           - EUCLIDEAN_DISTANCE: Computes the Euclidean distance between                               two vectors. This metric considers the geometric distance in                               the vector space, and might be more suitable for embeddings                               that rely on spatial relationships. This metric is not
that rely on spatial relationships. This metric is not                               compatible with the WEIGHTED_SUM search strategy. :type distance_strategy: DistanceStrategy, optional :param table_name: Specifies the name of the table in use. Defaults to "embeddings". :type table_name: str, optional :param content_field: Specifies the field to store the content. Defaults to "content". :type content_field: str, optional :param metadata_field: Specifies the field to store metadata. Defaults to "metadata". :type metadata_field: str, optional :param vector_field: Specifies the field to store the vector. Defaults to "vector". :type vector_field: str, optional :param id_field: Specifies the field to store the id. Defaults to "id". :type id_field: str, optional :param use_vector_index: Toggles the use of a vector index. Works only with SingleStoreDB 8.5 or later. Defaults to False. If set to True, vector_size parameter is required to be set to                          a proper value. :type use_vector_index: bool, optional :param vector_index_name: Specifies the name of the vector index. Defaults to empty. Will be ignored if use_vector_index is set to False. :type vector_index_name: str, optional :param vector_index_options: Specifies the options for                              the vector index. Defaults to {}. Will be ignored if use_vector_index is set to False. The options are:
index_type (str, optional): Specifies the type of the index. Defaults to IVF_PQFS. For more options, please refer to the SingleStoreDB documentation:                              https://docs.singlestore.com/cloud/reference/sql-reference/vector-functions/vector-indexing/ :type vector_index_options: dict, optional :param vector_size: Specifies the size of the vector. Defaults to 1536. Required if use_vector_index is set to True. Should be set to the same value as the size of the vectors                     stored in the vector_field. :type vector_size: int, optional :param use_full_text_search: Toggles the use a full-text index                              on the document content. Defaults to False. If set to True, the table                              will be created with a full-text index on the content field,                              and the simularity_search method will all using TEXT_ONLY,                              FILTER_BY_TEXT, FILTER_BY_VECTOR, and WIGHTED_SUM search strategies. If set to False, the simularity_search method will only allow