Tony20100/langchain_full huggingface dataset:
Initialize with Lance DB vectorstore
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Turn texts into embedding and add it to the database
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
create_index([col_name, vector_col, ...])
Create a scalar(for non-vector cols) or a vector index on a table.
delete([ids, delete_all, filter, ...])
delete([ids, delete_all, filter, ...])
Allows deleting rows by filtering, by ids or drop columns from the table.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Return VectorStore initialized from texts and embeddings.
get_table([name, set_default])
Fetches a table object from the database.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, name])
Return documents most similar to the query
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
__init__(connection: Optional[Any] = None, embedding: Optional[Embeddings] = None, uri: Optional[str] = '/tmp/lancedb', vector_key: Optional[str] = 'vector', id_key: Optional[str] = 'id', text_key: Optional[str] = 'text', table_name: Optional[str] = 'vectorstore', api_key: Optional[str] = None, region: Optional[str] = None, mode: Optional[str] = 'overwrite')[source]¶
Initialize with Lance DB vectorstore
Parameters
connection (Optional[Any]) – 
embedding (Optional[Embeddings]) – 
uri (Optional[str]) –
embedding (Optional[Embeddings]) – 
uri (Optional[str]) – 
vector_key (Optional[str]) – 
id_key (Optional[str]) – 
text_key (Optional[str]) – 
table_name (Optional[str]) – 
api_key (Optional[str]) – 
region (Optional[str]) – 
mode (Optional[str]) – 
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Turn texts into embedding and add it to the database
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
ids (Optional[List[str]]) – Optional list of ids to associate with the texts.
kwargs (Any) – 
Returns
List of ids of the added texts.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
create_index(col_name: Optional[str] = None, vector_col: Optional[str] = None, num_partitions: Optional[int] = 256, num_sub_vectors: Optional[int] = 96, index_cache_size: Optional[int] = None, metric: Optional[str] = 'L2') → None[source]¶
Create a scalar(for non-vector cols) or a vector index on a table.
Make sure your vector column has enough data before creating an index on it.
Parameters
vector_col (Optional[str]) – Provide if you want to create index on a vector column.
vector_col (Optional[str]) – Provide if you want to create index on a vector column.
col_name (Optional[str]) – Provide if you want to create index on a non-vector column.
metric (Optional[str]) – Provide the metric to use for vector index. Defaults to ‘L2’
choice of metrics: ‘L2’, ‘dot’, ‘cosine’
num_partitions (Optional[int]) – 
num_sub_vectors (Optional[int]) – 
index_cache_size (Optional[int]) – 
Returns
None
Return type
None
delete(ids: Optional[List[str]] = None, delete_all: Optional[bool] = None, filter: Optional[str] = None, drop_columns: Optional[List[str]] = None, name: Optional[str] = None, **kwargs: Any) → None[source]¶
Allows deleting rows by filtering, by ids or drop columns from the table.
Parameters
filter (Optional[str]) – Provide a string SQL expression -  “{col} {operation} {value}”.
ids (Optional[List[str]]) – Provide list of ids to delete from the table.
drop_columns (Optional[List[str]]) – Provide list of columns to drop from the table.
delete_all (Optional[bool]) – If True, delete all rows from the table.
name (Optional[str]) – 
kwargs (Any) – 
Return type
None
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
kwargs (Any) – 
Return type
VST
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, connection: Optional[Any] = None, vector_key: Optional[str] = 'vector', id_key: Optional[str] = 'id', text_key: Optional[str] = 'text', table_name: Optional[str] = 'vectorstore', **kwargs: Any) → LanceDB[source]¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
embedding (Embeddings) – Embedding function to use.
connection (Optional[Any]) – 
vector_key (Optional[str]) – 
id_key (Optional[str]) – 
text_key (Optional[str]) – 
table_name (Optional[str]) – 
kwargs (Any) – 
Return type
LanceDB
get_table(name: Optional[str] = None, set_default: Optional[bool] = False) → Any[source]¶
Fetches a table object from the database.
Parameters
name (str, optional) – The name of the table to fetch. Defaults to None
and fetches current table object.
set_default (bool, optional) – Sets fetched table as the default table.
Defaults to False.
Returns
The fetched table object.
Return type
Any
Raises
ValueError – If the specified table is not found in the database.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 4, name: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return documents most similar to the query
Parameters
query (str) – String to query the vectorstore with.
k (int) – Number of documents to return.
filter (Optional[Dict]) – Optional filter arguments
sql_filter(Optional[string]): SQL filter to apply to the query.
prefilter(Optional[bool]): Whether to apply the filter prior
to the vector search.
name (Optional[str]) – 
kwargs (Any) – 
Raises
ValueError – If the specified table is not found in the database.
Returns
List of documents most similar to the query.
Return type
List[Document]
Examples:
# Retrieve documents with filtering based on a metadata file_type
vector_store.as_retriever(search_kwargs={"k": 4, "filter":{
                                            'sql_filter':"file_type='notice'",
                                             'prefilter': True
                                             }
                                         })
# Retrieve documents with filtering on a specific file name
vector_store.as_retriever(search_kwargs={"k": 4, "filter":{
                                             'sql_filter':"source='my-file.txt'",
                                             'prefilter': True
                                             }
                                        })
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
Examples using LanceDB¶
How to create and query vector stores
LanceDB
langchain_community.vectorstores.pgvector.DistanceStrategy¶
class langchain_community.vectorstores.pgvector.DistanceStrategy(value)[source]¶
Enumerator of the Distance strategies.
EUCLIDEAN = 'l2'¶
COSINE = 'cosine'¶
MAX_INNER_PRODUCT = 'inner'¶
Examples using DistanceStrategy¶
Google BigQuery Vector Search
Kinetica Vectorstore API
Oracle AI Vector Search: Vector Store
SAP HANA Cloud Vector Engine
SemaDB
SingleStoreDB
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
delete([ids, where_str])
Delete by vector ID or other criteria.
drop()
Helper function: Drop data
escape_str(value)
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Create Myscale wrapper with existing texts
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, where_str])
Perform a similarity search with MyScale
similarity_search_by_vector(embedding[, k, ...])
Perform a similarity search with MyScale by vectors
similarity_search_with_relevance_scores(query)
Perform a similarity search with MyScale
similarity_search_with_relevance_scores(query)
Perform a similarity search with MyScale
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
Parameters
embedding (Embeddings) – 
config (Optional[MyScaleSettings]) – 
kwargs (Any) – 
__init__(embedding: Embeddings, config: Optional[MyScaleSettings] = None, **kwargs: Any) → None[source]¶
MyScale Wrapper to LangChain
embedding (Embeddings):
config (MyScaleSettings): Configuration to MyScale Client
Other keyword arguments will pass into
[clickhouse-connect](https://docs.myscale.com/)
Parameters
embedding (Embeddings) – 
config (Optional[MyScaleSettings]) – 
kwargs (Any) – 
Return type
None
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, batch_size: int = 32, ids: Optional[Iterable[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
ids (Optional[Iterable[str]]) – Optional list of ids to associate with the texts.
batch_size (int) – Batch size of insertion
metadata – Optional column data to be inserted
metadatas (Optional[List[dict]]) – 
kwargs (Any) – 
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
delete(ids: Optional[List[str]] = None, where_str: Optional[str] = None, **kwargs: Any) → Optional[bool][source]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
where_str (Optional[str]) – 
**kwargs – 
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
drop() → None[source]¶
Helper function: Drop data
Return type
None
escape_str(value: str) → str[source]¶
Parameters
value (str) – 
Return type
str
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
classmethod from_texts(texts: Iterable[str], embedding: Embeddings, metadatas: Optional[List[Dict[Any, Any]]] = None, config: Optional[MyScaleSettings] = None, text_ids: Optional[Iterable[str]] = None, batch_size: int = 32, **kwargs: Any) → MyScale[source]¶
Create Myscale wrapper with existing texts
Parameters
texts (Iterable[str]) – List or tuple of strings to be added
embedding (Embeddings) – Function to extract text embedding
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Perform a similarity search with MyScale
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
kwargs (Any) – 
Returns
List of documents most similar to the query text
and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Document]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
Examples using MyScale¶
MyScale
langchain_community.vectorstores.pgvector.PGVector¶
class langchain_community.vectorstores.pgvector.PGVector(connection_string: str, embedding_function: Embeddings, embedding_length: Optional[int] = None, collection_name: str = 'langchain', collection_metadata: Optional[dict] = None, distance_strategy: DistanceStrategy = DistanceStrategy.COSINE, pre_delete_collection: bool = False, logger: Optional[Logger] = None, relevance_score_fn: Optional[Callable[[float], float]] = None, *, connection: Optional[Connection] = None, engine_args: Optional[dict[str, Any]] = None, use_jsonb: bool = False, create_extension: bool = True)[source]¶
[Deprecated] Postgres/PGVector vector store.
DEPRECATED: This class is pending deprecation and will likely receiveno updates. An improved version of this class is available in
langchain_postgres as PGVector. Please use that class instead.
When migrating please keep in mind that:
The new implementation works with psycopg3, not with psycopg2
(This implementation does not work with psycopg3).
Filtering syntax has changed to use $ prefixed operators for JSONB
metadata fields. (New implementation only uses JSONB field for metadata)
The new implementation made some schema changes to address issues
with the existing implementation. So you will need to re-create
your tables and re-index your data or else carry out a manual
migration.
To use, you should have the pgvector python package installed.
Parameters
connection_string (str) – Postgres connection string.
embedding_function (Embeddings) – Any embedding function implementing
langchain.embeddings.base.Embeddings interface.
embedding_length (Optional[int]) – The length of the embedding vector. (default: None)
NOTE: This is not mandatory. Defining it will prevent vectors of