Tony20100/langchain_full huggingface dataset:
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_components(project_id: str, region: str, gcs_bucket_name: str, index_id: str, endpoint_id: str, private_service_connect_ip_address: Optional[str] = None, credentials_path: Optional[str] = None, embedding: Optional[Embeddings] = None, stream_update: bool = False, **kwargs: Any) → VectorSearchVectorStore[source]¶
Takes the object creation out of the constructor.
:param project_id: The GCP project id.
:param region: The default location making the API calls. It must have
:param the same location as the GCS bucket and must be regional.:
Return docs most similar to query and their cosine distance from the query.
:param query: String query look up documents similar to.
:param k: Number of Documents to return. Defaults to 4.
:param filter: Optional. A list of Namespaces for filtering
the matching results.
For example:
[Namespace(“color”, [“red”], []), Namespace(“shape”, [], [“squared”])]
will match datapoints that satisfy “red color” but not include
datapoints with “squared shape”. Please refer to
https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#json
for more detail.
Parameters
numeric_filter (Optional[List[NumericNamespace]]) – Optional. A list of NumericNamespaces for filterning
the matching results. Please refer to
https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#json
for more detail.
query (str) – 
k (int) – 
filter (Optional[List[Namespace]]) – 
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
Examples using VectorSearchVectorStore¶
Google
Google Vertex AI Vector Search
langchain_astradb.vectorstores.AstraDBVectorStore¶
class langchain_astradb.vectorstores.AstraDBVectorStore(*, collection_name: str, embedding: Optional[Embeddings] = None, token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[AstraDB] = None, async_astra_db_client: Optional[AsyncAstraDB] = None, namespace: Optional[str] = None, metric: Optional[str] = None, batch_size: Optional[int] = None, bulk_insert_batch_concurrency: Optional[int] = None, bulk_insert_overwrite_concurrency: Optional[int] = None, bulk_delete_concurrency: Optional[int] = None, setup_mode: SetupMode = SetupMode.SYNC, pre_delete_collection: bool = False, metadata_indexing_include: Optional[Iterable[str]] = None, metadata_indexing_exclude: Optional[Iterable[str]] = None, collection_indexing_policy: Optional[Dict[str, Any]] = None, collection_vector_service_options: Optional[CollectionVectorServiceOptions] = None, collection_embedding_api_key: Optional[str] = None)[source]¶
Wrapper around DataStax Astra DB for vector-store workloads.
For quickstart and details, visit
https://docs.datastax.com/en/astra/astra-db-vector/
Example
from langchain_astradb.vectorstores import AstraDBVectorStore
from langchain_openai.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = AstraDBVectorStore(
    embedding=embeddings,
    collection_name="my_store",
    token="AstraCS:...",
    api_endpoint="https://<DB-ID>-<REGION>.apps.astra.datastax.com"
)
vectorstore.add_texts(["Giraffes", "All good here"])
)
vectorstore.add_texts(["Giraffes", "All good here"])
results = vectorstore.similarity_search("Everything's ok", k=1)
Parameters
embedding (Optional[Embeddings]) – the embeddings function or service to use.
This enables client-side embedding functions or calls to external
embedding providers. If embedding is provided, arguments
collection_vector_service_options and
collection_embedding_api_key cannot be provided.
collection_name (str) – name of the Astra DB collection to create/use.
token (Optional[str]) – API token for Astra DB usage. If not provided, the environment
variable ASTRA_DB_APPLICATION_TOKEN is inspected.
api_endpoint (Optional[str]) – full URL to the API endpoint, such as
https://<DB-ID>-us-east1.apps.astra.datastax.com. If not provided,
the environment variable ASTRA_DB_API_ENDPOINT is inspected.
astra_db_client (Optional[AstraDBClient]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AstraDB’ instance.
async_astra_db_client (Optional[AsyncAstraDBClient]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AsyncAstraDB’ instance.
namespace (Optional[str]) – namespace (aka keyspace) where the collection is created.
If not provided, the environment variable ASTRA_DB_KEYSPACE is
inspected. Defaults to the database’s “default namespace”.
metric (Optional[str]) – similarity function to use out of those available in Astra DB.
If left out, it will use Astra DB API’s defaults (i.e. “cosine” - but,
for performance reasons, “dot_product” is suggested if embeddings are
normalized to one).
batch_size (Optional[int]) – Size of batches for bulk insertions.
batch_size (Optional[int]) – Size of batches for bulk insertions.
bulk_insert_batch_concurrency (Optional[int]) – Number of threads or coroutines to insert
batches concurrently.
bulk_insert_overwrite_concurrency (Optional[int]) – Number of threads  or coroutines in a
batch to insert pre-existing entries.
bulk_delete_concurrency (Optional[int]) – Number of threads (for deleting multiple rows
concurrently).
pre_delete_collection (bool) – whether to delete the collection before creating it.
If False and the collection already exists, the collection will be used
as is.
metadata_indexing_include (Optional[Iterable[str]]) – an allowlist of the specific metadata subfields
that should be indexed for later filtering in searches.
metadata_indexing_exclude (Optional[Iterable[str]]) – a denylist of the specific metadata subfields
that should not be indexed for later filtering in searches.
collection_indexing_policy (Optional[Dict[str, Any]]) – a full “indexing” specification for
what fields should be indexed for later filtering in searches.
This dict must conform to to the API specifications
(see docs.datastax.com/en/astra/astra-db-vector/api-reference/
data-api-commands.html#advanced-feature-indexing-clause-on-createcollection)
collection_vector_service_options (Optional[CollectionVectorServiceOptions]) – specifies the use of server-side
embeddings within Astra DB. If passing this parameter, embedding
cannot be provided.
collection_embedding_api_key (Optional[str]) – for usage of server-side embeddings
within Astra DB, with this parameter one can supply an API Key
that will be passed to Astra DB with each data request.
This is useful when the service is configured for the collection,
but no corresponding secret is stored within
Astra’s key management system.
This parameter cannot be provided without
specifying collection_vector_service_options.
This parameter cannot be provided without
specifying collection_vector_service_options.
setup_mode (SetupMode) – 
Note
For concurrency in synchronous add_texts():, as a rule of thumb, on a
typical client machine it is suggested to keep the quantity
bulk_insert_batch_concurrency * bulk_insert_overwrite_concurrency
much below 1000 to avoid exhausting the client multithreading/networking
resources. The hardcoded defaults are somewhat conservative to meet
most machines’ specs, but a sensible choice to test may be:
bulk_insert_batch_concurrency = 80
bulk_insert_overwrite_concurrency = 10
A bit of experimentation is required to nail the best results here,
depending on both the machine/network specs and the expected workload
(specifically, how often a write is an update of an existing id).
Remember you can pass concurrency settings to individual calls to
add_texts() and add_documents() as well.
Attributes
embeddings
Accesses the supplied embeddings object.
Methods
__init__(*, collection_name[, embedding, ...])
Wrapper around DataStax Astra DB for vector-store workloads.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas, ids, ...])
Run texts through the embeddings and add them to the vectorstore.
aclear()
Empty the collection of all its stored entries.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, ...])
Run texts through the embeddings and add them to the vectorstore.
adelete([ids, concurrency])
Delete by vector ids.
adelete_by_document_id(document_id)
Delete by vector ids.
adelete_by_document_id(document_id)
Remove a single document from the store, given its document ID.
adelete_collection()
Completely delete the collection from the database (as opposed to aclear(), which empties it only).
afrom_documents(documents[, embedding, ...])
Create an Astra DB vectorstore from a document list.
afrom_texts(texts[, embedding, metadatas, ...])
Create an Astra DB vectorstore from raw texts.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k, filter])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(query[, k, filter])
Return docs most similar to query with score.
asimilarity_search_with_score_by_vector(...)
Return docs most similar to embedding vector with score.
asimilarity_search_with_score_id(query[, k, ...])
Return docs most similar to the query with score and id.
asimilarity_search_with_score_id_by_vector(...)
Return docs most similar to embedding vector with score and id.
clear()
Empty the collection of all its stored entries.
delete([ids, concurrency])
Empty the collection of all its stored entries.
delete([ids, concurrency])
Delete by vector ids.
delete_by_document_id(document_id)
Remove a single document from the store, given its document ID.
delete_collection()
Completely delete the collection from the database (as opposed to clear(), which empties it only).
from_documents(documents[, embedding, ...])
Create an Astra DB vectorstore from a document list.
from_texts(texts[, embedding, metadatas, ...])
Create an Astra DB vectorstore from raw texts.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, filter])
Return docs most similar to query with score.
similarity_search_with_score_by_vector(embedding)
Return docs most similar to embedding vector with score.
similarity_search_with_score_id(query[, k, ...])
Return docs most similar to the query with score and id.
similarity_search_with_score_id_by_vector(...)
Return docs most similar to embedding vector with score and id.
Return docs most similar to embedding vector with score and id.
__init__(*, collection_name: str, embedding: Optional[Embeddings] = None, token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[AstraDB] = None, async_astra_db_client: Optional[AsyncAstraDB] = None, namespace: Optional[str] = None, metric: Optional[str] = None, batch_size: Optional[int] = None, bulk_insert_batch_concurrency: Optional[int] = None, bulk_insert_overwrite_concurrency: Optional[int] = None, bulk_delete_concurrency: Optional[int] = None, setup_mode: SetupMode = SetupMode.SYNC, pre_delete_collection: bool = False, metadata_indexing_include: Optional[Iterable[str]] = None, metadata_indexing_exclude: Optional[Iterable[str]] = None, collection_indexing_policy: Optional[Dict[str, Any]] = None, collection_vector_service_options: Optional[CollectionVectorServiceOptions] = None, collection_embedding_api_key: Optional[str] = None) → None[source]¶
Wrapper around DataStax Astra DB for vector-store workloads.
For quickstart and details, visit
https://docs.datastax.com/en/astra/astra-db-vector/
Example
from langchain_astradb.vectorstores import AstraDBVectorStore
from langchain_openai.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = AstraDBVectorStore(
    embedding=embeddings,
    collection_name="my_store",
    token="AstraCS:...",
    api_endpoint="https://<DB-ID>-<REGION>.apps.astra.datastax.com"
)
vectorstore.add_texts(["Giraffes", "All good here"])
results = vectorstore.similarity_search("Everything's ok", k=1)
results = vectorstore.similarity_search("Everything's ok", k=1)
Parameters
embedding (Optional[Embeddings]) – the embeddings function or service to use.
This enables client-side embedding functions or calls to external
embedding providers. If embedding is provided, arguments
collection_vector_service_options and
collection_embedding_api_key cannot be provided.
collection_name (str) – name of the Astra DB collection to create/use.
token (Optional[str]) – API token for Astra DB usage. If not provided, the environment
variable ASTRA_DB_APPLICATION_TOKEN is inspected.
api_endpoint (Optional[str]) – full URL to the API endpoint, such as
https://<DB-ID>-us-east1.apps.astra.datastax.com. If not provided,
the environment variable ASTRA_DB_API_ENDPOINT is inspected.
astra_db_client (Optional[AstraDB]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AstraDB’ instance.
async_astra_db_client (Optional[AsyncAstraDB]) – alternative to token+api_endpoint,
you can pass an already-created ‘astrapy.db.AsyncAstraDB’ instance.
namespace (Optional[str]) – namespace (aka keyspace) where the collection is created.
If not provided, the environment variable ASTRA_DB_KEYSPACE is
inspected. Defaults to the database’s “default namespace”.
metric (Optional[str]) – similarity function to use out of those available in Astra DB.
If left out, it will use Astra DB API’s defaults (i.e. “cosine” - but,
for performance reasons, “dot_product” is suggested if embeddings are
normalized to one).
batch_size (Optional[int]) – Size of batches for bulk insertions.
bulk_insert_batch_concurrency (Optional[int]) – Number of threads or coroutines to insert
batches concurrently.
batches concurrently.
bulk_insert_overwrite_concurrency (Optional[int]) – Number of threads  or coroutines in a
batch to insert pre-existing entries.
bulk_delete_concurrency (Optional[int]) – Number of threads (for deleting multiple rows
concurrently).
pre_delete_collection (bool) – whether to delete the collection before creating it.
If False and the collection already exists, the collection will be used
as is.
metadata_indexing_include (Optional[Iterable[str]]) – an allowlist of the specific metadata subfields
that should be indexed for later filtering in searches.
metadata_indexing_exclude (Optional[Iterable[str]]) – a denylist of the specific metadata subfields
that should not be indexed for later filtering in searches.
collection_indexing_policy (Optional[Dict[str, Any]]) – a full “indexing” specification for
what fields should be indexed for later filtering in searches.
This dict must conform to to the API specifications
(see docs.datastax.com/en/astra/astra-db-vector/api-reference/
data-api-commands.html#advanced-feature-indexing-clause-on-createcollection)
collection_vector_service_options (Optional[CollectionVectorServiceOptions]) – specifies the use of server-side
embeddings within Astra DB. If passing this parameter, embedding
cannot be provided.
collection_embedding_api_key (Optional[str]) – for usage of server-side embeddings
within Astra DB, with this parameter one can supply an API Key
that will be passed to Astra DB with each data request.
This is useful when the service is configured for the collection,
but no corresponding secret is stored within
Astra’s key management system.
This parameter cannot be provided without
specifying collection_vector_service_options.
setup_mode (SetupMode) – 
Return type
None
Note
For concurrency in synchronous add_texts():, as a rule of thumb, on a
Note
For concurrency in synchronous add_texts():, as a rule of thumb, on a
typical client machine it is suggested to keep the quantity
bulk_insert_batch_concurrency * bulk_insert_overwrite_concurrency
much below 1000 to avoid exhausting the client multithreading/networking
resources. The hardcoded defaults are somewhat conservative to meet
most machines’ specs, but a sensible choice to test may be:
bulk_insert_batch_concurrency = 80
bulk_insert_overwrite_concurrency = 10
A bit of experimentation is required to nail the best results here,
depending on both the machine/network specs and the expected workload
(specifically, how often a write is an update of an existing id).
Remember you can pass concurrency settings to individual calls to
add_texts() and add_documents() as well.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, *, batch_size: Optional[int] = None, batch_concurrency: Optional[int] = None, overwrite_concurrency: Optional[int] = None, **kwargs: Any) → List[str][source]¶
Run texts through the embeddings and add them to the vectorstore.
If passing explicit ids, those entries whose id is in the store already
will be replaced.
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas.
metadatas (Optional[List[dict]]) – Optional list of metadatas.
ids (Optional[List[str]]) – Optional list of ids.
batch_size (Optional[int]) – Number of documents in each API call.
Check the underlying Astra DB HTTP API specs for the max value
(20 at the time of writing this). If not provided, defaults
to the instance-level setting.
batch_concurrency (Optional[int]) – number of threads to process
insertion batches concurrently. Defaults to instance-level
setting if not provided.
overwrite_concurrency (Optional[int]) – number of threads to process
pre-existing documents in each batch (which require individual
API calls). Defaults to instance-level setting if not provided.
kwargs (Any) – 
Return type
List[str]
Note
There are constraints on the allowed field names
in the metadata dictionaries, coming from the underlying Astra DB API.
For instance, the $ (dollar sign) cannot be used in the dict keys.
See this document for details:
https://docs.datastax.com/en/astra/astra-db-vector/api-reference/data-api.html
Returns
The list of ids of the added texts.
Parameters
texts (Iterable[str]) – 
metadatas (Optional[List[dict]]) – 
ids (Optional[List[str]]) – 
batch_size (Optional[int]) – 
batch_concurrency (Optional[int]) – 
overwrite_concurrency (Optional[int]) – 
kwargs (Any) – 
Return type
List[str]
async aclear() → None[source]¶
Empty the collection of all its stored entries.
Return type
None
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, *, batch_size: Optional[int] = None, batch_concurrency: Optional[int] = None, overwrite_concurrency: Optional[int] = None, **kwargs: Any) → List[str][source]¶
Run texts through the embeddings and add them to the vectorstore.
If passing explicit ids, those entries whose id is in the store already
will be replaced.
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas.
ids (Optional[List[str]]) – Optional list of ids.
batch_size (Optional[int]) – Number of documents in each API call.
Check the underlying Astra DB HTTP API specs for the max value
(20 at the time of writing this). If not provided, defaults
to the instance-level setting.
batch_concurrency (Optional[int]) – number of threads to process
insertion batches concurrently. Defaults to instance-level
setting if not provided.
overwrite_concurrency (Optional[int]) – number of threads to process
pre-existing documents in each batch (which require individual
API calls). Defaults to instance-level setting if not provided.
kwargs (Any) – 
Return type
List[str]
Note
There are constraints on the allowed field names
in the metadata dictionaries, coming from the underlying Astra DB API.
For instance, the $ (dollar sign) cannot be used in the dict keys.
See this document for details:
See this document for details:
https://docs.datastax.com/en/astra/astra-db-vector/api-reference/data-api.html
Returns
The list of ids of the added texts.
Parameters
texts (Iterable[str]) – 
metadatas (Optional[List[dict]]) – 
ids (Optional[List[str]]) – 
batch_size (Optional[int]) – 
batch_concurrency (Optional[int]) – 
overwrite_concurrency (Optional[int]) – 
kwargs (Any) – 
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, concurrency: Optional[int] = None, **kwargs: Any) → Optional[bool][source]¶
Delete by vector ids.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
concurrency (Optional[int]) – max concurrency of single-doc delete requests.
Defaults to instance-level setting.
kwargs (Any) – 
Returns
True if deletion is successful, False otherwise.
Return type
Optional[bool]
async adelete_by_document_id(document_id: str) → bool[source]¶
Remove a single document from the store, given its document ID.
Parameters
document_id (str) – The document ID
Return type
bool
ReturnsTrue if a document has indeed been deleted, False if ID not found.
async adelete_collection() → None[source]¶
Completely delete the collection from the database (as opposed
to aclear(), which empties it only).
Stored data is lost and unrecoverable, resources are freed.
Use with caution.
Return type
None
async classmethod afrom_documents(documents: List[Document], embedding: Optional[Embeddings] = None, collection_vector_service_options: Optional[CollectionVectorServiceOptions] = None, **kwargs: Any) → AstraDBVectorStore[source]¶