Tony20100/langchain_full huggingface dataset:
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
kwargs (Any) – vectorstore specific parameters
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
kwargs (Any) – 
Returns
List of Documents most similar to the query.
Return type
List[Document]
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
associate(source: str, target: str)[source]¶
The vectorstore associates a source phrase with a target phrase.
The vectorstore associates a source phrase with a target phrase.
When the vectorstore sees the source phrase, it will also consider results
that are relevant to the target phrase.
Parameters
source (str) – text to associate to target.
target (str) – text to associate source to.
associate_batch(text_pairs: List[Tuple[str, str]])[source]¶
Given a batch of (source, target) pairs, the vectorstore associates
each source phrase with the corresponding target phrase.
Parameters
text_pairs (List[Tuple[str, str]]) – list of (source, target) text pairs. For each pair in
list (this) – 
target. (the source will be associated with the) – 
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_checkpoint(checkpoint: Union[str, Path], thirdai_key: Optional[str] = None)[source]¶
Create a NeuralDBVectorStore with a base model from a saved checkpoint
To use, set the THIRDAI_KEY environment variable with your ThirdAI
API key, or pass thirdai_key as a named parameter.
Example
from langchain_community.vectorstores import NeuralDBVectorStore
vectorstore = NeuralDBVectorStore.from_checkpoint(
    checkpoint="/path/to/checkpoint.ndb",
    thirdai_key="your-thirdai-key",
)
vectorstore.insert([
    "/path/to/doc.pdf",
    "/path/to/doc.docx",
    "/path/to/doc.csv",
])
"/path/to/doc.docx",
    "/path/to/doc.csv",
])
documents = vectorstore.similarity_search("AI-driven music therapy")
Parameters
checkpoint (Union[str, Path]) – 
thirdai_key (Optional[str]) – 
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
classmethod from_scratch(thirdai_key: Optional[str] = None, **model_kwargs)[source]¶
Create a NeuralDBVectorStore from scratch.
To use, set the THIRDAI_KEY environment variable with your ThirdAI
API key, or pass thirdai_key as a named parameter.
Example
from langchain_community.vectorstores import NeuralDBVectorStore
vectorstore = NeuralDBVectorStore.from_scratch(
    thirdai_key="your-thirdai-key",
)
vectorstore.insert([
    "/path/to/doc.pdf",
    "/path/to/doc.docx",
    "/path/to/doc.csv",
])
documents = vectorstore.similarity_search("AI-driven music therapy")
Parameters
thirdai_key (Optional[str]) – 
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → NeuralDBVectorStore[source]¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – 
embedding (Embeddings) – 
metadatas (Optional[List[dict]]) – 
kwargs (Any) – 
Return type
NeuralDBVectorStore
kwargs (Any) – 
Return type
NeuralDBVectorStore
insert(sources: List[Any], train: bool = True, fast_mode: bool = True, **kwargs)[source]¶
Inserts files / document sources into the vectorstore.
Parameters
train (bool) – When True this means that the underlying model in the
files. (NeuralDB will undergo unsupervised pretraining on the inserted) – 
True. (Defaults to) – 
fast_mode (bool) – Much faster insertion with a slight drop in performance.
True. – 
sources (List[Any]) – 
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
save(path: str)[source]¶
Saves a NeuralDB instance to disk. Can be loaded into memory by
calling NeuralDB.from_checkpoint(path)
Parameters
path (str) – path on disk to save the NeuralDB instance to.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 10, **kwargs: Any) → List[Document][source]¶
Retrieve {k} contexts with for a given query
Parameters
query (str) – Query to submit to the model
k (int) – The max number of context results to retrieve. Defaults to 10.
kwargs (Any) – 
Return type
List[Document]
kwargs (Any) – 
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
kwargs (Any) – 
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
upvote(query: str, document_id: Union[int, str])[source]¶
The vectorstore upweights the score of a document for a specific query.
This is useful for fine-tuning the vectorstore to user behavior.
This is useful for fine-tuning the vectorstore to user behavior.
Parameters
query (str) – text to associate with document_id
document_id (Union[int, str]) – id of the document to associate query with.
upvote_batch(query_id_pairs: List[Tuple[str, int]])[source]¶
Given a batch of (query, document id) pairs, the vectorstore upweights
the scores of the document for the corresponding queries.
This is useful for fine-tuning the vectorstore to user behavior.
Parameters
query_id_pairs (List[Tuple[str, int]]) – list of (query, document id) pairs. For each pair in
list (this) – 
query. (the model will upweight the document id for the) – 
classmethod validate_environments(values: Dict) → Dict[source]¶
Validate ThirdAI environment variables.
Parameters
values (Dict) – 
Return type
Dict
Examples using NeuralDBVectorStore¶
ThirdAI NeuralDB
langchain_community.vectorstores.vectara.VectaraQueryConfig¶
class langchain_community.vectorstores.vectara.VectaraQueryConfig(k: int = 10, lambda_val: float = 0.0, filter: str = '', score_threshold: Optional[float] = None, n_sentence_before: int = 2, n_sentence_after: int = 2, n_sentence_context: Optional[int] = None, mmr_config: Optional[MMRConfig] = None, summary_config: Optional[SummaryConfig] = None, rerank_config: Optional[RerankConfig] = None)[source]¶
Configuration for Vectara query.
k: Number of Documents to return. Defaults to 10.
lambda_val: lexical match parameter for hybrid search.
filter Dictionary of argument(s) to filter on metadata. For example a
filter can be “doc.rating > 3.0 and part.lang = ‘deu’”} see
https://docs.vectara.com/docs/search-apis/sql/filter-overview
for more details.
score_threshold: minimal score threshold for the result.If defined, results with score less than this value will be
filtered out.
n_sentence_before: number of sentences before the matching segmentto add, defaults to 2
n_sentence_after: number of sentences before the matching segmentto add, defaults to 2
rerank_config: RerankConfig configuration dataclass
summary_config: SummaryConfig configuration dataclass
Attributes
filter
k
lambda_val
n_sentence_after
n_sentence_before
score_threshold
rerank_config
summary_config
Methods
__init__([k, lambda_val, filter, ...])
Parameters
k (int) – 
lambda_val (float) – 
filter (str) – 
score_threshold (Optional[float]) – 
n_sentence_before (int) –
score_threshold (Optional[float]) – 
n_sentence_before (int) – 
n_sentence_after (int) – 
n_sentence_context (Optional[int]) – 
mmr_config (Optional[MMRConfig]) – 
summary_config (SummaryConfig) – 
rerank_config (RerankConfig) – 
__init__(k: int = 10, lambda_val: float = 0.0, filter: str = '', score_threshold: Optional[float] = None, n_sentence_before: int = 2, n_sentence_after: int = 2, n_sentence_context: Optional[int] = None, mmr_config: Optional[MMRConfig] = None, summary_config: Optional[SummaryConfig] = None, rerank_config: Optional[RerankConfig] = None)[source]¶
Parameters
k (int) – 
lambda_val (float) – 
filter (str) – 
score_threshold (Optional[float]) – 
n_sentence_before (int) – 
n_sentence_after (int) – 
n_sentence_context (Optional[int]) – 
mmr_config (Optional[MMRConfig]) – 
summary_config (Optional[SummaryConfig]) – 
rerank_config (Optional[RerankConfig]) –
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, refresh])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
create_index(**kwargs)
Creates an index in your project.
delete([ids])
Delete by vector ID or other criteria.
from_documents(documents[, embedding, ids, ...])
Create an AtlasDB vectorstore from a list of documents.
from_texts(texts[, embedding, metadatas, ...])
Create an AtlasDB vectorstore from a raw documents.
max_marginal_relevance_search(query[, k, ...])
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Run similarity search with AtlasDB
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
__init__(name: str, embedding_function: Optional[Embeddings] = None, api_key: Optional[str] = None, description: str = 'A description for your project', is_public: bool = True, reset_project_if_exists: bool = False) → None[source]¶
Initialize the Atlas Client
Parameters
name (str) – The name of your project. If the project already exists,
it will be loaded.
embedding_function (Optional[Embeddings]) – An optional function used for
embedding your data. If None, data will be embedded with
Nomic’s embed model.
api_key (str) – Your nomic API key
description (str) – A description for your project.
is_public (bool) – Whether your project is publicly accessible.
True by default.
reset_project_if_exists (bool) – Whether to reset this project if it
already exists. Default False.
Generally useful during development and testing.
Return type
None
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, refresh: bool = True, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]]) – An optional list of ids.
refresh (bool) – Whether or not to refresh indices with the updated data.
Default True.
kwargs (Any) – 
Returns
Default True.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
create_index(**kwargs: Any) → Any[source]¶
Creates an index in your project.
See
https://docs.nomic.ai/atlas_api.html#nomic.project.AtlasProject.create_index
for full detail.
Parameters
kwargs (Any) – 
Return type
Any
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, ids: Optional[List[str]] = None, name: Optional[str] = None, api_key: Optional[str] = None, persist_directory: Optional[str] = None, description: str = 'A description for your project', is_public: bool = True, reset_project_if_exists: bool = False, index_kwargs: Optional[dict] = None, **kwargs: Any) → AtlasDB[source]¶
Create an AtlasDB vectorstore from a list of documents.
Parameters
name (str) – Name of the collection to create.
api_key (str) – Your nomic API key,
documents (List[Document]) – List of documents to add to the vectorstore.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
ids (Optional[List[str]]) – Optional list of document IDs. If None,
ids will be auto created
description (str) – A description for your project.
is_public (bool) – Whether your project is publicly accessible.
True by default.
reset_project_if_exists (bool) – Whether to reset this project if
it already exists. Default False.
Generally useful during development and testing.
index_kwargs (Optional[dict]) – Dict of kwargs for index creation.
See https://docs.nomic.ai/atlas_api.html
persist_directory (Optional[str]) – 
kwargs (Any) – 
Returns
Nomic’s neural database and finest rhizomatic instrument
Return type
AtlasDB
Returns
Nomic’s neural database and finest rhizomatic instrument
Return type
AtlasDB
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, name: Optional[str] = None, api_key: Optional[str] = None, description: str = 'A description for your project', is_public: bool = True, reset_project_if_exists: bool = False, index_kwargs: Optional[dict] = None, **kwargs: Any) → AtlasDB[source]¶
Create an AtlasDB vectorstore from a raw documents.
Parameters
texts (List[str]) – The list of texts to ingest.
name (str) – Name of the project to create.
api_key (str) – Your nomic API key,
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
ids (Optional[List[str]]) – Optional list of document IDs. If None,
ids will be auto created
description (str) – A description for your project.
is_public (bool) – Whether your project is publicly accessible.
True by default.
reset_project_if_exists (bool) – Whether to reset this project if it
already exists. Default False.
Generally useful during development and testing.
index_kwargs (Optional[dict]) – Dict of kwargs for index creation.
See https://docs.nomic.ai/atlas_api.html
kwargs (Any) – 
Returns
Nomic’s neural database and finest rhizomatic instrument
Return type
AtlasDB