Tony20100/langchain_full huggingface dataset:
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
**kwargs (Any) – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
Return type
List[str]
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – 
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, is_duplicate_texts: Optional[bool] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
:param texts: Iterable of strings to add to the vectorstore.
:param metadatas: Optional list of metadatas associated with the texts.
:param is_duplicate_texts: Optional whether to duplicate texts. Defaults to True.
:param kwargs: any possible extend parameters in the future.
Returns
List of ids from adding the texts into the vectorstore.
Parameters
texts (Iterable[str]) –
Parameters
texts (Iterable[str]) – 
metadatas (Optional[List[dict]]) – 
is_duplicate_texts (Optional[bool]) – 
kwargs (Any) – 
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids (Optional[List[str]]) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
Parameters
documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
Parameters
texts (List[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]]) – Optional list of metadatas associated with the texts.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – 
Return type
VST
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Returns
List of Tuples of (doc, similarity_score)
Parameters
args (Any) – 
kwargs (Any) – 
Return type
List[Tuple[Document, float]]
create_table(table_name: str, **kwargs: Any) → bool[source]¶
Create a new table.
Parameters
table_name (str) – 
kwargs (Any) – 
Return type
bool
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Delete the documents which have the specified ids.
Parameters
ids (Optional[List[str]]) – The ids of the embedding vectors.
**kwargs (Any) – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful.
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, table_name: str = 'langchain_awadb', log_and_data_dir: Optional[str] = None, client: Optional[awadb.Client] = None, **kwargs: Any) → AwaDB[source]¶
Create an AwaDB vectorstore from a list of documents.
If a log_and_data_dir specified, the table will be persisted there.
Parameters
documents (List[Document]) – List of documents to add to the vectorstore.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
table_name (str) – Name of the table to create.
log_and_data_dir (Optional[str]) – Directory to persist the table.
client (Optional[awadb.Client]) – AwaDB client.
Any – Any possible parameters in the future
kwargs (Any) – 
Returns
AwaDB vectorstore.
Return type
AwaDB
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, table_name: str = 'langchain_awadb', log_and_data_dir: Optional[str] = None, client: Optional[awadb.Client] = None, **kwargs: Any) → AwaDB[source]¶
Create an AwaDB vectorstore from a raw documents.
Parameters
texts (List[str]) – List of texts to add to the table.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
table_name (str) – Name of the table to create.
log_and_data_dir (Optional[str]) – Directory of logging and persistence.
client (Optional[awadb.Client]) – AwaDB client
kwargs (Any) – 
Returns
AwaDB vectorstore.
Return type
AwaDB
Returns
AwaDB vectorstore.
Return type
AwaDB
get(ids: Optional[List[str]] = None, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, not_include_fields: Optional[Set[str]] = None, limit: Optional[int] = None, **kwargs: Any) → Dict[str, Document][source]¶
Return docs according ids.
Parameters
ids (Optional[List[str]]) – The ids of the embedding vectors.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by any metadata of the document.
not_include_fields (Optional[Set[str]]) – Not pack the specified fields of each document.
limit (Optional[int]) – The number of documents to return. Defaults to 5. Optional.
kwargs (Any) – 
Returns
Documents which satisfy the input conditions.
Return type
Dict[str, Document]
get_current_table(**kwargs: Any) → str[source]¶
Get the current table.
Parameters
kwargs (Any) – 
Return type
str
list_tables(**kwargs: Any) → List[str][source]¶
List all the tables created by the client.
Parameters
kwargs (Any) – 
Return type
List[str]
load_local(table_name: str, **kwargs: Any) → bool[source]¶
Load the local specified table.
Parameters
table_name (str) – Table name
kwargs (Any) – Any possible extend parameters in the future.
Returns
Success or failure of loading the local specified table
Return type
bool
Returns
Success or failure of loading the local specified table
Return type
bool
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
kwargs (Any) – 
Returns
List of Documents selected by maximal marginal relevance.
Return type
List[Document]
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
Parameters
query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”,
“mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.
Return type
List[Document]
similarity_search(query: str, k: int = 4, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
Parameters
query (str) – Text query.
k (int) – The maximum number of documents to return.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
`{"color" (E.g.) – ”red”, “price”: 4.20}`. Optional.
`{"max_price" (E.g.) – 15.66, “min_price”: 4.20}`
field (price is the metadata) – 
filter (means range) – 
`{"maxe_price" (E.g.) – 15.66, “mine_price”: 4.20}`
field – 
filter – 
kwargs (Any) – Any possible extend parameters in the future.
Returns
Returns the k most similar documents to the specified text query.
Return type
List[Document]
similarity_search_by_vector(embedding: Optional[List[float]] = None, k: int = 4, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, not_include_fields_in_metadata: Optional[Set[str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
Parameters
embedding (Optional[List[float]]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
not_incude_fields_in_metadata – Not include meta fields of each document.
not_include_fields_in_metadata (Optional[Set[str]]) – 
kwargs (Any) – 
Returns
List of Documents which are the most similar to the query vector.
Return type
List[Document]
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
Return type
List[Tuple[Document, float]]
similarity_search_with_score(query: str, k: int = 4, text_in_page_content: Optional[str] = None, meta_filter: Optional[dict] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
The most k similar documents and scores of the specified query.
Parameters
query (str) – Text query.
k (int) – The k most similar documents to the text query.
text_in_page_content (Optional[str]) – Filter by the text in page_content of Document.
meta_filter (Optional[dict]) – Filter by metadata. Defaults to None.
kwargs (Any) – Any possible extend parameters in the future.
Returns
The k most similar documents to the specified text query.
0 is dissimilar, 1 is the most similar.
Return type
List[Tuple[Document, float]]
update(ids: List[str], texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str][source]¶
Update the documents which have the specified ids.
Parameters
Update the documents which have the specified ids.
Parameters
ids (List[str]) – The id list of the updating embedding vector.
texts (Iterable[str]) – The texts of the updating documents.
metadatas (Optional[List[dict]]) – The metadatas of the updating documents.
kwargs (Any) – 
Returns
the ids of the updated documents.
Return type
List[str]
use(table_name: str, **kwargs: Any) → bool[source]¶
Use the specified table. Don’t know the tables, please invoke list_tables.
Parameters
table_name (str) – 
kwargs (Any) – 
Return type
bool
Examples using AwaDB¶
AwaDB
langchain_community.vectorstores.vectara.VectaraRAG¶
class langchain_community.vectorstores.vectara.VectaraRAG(vectara: Vectara, config: VectaraQueryConfig, chat: bool = False)[source]¶
Attributes
InputType
The type of input this runnable accepts specified as a type annotation.
OutputType
The type of output this runnable produces specified as a type annotation.
config_specs
List configurable fields for this runnable.
input_schema
The type of input this runnable accepts specified as a pydantic model.
name
The name of the runnable.
output_schema
The type of output this runnable produces specified as a pydantic model.
Methods
__init__(vectara, config[, chat])
abatch(inputs[, config, return_exceptions])
Default implementation runs ainvoke in parallel using asyncio.gather.
abatch_as_completed(inputs[, config, ...])
Run ainvoke in parallel on a list of inputs, yielding results as they complete.
ainvoke(input[, config])
Default implementation of ainvoke, calls invoke from a thread.
assign(**kwargs)
Assigns new fields to the dict output of this runnable.
astream(input[, config])
Default implementation of astream, which calls ainvoke.
astream_events(input[, config, ...])
[Beta] Generate a stream of events.
astream_log(input[, config, diff, ...])
Stream all output from a runnable, as reported to the callback system.
atransform(input[, config])
Default implementation of atransform, which buffers input and calls astream.
batch(inputs[, config, return_exceptions])
Default implementation runs invoke in parallel using a thread pool executor.
batch_as_completed(inputs[, config, ...])
batch_as_completed(inputs[, config, ...])
Run invoke in parallel on a list of inputs, yielding results as they complete.
bind(**kwargs)
Bind arguments to a Runnable, returning a new Runnable.
config_schema(*[, include])
The type of config this runnable accepts specified as a pydantic model.
get_graph([config])
Return a graph representation of this runnable.
get_input_schema([config])
Get a pydantic model that can be used to validate input to the runnable.
get_name([suffix, name])
Get the name of the runnable.
get_output_schema([config])
Get a pydantic model that can be used to validate output to the runnable.
get_prompts([config])
invoke(input[, config])
Transform a single input into an output.
map()
Return a new Runnable that maps a list of inputs to a list of outputs, by calling invoke() with each input.
pick(keys)
Pick keys from the dict output of this runnable.
pipe(*others[, name])
Compose this Runnable with Runnable-like objects to make a RunnableSequence.
stream(input[, config])
get streaming output from Vectara RAG
transform(input[, config])
Default implementation of transform, which buffers input and then calls stream.
with_alisteners(*[, on_start, on_end, on_error])
Bind asynchronous lifecycle listeners to a Runnable, returning a new Runnable.
with_config([config])
Bind config to a Runnable, returning a new Runnable.
with_fallbacks(fallbacks, *[, ...])
Add fallbacks to a runnable, returning a new Runnable.
with_listeners(*[, on_start, on_end, on_error])
Bind lifecycle listeners to a Runnable, returning a new Runnable.
Bind lifecycle listeners to a Runnable, returning a new Runnable.
with_retry(*[, retry_if_exception_type, ...])
Create a new Runnable that retries the original runnable on exceptions.
with_types(*[, input_type, output_type])
Bind input and output types to a Runnable, returning a new Runnable.
Parameters
vectara (Vectara) – 
config (VectaraQueryConfig) – 
chat (bool) – 
__init__(vectara: Vectara, config: VectaraQueryConfig, chat: bool = False)[source]¶
Parameters
vectara (Vectara) – 
config (VectaraQueryConfig) – 
chat (bool) – 
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Input]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
List[Output]
async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]¶
Run ainvoke in parallel on a list of inputs,
yielding results as they complete.
Parameters
inputs (Sequence[Input]) –
yielding results as they complete.
Parameters
inputs (Sequence[Input]) – 
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Tuple[int, Union[Output, Exception]]]
async ainvoke(input: Input, config: Optional[RunnableConfig] = None, **kwargs: Any) → Output¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
Parameters
input (Input) – 
config (Optional[RunnableConfig]) – 
kwargs (Any) – 
Return type
Output
assign(**kwargs: Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any], Mapping[str, Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any]]]]) → RunnableSerializable[Any, Any]¶
Assigns new fields to the dict output of this runnable.
Returns a new runnable.
from langchain_community.llms.fake import FakeStreamingListLLM
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import SystemMessagePromptTemplate
from langchain_core.runnables import Runnable
from operator import itemgetter
prompt = (
    SystemMessagePromptTemplate.from_template("You are a nice assistant.")
    + "{question}"
)
llm = FakeStreamingListLLM(responses=["foo-lish"])
chain: Runnable = prompt | llm | {"str": StrOutputParser()}
chain_with_assign = chain.assign(hello=itemgetter("str") | llm)
print(chain_with_assign.input_schema.schema())
print(chain_with_assign.input_schema.schema())
# {'title': 'PromptInput', 'type': 'object', 'properties':
{'question': {'title': 'Question', 'type': 'string'}}}
print(chain_with_assign.output_schema.schema()) #
{'title': 'RunnableSequenceOutput', 'type': 'object', 'properties':
{'str': {'title': 'Str',
'type': 'string'}, 'hello': {'title': 'Hello', 'type': 'string'}}}
Parameters
kwargs (Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any], Mapping[str, Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any]]]]) – 
Return type
RunnableSerializable[Any, Any]
async astream(input: Input, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → AsyncIterator[Output]¶
Default implementation of astream, which calls ainvoke.
Subclasses should override this method if they support streaming output.
Parameters
input (Input) – 
config (Optional[RunnableConfig]) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Output]
astream_events(input: Any, config: Optional[RunnableConfig] = None, *, version: Literal['v1', 'v2'], include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Any) → AsyncIterator[StreamEvent]¶
[Beta] Generate a stream of events.
Use to create an iterator over StreamEvents that provide real-time information
diff (bool) – Whether to yield diffs between each step, or the current state.
with_streamed_output_list (bool) – Whether to yield the streamed_output list.
include_names (Optional[Sequence[str]]) – Only include logs with these names.
include_types (Optional[Sequence[str]]) – Only include logs with these types.
include_tags (Optional[Sequence[str]]) – Only include logs with these tags.
exclude_names (Optional[Sequence[str]]) – Exclude logs with these names.
exclude_types (Optional[Sequence[str]]) – Exclude logs with these types.
exclude_tags (Optional[Sequence[str]]) – Exclude logs with these tags.
kwargs (Any) – 
Return type
Union[AsyncIterator[RunLogPatch], AsyncIterator[RunLog]]
async atransform(input: AsyncIterator[Input], config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → AsyncIterator[Output]¶
Default implementation of atransform, which buffers input and calls astream.
Subclasses should override this method if they can start producing output while
input is still being generated.
Parameters
input (AsyncIterator[Input]) – 
config (Optional[RunnableConfig]) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Output]
batch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs invoke in parallel using a thread pool executor.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Input]) –