Tony20100/langchain_full huggingface dataset:
Return a new Runnable that maps a list of inputs to a list of outputs,
by calling invoke() with each input.
Example
from langchain_core.runnables import RunnableLambda
def _lambda(x: int) -> int:
    return x + 1
runnable = RunnableLambda(_lambda)
print(runnable.map().invoke([1, 2, 3])) # [2, 3, 4]
Return type
Runnable[List[Input], List[Output]]
parse(text: str) → List[str][source]¶
Parse the output of a bash command.
Parameters
text (str) – 
Return type
List[str]
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
path (Union[str, Path]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod parse_obj(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
b (Union[str, bytes]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
parse_result(result: List[Generation], *, partial: bool = False) → T¶
Parse a list of candidate model Generations into a specific format.
Parse a list of candidate model Generations into a specific format.
The return value is parsed from only the first Generation in the result, whichis assumed to be the highest-likelihood Generation.
Parameters
result (List[Generation]) – A list of Generations to be parsed. The Generations are assumed
to be different candidate outputs for a single model input.
partial (bool) – 
Returns
Structured output.
Return type
T
parse_with_prompt(completion: str, prompt: PromptValue) → Any¶
Parse the output of an LLM call with the input prompt for context.
The prompt is largely provided in the event the OutputParser wants
to retry or fix the output in some way, and needs information from
the prompt to do so.
Parameters
completion (str) – String output of a language model.
prompt (PromptValue) – Input PromptValue.
Returns
Structured output
Return type
Any
pick(keys: Union[str, List[str]]) → RunnableSerializable[Any, Any]¶
Pick keys from the dict output of this runnable.
Pick single key:import json
from langchain_core.runnables import RunnableLambda, RunnableMap
as_str = RunnableLambda(str)
as_json = RunnableLambda(json.loads)
chain = RunnableMap(str=as_str, json=as_json)
chain.invoke("[1, 2, 3]")
# -> {"str": "[1, 2, 3]", "json": [1, 2, 3]}
json_only_chain = chain.pick("json")
json_only_chain.invoke("[1, 2, 3]")
# -> [1, 2, 3]
Pick list of keys:from typing import Any
import json
from langchain_core.runnables import RunnableLambda, RunnableMap
as_str = RunnableLambda(str)
as_json = RunnableLambda(json.loads)
as_str = RunnableLambda(str)
as_json = RunnableLambda(json.loads)
def as_bytes(x: Any) -> bytes:
    return bytes(x, "utf-8")
chain = RunnableMap(
    str=as_str,
    json=as_json,
    bytes=RunnableLambda(as_bytes)
)
chain.invoke("[1, 2, 3]")
# -> {"str": "[1, 2, 3]", "json": [1, 2, 3], "bytes": b"[1, 2, 3]"}
json_and_bytes_chain = chain.pick(["json", "bytes"])
json_and_bytes_chain.invoke("[1, 2, 3]")
# -> {"json": [1, 2, 3], "bytes": b"[1, 2, 3]"}
Parameters
keys (Union[str, List[str]]) – 
Return type
RunnableSerializable[Any, Any]
pipe(*others: Union[Runnable[Any, Other], Callable[[Any], Other]], name: Optional[str] = None) → RunnableSerializable[Input, Other]¶
Compose this Runnable with Runnable-like objects to make a RunnableSequence.
Equivalent to RunnableSequence(self, *others) or self | others[0] | …
Example
from langchain_core.runnables import RunnableLambda
def add_one(x: int) -> int:
    return x + 1
def mul_two(x: int) -> int:
    return x * 2
runnable_1 = RunnableLambda(add_one)
runnable_2 = RunnableLambda(mul_two)
sequence = runnable_1.pipe(runnable_2)
# Or equivalently:
# sequence = runnable_1 | runnable_2
# sequence = RunnableSequence(first=runnable_1, last=runnable_2)
sequence.invoke(1)
sequence.invoke(1)
await sequence.ainvoke(1)
# -> 4
sequence.batch([1, 2, 3])
await sequence.abatch([1, 2, 3])
# -> [4, 6, 8]
Parameters
others (Union[Runnable[Any, Other], Callable[[Any], Other]]) – 
name (Optional[str]) – 
Return type
RunnableSerializable[Input, Other]
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
stream(input: Input, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → Iterator[Output]¶
Default implementation of stream, which calls invoke.
Subclasses should override this method if they support streaming output.
Parameters
input (Input) – 
config (Optional[RunnableConfig]) – 
kwargs (Optional[Any]) – 
Return type
Iterator[Output]
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
Serialize the runnable to JSON.
Return type
Union[SerializedConstructor, SerializedNotImplemented]
to_json_not_implemented() → SerializedNotImplemented¶
Return type
SerializedNotImplemented
transform(input: Iterator[Input], config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → Iterator[Output]¶
Default implementation of transform, which buffers input and then calls stream.
Default implementation of transform, which buffers input and then calls stream.
Subclasses should override this method if they can start producing output while
input is still being generated.
Parameters
input (Iterator[Input]) – 
config (Optional[RunnableConfig]) – 
kwargs (Optional[Any]) – 
Return type
Iterator[Output]
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
with_alisteners(*, on_start: Optional[AsyncListener] = None, on_end: Optional[AsyncListener] = None, on_error: Optional[AsyncListener] = None) → Runnable[Input, Output]¶
Bind asynchronous lifecycle listeners to a Runnable, returning a new Runnable.
on_start: Asynchronously called before the runnable starts running.
on_end: Asynchronously called after the runnable finishes running.
on_error: Asynchronously called if the runnable throws an error.
The Run object contains information about the run, including its id,
type, input, output, error, start_time, end_time, and any tags or metadata
added to the run.
Example:
Parameters
on_start (Optional[AsyncListener]) – 
on_end (Optional[AsyncListener]) – 
on_error (Optional[AsyncListener]) – 
Return type
Runnable[Input, Output]
with_config(config: Optional[RunnableConfig] = None, **kwargs: Any) → Runnable[Input, Output]¶
Bind config to a Runnable, returning a new Runnable.
Parameters
config (Optional[RunnableConfig]) – 
kwargs (Any) – 
Return type
config (Optional[RunnableConfig]) – 
kwargs (Any) – 
Return type
Runnable[Input, Output]
with_fallbacks(fallbacks: Sequence[Runnable[Input, Output]], *, exceptions_to_handle: Tuple[Type[BaseException], ...] = (<class 'Exception'>,), exception_key: Optional[str] = None) → RunnableWithFallbacksT[Input, Output]¶
Add fallbacks to a runnable, returning a new Runnable.
Example
from typing import Iterator
from langchain_core.runnables import RunnableGenerator
def _generate_immediate_error(input: Iterator) -> Iterator[str]:
    raise ValueError()
    yield ""
def _generate(input: Iterator) -> Iterator[str]:
    yield from "foo bar"
runnable = RunnableGenerator(_generate_immediate_error).with_fallbacks(
    [RunnableGenerator(_generate)]
    )
print(''.join(runnable.stream({}))) #foo bar
Parameters
fallbacks (Sequence[Runnable[Input, Output]]) – A sequence of runnables to try if the original runnable fails.
exceptions_to_handle (Tuple[Type[BaseException], ...]) – A tuple of exception types to handle.
exception_key (Optional[str]) – If string is specified then handled exceptions will be passed
to fallbacks as part of the input under the specified key. If None,
exceptions will not be passed to fallbacks. If used, the base runnable
and its fallbacks must accept a dictionary as input.
Returns
A new Runnable that will try the original runnable, and then each
fallback in order, upon failures.
Return type
RunnableWithFallbacksT[Input, Output]
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: Any¶
The type of input this runnable accepts specified as a type annotation.
property OutputType: Type[T]¶
The type of output this runnable produces specified as a type annotation.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
langchain_google_community.bigquery.BigQueryLoader¶
class langchain_google_community.bigquery.BigQueryLoader(query: str, project: Optional[str] = None, page_content_columns: Optional[List[str]] = None, metadata_columns: Optional[List[str]] = None, credentials: Optional[Credentials] = None)[source]¶
Load from the Google Cloud Platform BigQuery.
Each document represents one row of the result. The page_content_columns
are written into the page_content of the document. The metadata_columns
are written into the metadata of the document. By default, all columns
are written into the page_content and none into the metadata.
Initialize BigQuery document loader.
Parameters
query (str) – The query to run in BigQuery.
project (Optional[str]) – Optional. The project to run the query in.
page_content_columns (Optional[List[str]]) – Optional. The columns to write into the page_content
of the document.
metadata_columns (Optional[List[str]]) – Optional. The columns to write into the metadata of the
document.
credentials (Optional[Credentials]) – google.auth.credentials.Credentials, optional
Credentials for accessing Google APIs. Use this parameter to override
default credentials, such as to use Compute Engine
(google.auth.compute_engine.Credentials) or Service Account
(google.oauth2.service_account.Credentials) credentials directly.
Methods
__init__(query[, project, ...])
Initialize BigQuery document loader.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
A lazy loader for Documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
load_and_split([text_splitter])
Load Documents and split into chunks.
__init__(query: str, project: Optional[str] = None, page_content_columns: Optional[List[str]] = None, metadata_columns: Optional[List[str]] = None, credentials: Optional[Credentials] = None)[source]¶
Initialize BigQuery document loader.
Parameters
query (str) – The query to run in BigQuery.
project (Optional[str]) – Optional. The project to run the query in.
page_content_columns (Optional[List[str]]) – Optional. The columns to write into the page_content
of the document.
metadata_columns (Optional[List[str]]) – Optional. The columns to write into the metadata of the
document.
credentials (Optional[Credentials]) – google.auth.credentials.Credentials, optional
Credentials for accessing Google APIs. Use this parameter to override
default credentials, such as to use Compute Engine
(google.auth.compute_engine.Credentials) or Service Account
(google.oauth2.service_account.Credentials) credentials directly.
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document][source]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
langchain_text_splitters.python.PythonCodeTextSplitter¶
class langchain_text_splitters.python.PythonCodeTextSplitter(**kwargs: Any)[source]¶
Attempts to split the text along Python syntax.
Initialize a PythonCodeTextSplitter.
Methods
__init__(**kwargs)
Initialize a PythonCodeTextSplitter.
atransform_documents(documents, **kwargs)
Asynchronously transform a list of documents.
create_documents(texts[, metadatas])
Create documents from a list of texts.
from_huggingface_tokenizer(tokenizer, **kwargs)
Text splitter that uses HuggingFace tokenizer to count length.
from_language(language, **kwargs)
from_tiktoken_encoder([encoding_name, ...])
Text splitter that uses tiktoken encoder to count length.
get_separators_for_language(language)
split_documents(documents)
Split documents.
split_text(text)
Split text into multiple components.
transform_documents(documents, **kwargs)
Transform sequence of documents by splitting them.
Parameters
kwargs (Any) – 
__init__(**kwargs: Any) → None[source]¶
Initialize a PythonCodeTextSplitter.
Parameters
kwargs (Any) – 
Return type
None
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document]¶
Asynchronously transform a list of documents.
Parameters
documents (Sequence[Document]) – A sequence of Documents to be transformed.
kwargs (Any) – 
Returns
A list of transformed Documents.
Return type
Sequence[Document]
create_documents(texts: List[str], metadatas: Optional[List[dict]] = None) → List[Document]¶
Create documents from a list of texts.
Parameters
texts (List[str]) –
Create documents from a list of texts.
Parameters
texts (List[str]) – 
metadatas (Optional[List[dict]]) – 
Return type
List[Document]
classmethod from_huggingface_tokenizer(tokenizer: Any, **kwargs: Any) → TextSplitter¶
Text splitter that uses HuggingFace tokenizer to count length.
Parameters
tokenizer (Any) – 
kwargs (Any) – 
Return type
TextSplitter
classmethod from_language(language: Language, **kwargs: Any) → RecursiveCharacterTextSplitter¶
Parameters
language (Language) – 
kwargs (Any) – 
Return type
RecursiveCharacterTextSplitter
classmethod from_tiktoken_encoder(encoding_name: str = 'gpt2', model_name: Optional[str] = None, allowed_special: Union[Literal['all'], AbstractSet[str]] = {}, disallowed_special: Union[Literal['all'], Collection[str]] = 'all', **kwargs: Any) → TS¶
Text splitter that uses tiktoken encoder to count length.
Parameters
encoding_name (str) – 
model_name (Optional[str]) – 
allowed_special (Union[Literal['all'], ~typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], ~typing.Collection[str]]) – 
kwargs (Any) – 
Return type
TS
static get_separators_for_language(language: Language) → List[str]¶
Parameters
language (Language) – 
Return type
List[str]
split_documents(documents: Iterable[Document]) → List[Document]¶
Split documents.
Parameters
documents (Iterable[Document]) – 
Return type
List[Document]
split_text(text: str) → List[str]¶
Split text into multiple components.
Parameters
text (str) – 
Return type
List[str]
langchain_community.docstore.base.Docstore¶
class langchain_community.docstore.base.Docstore[source]¶
Interface to access to place that stores documents.
Methods
__init__()
delete(ids)
Deleting IDs from in memory dictionary.
search(search)
Search for document.
__init__()¶
delete(ids: List) → None[source]¶
Deleting IDs from in memory dictionary.
Parameters
ids (List) – 
Return type
None
abstract search(search: str) → Union[str, Document][source]¶
Search for document.
If page exists, return the page summary, and a Document object.
If page does not exist, return similar entries.
Parameters
search (str) – 
Return type
Union[str, Document]
langchain_community.docstore.in_memory.InMemoryDocstore¶
class langchain_community.docstore.in_memory.InMemoryDocstore(_dict: Optional[Dict[str, Document]] = None)[source]¶
Simple in memory docstore in the form of a dict.
Initialize with dict.
Methods
__init__([_dict])
Initialize with dict.
add(texts)
Add texts to in memory dictionary.
delete(ids)
Deleting IDs from in memory dictionary.
search(search)
Search via direct lookup.
Parameters
_dict (Optional[Dict[str, Document]]) – 
__init__(_dict: Optional[Dict[str, Document]] = None)[source]¶
Initialize with dict.
Parameters
_dict (Optional[Dict[str, Document]]) – 
add(texts: Dict[str, Document]) → None[source]¶
Add texts to in memory dictionary.
Parameters
texts (Dict[str, Document]) – dictionary of id -> document.
Returns
None
Return type
None
delete(ids: List) → None[source]¶
Deleting IDs from in memory dictionary.
Parameters
ids (List) – 
Return type
None
search(search: str) → Union[str, Document][source]¶
Search via direct lookup.
Parameters
search (str) – id of a document to search for.
Returns
Document if found, else error message.
Return type
Union[str, Document]
Examples using InMemoryDocstore¶
Annoy
How to use a time-weighted vector store retriever
langchain_community.docstore.arbitrary_fn.DocstoreFn¶
class langchain_community.docstore.arbitrary_fn.DocstoreFn(lookup_fn: Callable[[str], Union[Document, str]])[source]¶
Docstore via arbitrary lookup function.
This is useful when:
it’s expensive to construct an InMemoryDocstore/dict
you retrieve documents from remote sources
you just want to reuse existing objects
Methods
__init__(lookup_fn)
delete(ids)
Deleting IDs from in memory dictionary.
search(search)
Search for a document.
Parameters
lookup_fn (Callable[[str], Union[Document, str]]) – 
__init__(lookup_fn: Callable[[str], Union[Document, str]])[source]¶
Parameters
lookup_fn (Callable[[str], Union[Document, str]]) – 
delete(ids: List) → None¶
Deleting IDs from in memory dictionary.
Parameters
ids (List) – 
Return type
None
search(search: str) → Document[source]¶
Search for a document.
Parameters
search (str) – search string
Returns
Document if found, else error message.
Return type
Document
langchain_community.docstore.base.AddableMixin¶
class langchain_community.docstore.base.AddableMixin[source]¶
Mixin class that supports adding texts.
Methods
__init__()
add(texts)
Add more documents.
__init__()¶
abstract add(texts: Dict[str, Document]) → None[source]¶
Add more documents.
Parameters
texts (Dict[str, Document]) – 
Return type
None
langchain_community.docstore.wikipedia.Wikipedia¶
class langchain_community.docstore.wikipedia.Wikipedia[source]¶
Wikipedia API.
Check that wikipedia package is installed.
Methods
__init__()
Check that wikipedia package is installed.
delete(ids)
Deleting IDs from in memory dictionary.
search(search)
Try to search for wiki page.
__init__() → None[source]¶
Check that wikipedia package is installed.
Return type
None
delete(ids: List) → None¶
Deleting IDs from in memory dictionary.
Parameters
ids (List) – 
Return type
None
search(search: str) → Union[str, Document][source]¶
Try to search for wiki page.
If page exists, return the page summary, and a PageWithLookups object.
If page does not exist, return similar entries.
Parameters
search (str) – search string.
Return type
Union[str, Document]
Returns: a Document object or error message.
langchain_community.llms.huggingface_hub.HuggingFaceHub¶
class langchain_community.llms.huggingface_hub.HuggingFaceHub[source]¶
Bases: LLM
[Deprecated] HuggingFaceHub  models.
! This class is deprecated, you should use HuggingFaceEndpoint instead.
To use, you should have the huggingface_hub python package installed, and the
environment variable HUGGINGFACEHUB_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Supports text-generation, text2text-generation, conversational, translation,and summarization.
Example
from langchain_community.llms import HuggingFaceHub
hf = HuggingFaceHub(repo_id="gpt2", huggingfacehub_api_token="my-api-key")
Notes
Deprecated since version 0.0.21.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param huggingfacehub_api_token: Optional[str] = None¶
param metadata: Optional[Dict[str, Any]] = None¶
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Optional[dict] = None¶
Keyword arguments to pass to the model.
param repo_id: Optional[str] = None¶
Model name to use.
If not provided, the default model for the chosen task will be used.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param task: Optional[str] = None¶
Task to call the model with.
Should be a task that returns generated_text, summary_text,
or translation_text.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Any) – 
Return type
List[str]
async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]¶
Run ainvoke in parallel on a list of inputs,
yielding results as they complete.
Parameters
inputs (Sequence[Input]) – 
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Tuple[int, Union[Output, Exception]]]