viruPVS/langchain-docs huggingface dataset:
The unique identifier is a list of strings that describes the path
to the object.
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
to_json_not_implemented() → SerializedNotImplemented¶
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
Examples using Document¶
ChatGPT Plugin
Weaviate Hybrid Search
BM25
TF-IDF
Apify
Vectara Text Generation
PGVector
Annoy
Neo4j Vector Index
Postgres Embedding
Faiss
Nuclia Understanding API document transformer
OpenAI Functions Metadata Tagger
Doctran Extract Properties
Doctran Interrogate Documents
OpenAI Functions Metadata Tagger
Doctran Extract Properties
Doctran Interrogate Documents
Doctran Translate Documents
TensorFlow Datasets
Airbyte Salesforce
Airbyte CDK
Airbyte Stripe
Copy Paste
Airbyte Typeform
Apify Dataset
Docugami
Airbyte Hubspot
Airbyte Gong
Airbyte Shopify
Airbyte Zendesk Support
SageMakerEndpoint
LLM Caching integrations
Retrieve from vector stores directly
Multiple Retrieval Sources
Retrieve as you generate with FLARE
!pip install bs4
Plug-and-Plai
Custom Agent with PlugIn Retrieval
Set env var OPENAI_API_KEY or load from a .env file
SQL
Indexing
MultiVector Retriever
Milvus
Weaviate
Vectara
DashVector
Elasticsearch
Chroma
Pinecone
Supabase
Redis
MyScale
Deep Lake
Qdrant
Memory in the Multi-Input Chain
Custom agent with tool retrieval
langchain_core.documents.transformers.BaseDocumentTransformer¶
class langchain_core.documents.transformers.BaseDocumentTransformer[source]¶
Abstract base class for document transformation systems.
A document transformation system takes a sequence of Documents and returns a
sequence of transformed Documents.
Example
class EmbeddingsRedundantFilter(BaseDocumentTransformer, BaseModel):
    embeddings: Embeddings
    similarity_fn: Callable = cosine_similarity
    similarity_threshold: float = 0.95
    class Config:
        arbitrary_types_allowed = True
    def transform_documents(
        self, documents: Sequence[Document], **kwargs: Any
    ) -> Sequence[Document]:
        stateful_documents = get_stateful_documents(documents)
        embedded_documents = _get_embeddings_from_stateful_docs(
            self.embeddings, stateful_documents
        )
        included_idxs = _filter_similar_embeddings(
            embedded_documents, self.similarity_fn, self.similarity_threshold
        )
        return [stateful_documents[i] for i in sorted(included_idxs)]
    async def atransform_documents(
        self, documents: Sequence[Document], **kwargs: Any
    ) -> Sequence[Document]:
        raise NotImplementedError
Methods
__init__()
atransform_documents(documents, **kwargs)
Asynchronously transform a list of documents.
transform_documents(documents, **kwargs)
Transform a list of documents.
__init__()¶
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Asynchronously transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
abstract transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
langchain_core.utils.iter.Tee¶
class langchain_core.utils.iter.Tee(iterable: Iterator[T], n: int = 2, *, lock: Optional[ContextManager[Any]] = None)[source]¶
Create n separate asynchronous iterators over iterable
This splits a single iterable into multiple iterators, each providing
the same items in the same order.
All child iterators may advance separately but share the same items
from iterable – when the most advanced iterator retrieves an item,
it is buffered until the least advanced iterator has yielded it as well.
A tee works lazily and can handle an infinite iterable, provided
that all iterators advance.
async def derivative(sensor_data):
    previous, current = a.tee(sensor_data, n=2)
    await a.anext(previous)  # advance one iterator
    return a.map(operator.sub, previous, current)
Unlike itertools.tee(), tee() returns a custom type instead
of a tuple. Like a tuple, it can be indexed, iterated and unpacked
to get the child iterators. In addition, its aclose() method
immediately closes all children, and it can be used in an async with context
for the same effect.
If iterable is an iterator and read elsewhere, tee will not
provide these items. Also, tee must internally buffer each item until the
last iterator has yielded it; if the most and least advanced iterator differ
by most data, using a list is more efficient (but not lazy).
If the underlying iterable is concurrency safe (anext may be awaited
concurrently) the resulting iterators are concurrency safe as well. Otherwise,
the iterators are safe if there is only ever one single “most advanced” iterator.
To enforce sequential use of anext, provide a lock
- e.g. an asyncio.Lock instance in an asyncio application -
and access is automatically synchronised.
Methods
langchain_community.utils.openai_functions.convert_pydantic_to_openai_tool¶
langchain_community.utils.openai_functions.convert_pydantic_to_openai_tool(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → ToolDescription[source]¶
Converts a Pydantic model to a function description for the OpenAI API.
langchain_community.utils.openai_functions.convert_pydantic_to_openai_function¶
langchain_community.utils.openai_functions.convert_pydantic_to_openai_function(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → FunctionDescription[source]¶
Converts a Pydantic model to a function description for the OpenAI API.
langchain_core.utils.formatting.StrictFormatter¶
class langchain_core.utils.formatting.StrictFormatter[source]¶
A subclass of formatter that checks for extra keys.
Methods
__init__()
check_unused_args(used_args, args, kwargs)
Check to see if extra parameters are passed.
convert_field(value, conversion)
format(format_string, /, *args, **kwargs)
format_field(value, format_spec)
get_field(field_name, args, kwargs)
get_value(key, args, kwargs)
parse(format_string)
validate_input_variables(format_string, ...)
vformat(format_string, args, kwargs)
Check that no arguments are provided.
__init__()¶
check_unused_args(used_args: Sequence[Union[int, str]], args: Sequence, kwargs: Mapping[str, Any]) → None[source]¶
Check to see if extra parameters are passed.
convert_field(value, conversion)¶
format(format_string, /, *args, **kwargs)¶
format_field(value, format_spec)¶
get_field(field_name, args, kwargs)¶
get_value(key, args, kwargs)¶
parse(format_string)¶
validate_input_variables(format_string: str, input_variables: List[str]) → None[source]¶
vformat(format_string: str, args: Sequence, kwargs: Mapping[str, Any]) → str[source]¶
Check that no arguments are provided.
langchain_core.utils.utils.mock_now¶
langchain_core.utils.utils.mock_now(dt_value)[source]¶
Context manager for mocking out datetime.now() in unit tests.
Example:
with mock_now(datetime.datetime(2011, 2, 3, 10, 11)):
assert datetime.datetime.now() == datetime.datetime(2011, 2, 3, 10, 11)
langchain.utils.ernie_functions.convert_pydantic_to_ernie_function¶
langchain.utils.ernie_functions.convert_pydantic_to_ernie_function(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → FunctionDescription[source]¶
Converts a Pydantic model to a function description for the Ernie API.
langchain_core.utils.aiter.Tee¶
class langchain_core.utils.aiter.Tee(iterable: AsyncIterator[T], n: int = 2, *, lock: Optional[AsyncContextManager[Any]] = None)[source]¶
Create n separate asynchronous iterators over iterable
This splits a single iterable into multiple iterators, each providing
the same items in the same order.
All child iterators may advance separately but share the same items
from iterable – when the most advanced iterator retrieves an item,
it is buffered until the least advanced iterator has yielded it as well.
A tee works lazily and can handle an infinite iterable, provided
that all iterators advance.
async def derivative(sensor_data):
    previous, current = a.tee(sensor_data, n=2)
    await a.anext(previous)  # advance one iterator
    return a.map(operator.sub, previous, current)
Unlike itertools.tee(), tee() returns a custom type instead
of a tuple. Like a tuple, it can be indexed, iterated and unpacked
to get the child iterators. In addition, its aclose() method
immediately closes all children, and it can be used in an async with context
for the same effect.
If iterable is an iterator and read elsewhere, tee will not
provide these items. Also, tee must internally buffer each item until the
last iterator has yielded it; if the most and least advanced iterator differ
by most data, using a list is more efficient (but not lazy).
If the underlying iterable is concurrency safe (anext may be awaited
concurrently) the resulting iterators are concurrency safe as well. Otherwise,
the iterators are safe if there is only ever one single “most advanced” iterator.
To enforce sequential use of anext, provide a lock
- e.g. an asyncio.Lock instance in an asyncio application -
and access is automatically synchronised.
Methods
langchain_core.utils.utils.build_extra_kwargs¶
langchain_core.utils.utils.build_extra_kwargs(extra_kwargs: Dict[str, Any], values: Dict[str, Any], all_required_field_names: Set[str]) → Dict[str, Any][source]¶
Build extra kwargs from values and extra_kwargs.
Parameters
extra_kwargs – Extra kwargs passed in by user.
values – Values passed in by user.
all_required_field_names – All required field names for the pydantic class.
langchain_core.utils.aiter.py_anext¶
langchain_core.utils.aiter.py_anext(iterator: ~typing.AsyncIterator[~langchain_core.utils.aiter.T], default: ~typing.Union[~langchain_core.utils.aiter.T, ~typing.Any] = <object object>) → Awaitable[Union[T, None, Any]][source]¶
Pure-Python implementation of anext() for testing purposes.
Closely matches the builtin anext() C implementation.
Can be used to compare the built-in implementation of the inner
coroutines machinery to C-implementation of __anext__() and send()
or throw() on the returned generator.
langchain.utils.ernie_functions.convert_pydantic_to_ernie_tool¶
langchain.utils.ernie_functions.convert_pydantic_to_ernie_tool(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → ToolDescription[source]¶
Converts a Pydantic model to a function description for the Ernie API.
langchain_core.utils.html.extract_sub_links¶
langchain_core.utils.html.extract_sub_links(raw_html: str, url: str, *, base_url: Optional[str] = None, pattern: Optional[Union[str, Pattern]] = None, prevent_outside: bool = True, exclude_prefixes: Sequence[str] = ()) → List[str][source]¶
Extract all links from a raw html string and convert into absolute paths.
Parameters
raw_html – original html.
url – the url of the html.
base_url – the base url to check for outside links against.
pattern – Regex to use for extracting links from raw html.
prevent_outside – If True, ignore external links which are not children
of the base url.
exclude_prefixes – Exclude any URLs that start with one of these prefixes.
Returns
sub links
Return type
List[str]
langchain_core.utils.html.find_all_links¶
langchain_core.utils.html.find_all_links(raw_html: str, *, pattern: Optional[Union[str, Pattern]] = None) → List[str][source]¶
Extract all links from a raw html string.
Parameters
raw_html – original html.
pattern – Regex to use for extracting links from raw html.
Returns
all links
Return type
List[str]
langchain_community.utils.math.cosine_similarity_top_k¶
langchain_community.utils.math.cosine_similarity_top_k(X: Union[List[List[float]], List[ndarray], ndarray], Y: Union[List[List[float]], List[ndarray], ndarray], top_k: Optional[int] = 5, score_threshold: Optional[float] = None) → Tuple[List[Tuple[int, int]], List[float]][source]¶
Row-wise cosine similarity with optional top-k and score threshold filtering.
Parameters
X – Matrix.
Y – Matrix, same width as X.
top_k – Max number of results to return.
score_threshold – Minimum cosine similarity of results.
Returns
Tuple of two lists. First contains two-tuples of indices (X_idx, Y_idx),second contains corresponding cosine similarities.
langchain_experimental.smart_llm.base.SmartLLMChain¶
class langchain_experimental.smart_llm.base.SmartLLMChain[source]¶
Bases: Chain
Generalized implementation of SmartGPT (origin: https://youtu.be/wVzuvf9D9BU)
A SmartLLMChain is an LLMChain that instead of simply passing the prompt to the LLM
performs these 3 steps:
1. Ideate: Pass the user prompt to an ideation LLM n_ideas times,
each result is an “idea”
Critique: Pass the ideas to a critique LLM which looks for flaws in the ideas
& picks the best one
Resolve: Pass the critique to a resolver LLM which improves upon the best idea
& outputs only the (improved version of) the best output
In total, SmartLLMChain pass will use n_ideas+2 LLM calls
Note that SmartLLMChain will only improve results (compared to a basic LLMChain),
when the underlying models have the capability for reflection, which smaller models
often don’t.
Finally, a SmartLLMChain assumes that each underlying LLM outputs exactly 1 result.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param callback_manager: Optional[BaseCallbackManager] = None¶
Deprecated, use callbacks instead.
param callbacks: Callbacks = None¶
Optional list of callback handlers (or callback manager). Defaults to None.
Callback handlers are called throughout the lifecycle of a call to a chain,
starting with on_chain_start, ending with on_chain_end or on_chain_error.
Each custom chain can optionally call additional callback methods, see Callback docs
for full details.
Each custom chain can optionally call additional callback methods, see Callback docs
for full details.
param critique_llm: Optional[langchain_core.language_models.base.BaseLanguageModel] = None¶
LLM to use in critique step. If None given, ‘llm’ will be used.
param history: langchain_experimental.smart_llm.base.SmartLLMChain.SmartLLMChainHistory = <langchain_experimental.smart_llm.base.SmartLLMChain.SmartLLMChainHistory object>¶
param ideation_llm: Optional[langchain_core.language_models.base.BaseLanguageModel] = None¶
LLM to use in ideation step. If None given, ‘llm’ will be used.
param llm: Optional[langchain_core.language_models.base.BaseLanguageModel] = None¶
LLM to use for each steps, if no specific llm for that step is given.
param memory: Optional[BaseMemory] = None¶
Optional memory object. Defaults to None.
Memory is a class that gets called at the start
and at the end of every chain. At the start, memory loads variables and passes
them along in the chain. At the end, it saves any returned variables.
There are many different types of memory - please see memory docs
for the full catalog.
param metadata: Optional[Dict[str, Any]] = None¶
Optional metadata associated with the chain. Defaults to None.
This metadata will be associated with each call to this chain,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a chain with its use case.
param n_ideas: int = 3¶
Number of ideas to generate in idea step
param output_key: str = 'resolution'¶
param prompt: langchain_core.prompts.base.BasePromptTemplate [Required]¶
param prompt: langchain_core.prompts.base.BasePromptTemplate [Required]¶
Prompt object to use.
param resolver_llm: Optional[langchain_core.language_models.base.BaseLanguageModel] = None¶
LLM to use in resolve step. If None given, ‘llm’ will be used.
param return_intermediate_steps: bool = False¶
Whether to return ideas and critique, in addition to resolution.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the chain. Defaults to None.
These tags will be associated with each call to this chain,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a chain with its use case.
param verbose: bool [Optional]¶
Whether or not run in verbose mode. In verbose mode, some intermediate logs
will be printed to the console. Defaults to the global verbose value,
accessible via langchain.globals.get_verbose().
class SmartLLMChainHistory[source]¶
Bases: object
critique_prompt_inputs() → Dict[str, Any][source]¶
ideation_prompt_inputs() → Dict[str, Any][source]¶
resolve_prompt_inputs() → Dict[str, Any][source]¶
critique: str = ''¶
ideas: List[str] = []¶
property n_ideas: int¶
question: str = ''¶
__call__(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, include_run_info: bool = False) → Dict[str, Any]¶
Execute the chain.
Parameters
Execute the chain.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param. Should contain all inputs specified in
Chain.input_keys except for inputs that will be set by the chain’s
memory.
return_only_outputs – Whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. These will be called in
addition to callbacks passed to the chain during construction, but only
these runtime callbacks will propagate to calls to other objects.
tags – List of string tags to pass to all callbacks. These will be passed in
addition to tags passed to the chain during construction, but only
these runtime tags will propagate to calls to other objects.
metadata – Optional metadata associated with the chain. Defaults to None
include_run_info – Whether to include run info in the response. Defaults
to False.
Returns
A dict of named outputs. Should contain all outputs specified inChain.output_keys.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
e.g., if the underlying runnable uses an API which supports a batch mode.
async acall(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, include_run_info: bool = False) → Dict[str, Any]¶
Asynchronously execute the chain.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param. Should contain all inputs specified in
Chain.input_keys except for inputs that will be set by the chain’s
memory.
return_only_outputs – Whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. These will be called in
addition to callbacks passed to the chain during construction, but only
these runtime callbacks will propagate to calls to other objects.
tags – List of string tags to pass to all callbacks. These will be passed in
addition to tags passed to the chain during construction, but only
these runtime tags will propagate to calls to other objects.
metadata – Optional metadata associated with the chain. Defaults to None
include_run_info – Whether to include run info in the response. Defaults
to False.
Returns
A dict of named outputs. Should contain all outputs specified inChain.output_keys.
async ainvoke(input: Dict[str, Any], config: Optional[RunnableConfig] = None, **kwargs: Any) → Dict[str, Any]¶
Default implementation of ainvoke, calls invoke from a thread.
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
apply(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[Dict[str, str]]¶
Call the chain on all inputs in the list.
async arun(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Convenience method for executing chain.
The main difference between this method and Chain.__call__ is that this
method expects inputs to be passed directly in as positional arguments or
keyword arguments, whereas Chain.__call__ expects a single input dictionary
with all the inputs
Parameters
*args – If the chain expects a single input, it can be passed in as the
sole positional argument.
callbacks – Callbacks to use for this chain run. These will be called in
addition to callbacks passed to the chain during construction, but only
these runtime callbacks will propagate to calls to other objects.
tags – List of string tags to pass to all callbacks. These will be passed in
addition to tags passed to the chain during construction, but only
these runtime tags will propagate to calls to other objects.
**kwargs – If the chain expects multiple inputs, they can be passed in
directly as keyword arguments.
Returns
The chain output.
Example
# Suppose we have a single-input chain that takes a 'question' string:
await chain.arun("What's the temperature in Boise, Idaho?")
# -> "The temperature in Boise is..."