Tony20100/langchain_full huggingface dataset:
langchain_prompty.core.TemplateSettings¶
class langchain_prompty.core.TemplateSettings[source]¶
Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param parser: str = ''¶
param type: str = 'mustache'¶
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
langchain_text_splitters.konlpy.KonlpyTextSplitter¶
class langchain_text_splitters.konlpy.KonlpyTextSplitter(separator: str = '\n\n', **kwargs: Any)[source]¶
Splitting text using Konlpy package.
It is good for splitting Korean text.
Initialize the Konlpy text splitter.
Methods
__init__([separator])
Initialize the Konlpy text splitter.
atransform_documents(documents, **kwargs)
Asynchronously transform a list of documents.
create_documents(texts[, metadatas])
Create documents from a list of texts.
from_huggingface_tokenizer(tokenizer, **kwargs)
Text splitter that uses HuggingFace tokenizer to count length.
from_tiktoken_encoder([encoding_name, ...])
Text splitter that uses tiktoken encoder to count length.
split_documents(documents)
Split documents.
split_text(text)
Split incoming text and return chunks.
transform_documents(documents, **kwargs)
Transform sequence of documents by splitting them.
Parameters
separator (str) – 
kwargs (Any) – 
__init__(separator: str = '\n\n', **kwargs: Any) → None[source]¶
Initialize the Konlpy text splitter.
Parameters
separator (str) – 
kwargs (Any) – 
Return type
None
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document]¶
Asynchronously transform a list of documents.
Parameters
documents (Sequence[Document]) – A sequence of Documents to be transformed.
kwargs (Any) – 
Returns
A list of transformed Documents.
Return type
Sequence[Document]
langchain_core.utils.json.parse_partial_json¶
langchain_core.utils.json.parse_partial_json(s: str, *, strict: bool = False) → Any[source]¶
Parse a JSON string that may be missing closing braces.
Parameters
s (str) – The JSON string to parse.
strict (bool) – Whether to use strict parsing. Defaults to False.
Returns
The parsed JSON object as a Python dictionary.
Return type
Any
langchain_core.utils.aiter.Tee¶
class langchain_core.utils.aiter.Tee(iterable: AsyncIterator[T], n: int = 2, *, lock: Optional[AsyncContextManager[Any]] = None)[source]¶
Create n separate asynchronous iterators over iterable.
This splits a single iterable into multiple iterators, each providing
the same items in the same order.
All child iterators may advance separately but share the same items
from iterable – when the most advanced iterator retrieves an item,
it is buffered until the least advanced iterator has yielded it as well.
A tee works lazily and can handle an infinite iterable, provided
that all iterators advance.
async def derivative(sensor_data):
    previous, current = a.tee(sensor_data, n=2)
    await a.anext(previous)  # advance one iterator
    return a.map(operator.sub, previous, current)
Unlike itertools.tee(), tee() returns a custom type instead
of a tuple. Like a tuple, it can be indexed, iterated and unpacked
to get the child iterators. In addition, its aclose() method
immediately closes all children, and it can be used in an async with context
for the same effect.
If iterable is an iterator and read elsewhere, tee will not
provide these items. Also, tee must internally buffer each item until the
last iterator has yielded it; if the most and least advanced iterator differ
by most data, using a list is more efficient (but not lazy).
If the underlying iterable is concurrency safe (anext may be awaited
concurrently) the resulting iterators are concurrency safe as well. Otherwise,
the iterators are safe if there is only ever one single “most advanced” iterator.
To enforce sequential use of anext, provide a lock
- e.g. an asyncio.Lock instance in an asyncio application -
and access is automatically synchronised.
and access is automatically synchronised.
Methods
__init__(iterable[, n, lock])
aclose()
Parameters
iterable (AsyncIterator[T]) – 
n (int) – 
lock (Optional[AsyncContextManager[Any]]) – 
__init__(iterable: AsyncIterator[T], n: int = 2, *, lock: Optional[AsyncContextManager[Any]] = None)[source]¶
Parameters
iterable (AsyncIterator[T]) – 
n (int) – 
lock (Optional[AsyncContextManager[Any]]) – 
async aclose() → None[source]¶
Return type
None
langchain_community.utils.math.cosine_similarity_top_k¶
langchain_community.utils.math.cosine_similarity_top_k(X: Union[List[List[float]], List[ndarray], ndarray], Y: Union[List[List[float]], List[ndarray], ndarray], top_k: Optional[int] = 5, score_threshold: Optional[float] = None) → Tuple[List[Tuple[int, int]], List[float]][source]¶
Row-wise cosine similarity with optional top-k and score threshold filtering.
Parameters
X (Union[List[List[float]], List[ndarray], ndarray]) – Matrix.
Y (Union[List[List[float]], List[ndarray], ndarray]) – Matrix, same width as X.
top_k (Optional[int]) – Max number of results to return.
score_threshold (Optional[float]) – Minimum cosine similarity of results.
Returns
Tuple of two lists. First contains two-tuples of indices (X_idx, Y_idx),second contains corresponding cosine similarities.
Return type
Tuple[List[Tuple[int, int]], List[float]]
langchain_core.utils.env.get_from_dict_or_env¶
langchain_core.utils.env.get_from_dict_or_env(data: Dict[str, Any], key: str, env_key: str, default: Optional[str] = None) → str[source]¶
Get a value from a dictionary or an environment variable.
Parameters
data (Dict[str, Any]) – 
key (str) – 
env_key (str) – 
default (Optional[str]) – 
Return type
str
langchain_core.utils.function_calling.convert_pydantic_to_openai_function¶
langchain_core.utils.function_calling.convert_pydantic_to_openai_function(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None, rm_titles: bool = True) → FunctionDescription[source]¶
[Deprecated] Converts a Pydantic model to a function description for the OpenAI API.
Notes
Deprecated since version 0.1.16: Use langchain_core.utils.function_calling.convert_to_openai_function() instead.
Parameters
model (Type[BaseModel]) – 
name (Optional[str]) – 
description (Optional[str]) – 
rm_titles (bool) – 
Return type
FunctionDescription
Examples using convert_pydantic_to_openai_function¶
LangSmith LLM Runs
langchain_core.utils.input.get_color_mapping¶
langchain_core.utils.input.get_color_mapping(items: List[str], excluded_colors: Optional[List] = None) → Dict[str, str][source]¶
Get mapping for items to a support color.
Parameters
items (List[str]) – 
excluded_colors (Optional[List]) – 
Return type
Dict[str, str]
langchain_core.utils.function_calling.convert_to_openai_function¶
langchain_core.utils.function_calling.convert_to_openai_function(function: Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]) → Dict[str, Any][source]¶
Convert a raw function/class to an OpenAI function.
Parameters
function (Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]) – Either a dictionary, a pydantic.BaseModel class, or a Python function.
If a dictionary is passed in, it is assumed to already be a valid OpenAI
function or a JSON schema with top-level ‘title’ and ‘description’ keys
specified.
Returns
A dict version of the passed in function which is compatible with theOpenAI function-calling API.
Return type
Dict[str, Any]
Examples using convert_to_openai_function¶
How to add a semantic layer over graph database
How to convert tools to OpenAI Functions
langchain_milvus.utils.sparse.BaseSparseEmbedding¶
class langchain_milvus.utils.sparse.BaseSparseEmbedding[source]¶
Interface for Sparse embedding models.
You can inherit from it and implement your custom sparse embedding model.
Methods
__init__()
embed_documents(texts)
Embed search docs.
embed_query(query)
Embed query text.
__init__()¶
abstract embed_documents(texts: List[str]) → List[Dict[int, float]][source]¶
Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[Dict[int, float]]
abstract embed_query(query: str) → Dict[int, float][source]¶
Embed query text.
Parameters
query (str) – 
Return type
Dict[int, float]
langchain_core.utils.mustache.r_sa_check¶
langchain_core.utils.mustache.r_sa_check(template: str, tag_type: str, is_standalone: bool) → bool[source]¶
Do a final check to see if a tag could be a standalone.
Parameters
template (str) – 
tag_type (str) – 
is_standalone (bool) – 
Return type
bool
langchain_core.utils.function_calling.ToolDescription¶
class langchain_core.utils.function_calling.ToolDescription[source]¶
Representation of a callable function to the OpenAI API.
Attributes
type
function
Methods
__init__(*args, **kwargs)
clear()
copy()
fromkeys([value])
Create a new dictionary with keys from iterable and values set to value.
get(key[, default])
Return the value for key if key is in the dictionary, else default.
items()
keys()
pop(k[,d])
If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem()
Remove and return a (key, value) pair as a 2-tuple.
setdefault(key[, default])
Insert key with a value of default if key is not in the dictionary.
update([E, ]**F)
If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v In either case, this is followed by: for k in F:  D[k] = F[k]
values()
__init__(*args, **kwargs)¶
clear() → None.  Remove all items from D.¶
copy() → a shallow copy of D¶
fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
get(key, default=None, /)¶
Return the value for key if key is in the dictionary, else default.
items() → a set-like object providing a view on D's items¶
keys() → a set-like object providing a view on D's keys¶
langchain_core.utils.json_schema.dereference_refs¶
langchain_core.utils.json_schema.dereference_refs(schema_obj: dict, *, full_schema: Optional[dict] = None, skip_keys: Optional[Sequence[str]] = None) → dict[source]¶
Try to substitute $refs in JSON Schema.
Parameters
schema_obj (dict) – 
full_schema (Optional[dict]) – 
skip_keys (Optional[Sequence[str]]) – 
Return type
dict
langchain_core.utils.function_calling.FunctionDescription¶
class langchain_core.utils.function_calling.FunctionDescription[source]¶
Representation of a callable function to send to an LLM.
Attributes
name
The name of the function.
description
A description of the function.
parameters
The parameters of the function.
Methods
__init__(*args, **kwargs)
clear()
copy()
fromkeys([value])
Create a new dictionary with keys from iterable and values set to value.
get(key[, default])
Return the value for key if key is in the dictionary, else default.
items()
keys()
pop(k[,d])
If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem()
Remove and return a (key, value) pair as a 2-tuple.
setdefault(key[, default])
Insert key with a value of default if key is not in the dictionary.
update([E, ]**F)
If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v In either case, this is followed by: for k in F:  D[k] = F[k]
values()
__init__(*args, **kwargs)¶
clear() → None.  Remove all items from D.¶
copy() → a shallow copy of D¶
fromkeys(value=None, /)¶
Create a new dictionary with keys from iterable and values set to value.
get(key, default=None, /)¶
Return the value for key if key is in the dictionary, else default.
items() → a set-like object providing a view on D's items¶
langchain_core.utils.mustache.parse_tag¶
langchain_core.utils.mustache.parse_tag(template: str, l_del: str, r_del: str) → Tuple[Tuple[str, str], str][source]¶
Parse a tag from a template.
Parameters
template (str) – 
l_del (str) – 
r_del (str) – 
Return type
Tuple[Tuple[str, str], str]
langchain_core.utils.iter.tee_peer¶
langchain_core.utils.iter.tee_peer(iterator: Iterator[T], buffer: Deque[T], peers: List[Deque[T]], lock: ContextManager[Any]) → Generator[T, None, None][source]¶
An individual iterator of a tee()
Parameters
iterator (Iterator[T]) – 
buffer (Deque[T]) – 
peers (List[Deque[T]]) – 
lock (ContextManager[Any]) – 
Return type
Generator[T, None, None]
langchain_core.utils.html.find_all_links¶
langchain_core.utils.html.find_all_links(raw_html: str, *, pattern: Optional[Union[str, Pattern]] = None) → List[str][source]¶
Extract all links from a raw html string.
Parameters
raw_html (str) – original html.
pattern (Optional[Union[str, Pattern]]) – Regex to use for extracting links from raw html.
Returns
all links
Return type
List[str]
langchain_core.utils.utils.guard_import¶
langchain_core.utils.utils.guard_import(module_name: str, *, pip_name: Optional[str] = None, package: Optional[str] = None) → Any[source]¶
Dynamically import a module and raise an exception if the module is not
installed.
Parameters
module_name (str) – 
pip_name (Optional[str]) – 
package (Optional[str]) – 
Return type
Any
langchain_core.utils.loading.try_load_from_hub¶
langchain_core.utils.loading.try_load_from_hub(*args: Any, **kwargs: Any) → Any[source]¶
[Deprecated]
Notes
Deprecated since version 0.1.30: Using the hwchase17/langchain-hub repo for prompts is deprecated. Please use https://smith.langchain.com/hub instead.
Parameters
args (Any) – 
kwargs (Any) – 
Return type
Any
langchain_core.utils.function_calling.convert_to_openai_tool¶
langchain_core.utils.function_calling.convert_to_openai_tool(tool: Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]) → Dict[str, Any][source]¶
Convert a raw function/class to an OpenAI tool.
Parameters
tool (Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]) – Either a dictionary, a pydantic.BaseModel class, Python function, or
BaseTool. If a dictionary is passed in, it is assumed to already be a valid
OpenAI tool, OpenAI function, or a JSON schema with top-level ‘title’ and
‘description’ keys specified.
Returns
A dict version of the passed in tool which is compatible with theOpenAI tool-calling API.
Return type
Dict[str, Any]
langchain_core.utils.function_calling.convert_python_function_to_openai_function¶
langchain_core.utils.function_calling.convert_python_function_to_openai_function(function: Callable) → Dict[str, Any][source]¶
[Deprecated] Convert a Python function to an OpenAI function-calling API compatible dict.
Assumes the Python function has type hints and a docstring with a description. Ifthe docstring has Google Python style argument descriptions, these will be
included as well.
Notes
Deprecated since version 0.1.16: Use langchain_core.utils.function_calling.convert_to_openai_function() instead.
Parameters
function (Callable) – 
Return type
Dict[str, Any]
langchain_community.utils.ernie_functions.convert_pydantic_to_ernie_function¶
langchain_community.utils.ernie_functions.convert_pydantic_to_ernie_function(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → FunctionDescription[source]¶
Convert a Pydantic model to a function description for the Ernie API.
Parameters
model (Type[BaseModel]) – 
name (Optional[str]) – 
description (Optional[str]) – 
Return type
FunctionDescription
langchain_milvus.utils.sparse.BM25SparseEmbedding¶
class langchain_milvus.utils.sparse.BM25SparseEmbedding(corpus: List[str], language: str = 'en')[source]¶
This is a class that inherits BaseSparseEmbedding
and implements a sparse vector embedding model based on BM25.
This class uses the BM25 model in Milvus model to implement sparse vector embedding.
This model requires pymilvus[model] to be installed.
pip install pymilvus[model]
For more information please refer to:
https://milvus.io/docs/embed-with-bm25.md
Methods
__init__(corpus[, language])
embed_documents(texts)
Embed search docs.
embed_query(text)
Embed query text.
Parameters
corpus (List[str]) – 
language (str) – 
__init__(corpus: List[str], language: str = 'en')[source]¶
Parameters
corpus (List[str]) – 
language (str) – 
embed_documents(texts: List[str]) → List[Dict[int, float]][source]¶
Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[Dict[int, float]]
embed_query(text: str) → Dict[int, float][source]¶
Embed query text.
Parameters
text (str) – 
Return type
Dict[int, float]
langchain_community.utils.math.cosine_similarity¶
langchain_community.utils.math.cosine_similarity(X: Union[List[List[float]], List[ndarray], ndarray], Y: Union[List[List[float]], List[ndarray], ndarray]) → ndarray[source]¶
Row-wise cosine similarity between two equal-width matrices.
Parameters
X (Union[List[List[float]], List[ndarray], ndarray]) – 
Y (Union[List[List[float]], List[ndarray], ndarray]) – 
Return type
ndarray
Examples using cosine_similarity¶
How to route between sub-chains
langchain_core.utils.aiter.tee_peer¶
async langchain_core.utils.aiter.tee_peer(iterator: AsyncIterator[T], buffer: Deque[T], peers: List[Deque[T]], lock: AsyncContextManager[Any]) → AsyncGenerator[T, None][source]¶
An individual iterator of a tee()
Parameters
iterator (AsyncIterator[T]) – 
buffer (Deque[T]) – 
peers (List[Deque[T]]) – 
lock (AsyncContextManager[Any]) – 
Return type
AsyncGenerator[T, None]
langchain_community.utils.google.get_client_info¶
langchain_community.utils.google.get_client_info(module: Optional[str] = None) → Any[source]¶
Return a custom user agent header.
Parameters
module (Optional[str]) – Optional. The module for a custom user agent header.
Returns
google.api_core.gapic_v1.client_info.ClientInfo
Return type
Any
langchain_core.utils.iter.batch_iterate¶
langchain_core.utils.iter.batch_iterate(size: Optional[int], iterable: Iterable[T]) → Iterator[List[T]][source]¶
Utility batching function.
Parameters
size (Optional[int]) – The size of the batch. If None, returns a single batch.
iterable (Iterable[T]) – The iterable to batch.
Returns
An iterator over the batches.
Return type
Iterator[List[T]]
langchain_core.utils.utils.build_extra_kwargs¶
langchain_core.utils.utils.build_extra_kwargs(extra_kwargs: Dict[str, Any], values: Dict[str, Any], all_required_field_names: Set[str]) → Dict[str, Any][source]¶
Build extra kwargs from values and extra_kwargs.
Parameters
extra_kwargs (Dict[str, Any]) – Extra kwargs passed in by user.
values (Dict[str, Any]) – Values passed in by user.
all_required_field_names (Set[str]) – All required field names for the pydantic class.
Return type
Dict[str, Any]
langchain_core.utils.mustache.render¶
langchain_core.utils.mustache.render(template: Union[str, List[Tuple[str, str]]] = '', data: Dict[str, Any] = {}, partials_dict: Dict[str, str] = {}, padding: str = '', def_ldel: str = '{{', def_rdel: str = '}}', scopes: Optional[List[Union[Literal[False, 0], Dict[str, Any]]]] = None, warn: bool = False, keep: bool = False) → str[source]¶
Render a mustache template.
Renders a mustache template with a data scope and inline partial capability.
Arguments:
template      – A file-like object or a string containing the template
data          – A python dictionary with your data scope
partials_path – The path to where your partials are storedIf set to None, then partials won’t be loaded from the file system
(defaults to ‘.’)
partials_ext  – The extension that you want the parser to look for(defaults to ‘mustache’)
partials_dict – A python dictionary which will be search for partialsbefore the filesystem is. {‘include’: ‘foo’} is the same
as a file called include.mustache
(defaults to {})
padding       – This is for padding partials, and shouldn’t be used(but can be if you really want to)
def_ldel      – The default left delimiter(“{{” by default, as in spec compliant mustache)
def_rdel      – The default right delimiter(“}}” by default, as in spec compliant mustache)
scopes        – The list of scopes that get_key will look through
warn          – Log a warning when a template substitution isn’t found in the data
keep          – Keep unreplaced tags when a substitution isn’t found in the data
Returns:
A string containing the rendered template.
Parameters
Returns:
A string containing the rendered template.
Parameters
template (Union[str, List[Tuple[str, str]]]) – 
data (Dict[str, Any]) – 
partials_dict (Dict[str, str]) – 
padding (str) – 
def_ldel (str) – 
def_rdel (str) – 
scopes (Optional[List[Union[Literal[False, 0], ~typing.Dict[str, ~typing.Any]]]]) – 
warn (bool) – 
keep (bool) – 
Return type
str
langchain_prompty.utils.run¶
langchain_prompty.utils.run(prompt: Prompty, content: Union[Dict, List, str], configuration: Dict[str, Any] = {}, parameters: Dict[str, Any] = {}, raw: bool = False) → Any[source]¶
Parameters
prompt (Prompty) – 
content (Union[Dict, List, str]) – 
configuration (Dict[str, Any]) – 
parameters (Dict[str, Any]) – 
raw (bool) – 
Return type
Any
langchain_astradb.utils.mmr.maximal_marginal_relevance¶
langchain_astradb.utils.mmr.maximal_marginal_relevance(query_embedding: ndarray, embedding_list: list, lambda_mult: float = 0.5, k: int = 4) → List[int][source]¶
Calculate maximal marginal relevance.
Parameters
query_embedding (ndarray) – 
embedding_list (list) – 
lambda_mult (float) – 
k (int) – 
Return type
List[int]
langchain_core.utils.function_calling.convert_pydantic_to_openai_tool¶
langchain_core.utils.function_calling.convert_pydantic_to_openai_tool(model: Type[BaseModel], *, name: Optional[str] = None, description: Optional[str] = None) → ToolDescription[source]¶
[Deprecated] Converts a Pydantic model to a function description for the OpenAI API.
Notes
Deprecated since version 0.1.16: Use langchain_core.utils.function_calling.convert_to_openai_tool() instead.
Parameters
model (Type[BaseModel]) – 
name (Optional[str]) – 
description (Optional[str]) – 
Return type
ToolDescription
langchain_core.utils.html.extract_sub_links¶
langchain_core.utils.html.extract_sub_links(raw_html: str, url: str, *, base_url: Optional[str] = None, pattern: Optional[Union[str, Pattern]] = None, prevent_outside: bool = True, exclude_prefixes: Sequence[str] = (), continue_on_failure: bool = False) → List[str][source]¶
Extract all links from a raw html string and convert into absolute paths.
Parameters
raw_html (str) – original html.
url (str) – the url of the html.
base_url (Optional[str]) – the base url to check for outside links against.
pattern (Optional[Union[str, Pattern]]) – Regex to use for extracting links from raw html.
prevent_outside (bool) – If True, ignore external links which are not children
of the base url.
exclude_prefixes (Sequence[str]) – Exclude any URLs that start with one of these prefixes.
continue_on_failure (bool) – If True, continue if parsing a specific link raises an
exception. Otherwise, raise the exception.
Returns
sub links
Return type
List[str]
langchain_core.utils.function_calling.format_tool_to_openai_tool¶
langchain_core.utils.function_calling.format_tool_to_openai_tool(tool: BaseTool) → ToolDescription[source]¶
[Deprecated] Format tool into the OpenAI function API.
Notes
Deprecated since version 0.1.16: Use langchain_core.utils.function_calling.convert_to_openai_tool() instead.
Parameters
tool (BaseTool) – 
Return type
ToolDescription