Tony20100/langchain_full huggingface dataset:
into a Document.
Attributes
load_max_docs
The maximum number of documents to load.
sample_to_document_function
Custom function that transform a dataset sample into a Document.
Methods
__init__(dataset_name, split_name[, ...])
Initialize the TensorflowDatasetLoader.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
A lazy loader for Documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
__init__(dataset_name: str, split_name: str, load_max_docs: Optional[int] = 100, sample_to_document_function: Optional[Callable[[Dict], Document]] = None)[source]¶
Initialize the TensorflowDatasetLoader.
Parameters
dataset_name (str) – the name of the dataset to load
split_name (str) – the name of the split to load.
load_max_docs (Optional[int]) – a limit to the number of loaded documents. Defaults to 100.
sample_to_document_function (Optional[Callable[[Dict], Document]]) – a function that converts a dataset sample
into a Document.
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document][source]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using TensorflowDatasetLoader¶
TensorFlow Datasets
langchain_google_vertexai.vision_models.VertexAIVisualQnAChat¶
class langchain_google_vertexai.vision_models.VertexAIVisualQnAChat[source]¶
Bases: _BaseImageTextModel, BaseChatModel
Chat implementation of a visual QnA model
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param cached_client: Any = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED] Callback manager to add to the run trace.
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param language: str = 'en'¶
Language of the query
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_name: str = 'imagetext@001'¶
Name of the model to use
param number_of_results: int = 1¶
Number of results to return from one query
param project: Union[str, None] = None¶
Google cloud project
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Any¶
Get the output type for this runnable.
property client: ImageTextModel¶
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VertexAIVisualQnAChat¶
Google Imagen
langchain_google_vertexai.vision_models.VertexAIImageEditorChat¶
class langchain_google_vertexai.vision_models.VertexAIImageEditorChat[source]¶
Bases: _BaseVertexAIImageGenerator, BaseChatModel
Given an image and a prompt, edits the image.
Currently only supports mask free editing.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param cached_client: Any = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED] Callback manager to add to the run trace.
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param guidance_scale: Union[float, None] = None¶
Controls the strength of the prompt
param language: Union[str, None] = None¶
Language of the text prompt for the image Supported values are “en” for English,
“hi” for Hindi, “ja” for Japanese, “ko” for Korean, and “auto” for automatic
language detection
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_name: str = 'imagegeneration@002'¶
Name of the base model
param model_name: str = 'imagegeneration@002'¶
Name of the base model
param negative_prompt: Union[str, None] = None¶
A description of what you want to omit in
the generated images
param number_of_results: int = 1¶
Number of images to generate
param project: Union[str, None] = None¶
Google cloud project id
param seed: Union[int, None] = None¶
Random seed for the image generation
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(messages: List[BaseMessage], stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → BaseMessage¶
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
messages (List[BaseMessage]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
BaseMessage
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Input]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) –
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
List[Output]
async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]¶
Run ainvoke in parallel on a list of inputs,
yielding results as they complete.
Parameters
inputs (Sequence[Input]) – 
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Tuple[int, Union[Output, Exception]]]
async agenerate(messages: List[List[BaseMessage]], stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, run_id: Optional[UUID] = None, **kwargs: Any) → LLMResult¶
Asynchronously pass a sequence of prompts to a model and return generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
messages (List[List[BaseMessage]]) – List of list of messages.
Parameters
messages (List[List[BaseMessage]]) – List of list of messages.
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
run_name (Optional[str]) – 
run_id (Optional[UUID]) – 
**kwargs – 
Returns
An LLMResult, which contains a list of candidate Generations for each inputprompt and additional model provider-specific output.
Return type
LLMResult
async agenerate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → LLMResult¶
Asynchronously pass a sequence of prompts and return model generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
prompts (List[PromptValue]) – List of PromptValues. A PromptValue is an object that can be
converted to match the format of any language model (string for pure
text generation models and BaseMessages for chat models).
text generation models and BaseMessages for chat models).
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
Returns
An LLMResult, which contains a list of candidate Generations for each inputprompt and additional model provider-specific output.
Return type
LLMResult
async ainvoke(input: LanguageModelInput, config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → BaseMessage¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
Parameters
input (LanguageModelInput) – 
config (Optional[RunnableConfig]) – 
stop (Optional[List[str]]) – 
kwargs (Any) – 
Return type
BaseMessage
async apredict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → str¶
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use ainvoke instead.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → BaseMessage¶
[Deprecated]
Notes
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use ainvoke instead.
Parameters
messages (List[BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
BaseMessage
assign(**kwargs: Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any], Mapping[str, Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any]]]]) → RunnableSerializable[Any, Any]¶
Assigns new fields to the dict output of this runnable.
Returns a new runnable.
from langchain_community.llms.fake import FakeStreamingListLLM
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import SystemMessagePromptTemplate
from langchain_core.runnables import Runnable
from operator import itemgetter
prompt = (
    SystemMessagePromptTemplate.from_template("You are a nice assistant.")
    + "{question}"
)
llm = FakeStreamingListLLM(responses=["foo-lish"])
chain: Runnable = prompt | llm | {"str": StrOutputParser()}
chain_with_assign = chain.assign(hello=itemgetter("str") | llm)
print(chain_with_assign.input_schema.schema())
# {'title': 'PromptInput', 'type': 'object', 'properties':
{'question': {'title': 'Question', 'type': 'string'}}}
print(chain_with_assign.output_schema.schema()) #
{'title': 'RunnableSequenceOutput', 'type': 'object', 'properties':
{'str': {'title': 'Str',
'type': 'string'}, 'hello': {'title': 'Hello', 'type': 'string'}}}
Parameters
Parameters
kwargs (Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any], Mapping[str, Union[Runnable[Dict[str, Any], Any], Callable[[Dict[str, Any]], Any]]]]) – 
Return type
RunnableSerializable[Any, Any]
async astream(input: LanguageModelInput, config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → AsyncIterator[BaseMessageChunk]¶
Default implementation of astream, which calls ainvoke.
Subclasses should override this method if they support streaming output.
Parameters
input (LanguageModelInput) – 
config (Optional[RunnableConfig]) – 
stop (Optional[List[str]]) – 
kwargs (Any) – 
Return type
AsyncIterator[BaseMessageChunk]
astream_events(input: Any, config: Optional[RunnableConfig] = None, *, version: Literal['v1', 'v2'], include_names: Optional[Sequence[str]] = None, include_types: Optional[Sequence[str]] = None, include_tags: Optional[Sequence[str]] = None, exclude_names: Optional[Sequence[str]] = None, exclude_types: Optional[Sequence[str]] = None, exclude_tags: Optional[Sequence[str]] = None, **kwargs: Any) → AsyncIterator[StreamEvent]¶
[Beta] Generate a stream of events.
Use to create an iterator over StreamEvents that provide real-time information
about the progress of the runnable, including StreamEvents from intermediate
results.
A StreamEvent is a dictionary with the following schema:
event: str - Event names are of theformat: on_[runnable_type]_(start|stream|end).
name: str - The name of the runnable that generated the event.
Not implemented on this class.
Parameters
schema (Union[Dict, Type[BaseModel]]) – 
kwargs (Any) – 
Return type
Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Any¶
Get the output type for this runnable.
property client: ImageGenerationModel¶
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VertexAIImageEditorChat¶
Google Imagen
langchain_google_vertexai.vision_models.VertexAIImageCaptioningChat¶
class langchain_google_vertexai.vision_models.VertexAIImageCaptioningChat[source]¶
Bases: _BaseVertexAIImageCaptioning, BaseChatModel
Implementation of the Image Captioning model as a chat.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param cached_client: Any = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED] Callback manager to add to the run trace.
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param language: str = 'en'¶
Language of the query
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_name: str = 'imagetext@001'¶
Name of the model to use
param number_of_results: int = 1¶
Number of results to return from one query
param project: Union[str, None] = None¶
Google cloud project
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Any¶
Get the output type for this runnable.
property client: ImageTextModel¶
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
langchain_google_vertexai.vision_models.VertexAIImageGeneratorChat¶
class langchain_google_vertexai.vision_models.VertexAIImageGeneratorChat[source]¶
Bases: _BaseVertexAIImageGenerator, BaseChatModel
Generates an image from a prompt.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param cached_client: Any = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED] Callback manager to add to the run trace.
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param guidance_scale: Union[float, None] = None¶
Controls the strength of the prompt
param language: Union[str, None] = None¶
Language of the text prompt for the image Supported values are “en” for English,
“hi” for Hindi, “ja” for Japanese, “ko” for Korean, and “auto” for automatic
language detection
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_name: str = 'imagegeneration@002'¶
Name of the base model
param negative_prompt: Union[str, None] = None¶
A description of what you want to omit in
A description of what you want to omit in
the generated images
param number_of_results: int = 1¶
Number of images to generate
param project: Union[str, None] = None¶
Google cloud project id
param seed: Union[int, None] = None¶
Random seed for the image generation
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(messages: List[BaseMessage], stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → BaseMessage¶
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
messages (List[BaseMessage]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
BaseMessage
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Input]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
List[Output]
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Any¶
Get the output type for this runnable.
property client: ImageGenerationModel¶
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VertexAIImageGeneratorChat¶
Google Imagen