Callbacks
Callback handlers that allow listening to events in LangChain.
class langchain.callbacks.AimCallbackHandler(repo=None, experiment_name=None, system_tracking_interval=10, log_system_params=True)[source]
Bases: langchain.callbacks.aim_callback.BaseMetadataCallbackHandler, langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to Aim.
Parameters
repo (str, optional) – Aim repository path or Repo object to which
Run object is bound. If skipped, default Repo is used.
experiment_name (str, optional) – Sets Run’s experiment property.
‘default’ if not specified. Can be used later to query runs/sequences.
system_tracking_interval (int, optional) – Sets the tracking interval
in seconds for system usage metrics (CPU, Memory, etc.). Set to None
to disable system metrics tracking.
log_system_params (bool, optional) – Enable/Disable logging of system
params such as installed packages, git info, environment variables, etc.
Return type
None
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run
and then logs the response to Aim.
setup(**kwargs)[source]
Parameters
kwargs (Any) – 
Return type
None
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run when LLM generates a new token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run when agent is ending.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run when agent ends running.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
flush_tracker(repo=None, experiment_name=None, system_tracking_interval=10, log_system_params=True, langchain_asset=None, reset=True, finish=False)[source]
Flush the tracker and reset the session.
Parameters
repo (str, optional) – Aim repository path or Repo object to which
Run object is bound. If skipped, default Repo is used.
experiment_name (str, optional) – Sets Run’s experiment property.
‘default’ if not specified. Can be used later to query runs/sequences.
system_tracking_interval (int, optional) – Sets the tracking interval
in seconds for system usage metrics (CPU, Memory, etc.). Set to None
to disable system metrics tracking.
log_system_params (bool, optional) – Enable/Disable logging of system
params such as installed packages, git info, environment variables, etc.
langchain_asset (Any) – The langchain asset to save.
reset (bool) – Whether to reset the session.
finish (bool) – Whether to finish the run.
Returns – None
Return type
None
class langchain.callbacks.ArgillaCallbackHandler(dataset_name, workspace_name=None, api_url=None, api_key=None)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs into Argilla.
Parameters
dataset_name (str) – name of the FeedbackDataset in Argilla. Note that it must
exist in advance. If you need help on how to create a FeedbackDataset in
Argilla, please visit
https://docs.argilla.io/en/latest/guides/llms/practical_guides/use_argilla_callback_in_langchain.html.
workspace_name (Optional[str]) – name of the workspace in Argilla where the specified
FeedbackDataset lives in. Defaults to None, which means that the
default workspace will be used.
api_url (Optional[str]) – URL of the Argilla Server that we want to use, and where the
FeedbackDataset lives in. Defaults to None, which means that either
ARGILLA_API_URL environment variable or the default http://localhost:6900
will be used.
api_key (Optional[str]) – API Key to connect to the Argilla Server. Defaults to None, which
means that either ARGILLA_API_KEY environment variable or the default
argilla.apikey will be used.
Raises
ImportError – if the argilla package is not installed.
ConnectionError – if the connection to Argilla fails.
FileNotFoundError – if the FeedbackDataset retrieval from Argilla fails.
Return type
None
Examples
>>> from langchain.llms import OpenAI
>>> from langchain.callbacks import ArgillaCallbackHandler
>>> argilla_callback = ArgillaCallbackHandler(
...     dataset_name="my-dataset",
...     workspace_name="my-workspace",
...     api_url="http://localhost:6900",
...     api_key="argilla.apikey",
... )
>>> llm = OpenAI(
...     temperature=0,
...     callbacks=[argilla_callback],
...     verbose=True,
...     openai_api_key="API_KEY_HERE",
... )
>>> llm.generate([
...     "What is the best NLP-annotation tool out there? (no bias at all)",
... ])
"Argilla, no doubt about it."
on_llm_start(serialized, prompts, **kwargs)[source]
Save the prompts in memory when an LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Do nothing when a new token is generated.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Log records to Argilla when an LLM ends.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Do nothing when LLM outputs an error.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
If the key input is in inputs, then save it in self.prompts using
either the parent_run_id or the run_id as the key. This is done so that
we don’t log the same input prompt twice, once when the LLM starts and once
when the chain starts.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
If either the parent_run_id or the run_id is in self.prompts, then
log the outputs to Argilla, and pop the run from self.prompts. The behavior
differs if the output is a list or not.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Do nothing when LLM chain outputs an error.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Do nothing when tool starts.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Do nothing when agent takes a specific action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, observation_prefix=None, llm_prefix=None, **kwargs)[source]
Do nothing when tool ends.
Parameters
output (str) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Do nothing when tool outputs an error.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Do nothing
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Do nothing
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.ArizeCallbackHandler(model_id=None, model_version=None, SPACE_KEY=None, API_KEY=None)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to Arize.
Parameters
model_id (Optional[str]) – 
model_version (Optional[str]) – 
SPACE_KEY (Optional[str]) – 
API_KEY (Optional[str]) – 
Return type
None
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts running.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Do nothing.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Do nothing.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Do nothing.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, observation_prefix=None, llm_prefix=None, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run on arbitrary text.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run on agent end.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.AsyncIteratorCallbackHandler[source]
Bases: langchain.callbacks.base.AsyncCallbackHandler
Callback handler that returns an async iterator.
Return type
None
property always_verbose: bool
queue: asyncio.queues.Queue[str]
done: asyncio.locks.Event
async on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts running.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
async on_llm_new_token(token, **kwargs)[source]
Run on new LLM token. Only available when streaming is enabled.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
async on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
async on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
async aiter()[source]
Return type
AsyncIterator[str]
class langchain.callbacks.ClearMLCallbackHandler(task_type='inference', project_name='langchain_callback_demo', tags=None, task_name=None, visualize=False, complexity_metrics=False, stream_logs=False)[source]
Bases: langchain.callbacks.utils.BaseMetadataCallbackHandler, langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to ClearML.
Parameters
job_type (str) – The type of clearml task such as “inference”, “testing” or “qc”
project_name (str) – The clearml project name
tags (list) – Tags to add to the task
task_name (str) – Name of the clearml task
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics
stream_logs (bool) – Whether to stream callback actions to ClearML
task_type (Optional[str]) – 
Return type
None
This handler will utilize the associated callback method and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response to the ClearML console.
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run when LLM generates a new token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run when agent is ending.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run when agent ends running.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
analyze_text(text)[source]
Analyze text using textstat and spacy.
Parameters
text (str) – The text to analyze.
Returns
A dictionary containing the complexity metrics.
Return type
(dict)
flush_tracker(name=None, langchain_asset=None, finish=False)[source]
Flush the tracker and setup the session.
Everything after this will be a new table.
Parameters
name (Optional[str]) – Name of the preformed session so far so it is identifyable
langchain_asset (Any) – The langchain asset to save.
finish (bool) – Whether to finish the run.
Returns – None
Return type
None
class langchain.callbacks.CometCallbackHandler(task_type='inference', workspace=None, project_name=None, tags=None, name=None, visualizations=None, complexity_metrics=False, custom_metrics=None, stream_logs=True)[source]
Bases: langchain.callbacks.utils.BaseMetadataCallbackHandler, langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to Comet.
Parameters
job_type (str) – The type of comet_ml task such as “inference”,
“testing” or “qc”
project_name (str) – The comet_ml project name
tags (list) – Tags to add to the task
task_name (str) – Name of the comet_ml task
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics
stream_logs (bool) – Whether to stream callback actions to Comet
task_type (Optional[str]) – 
workspace (Optional[str]) – 
name (Optional[str]) – 
visualizations (Optional[List[str]]) – 
custom_metrics (Optional[Callable]) – 
Return type
None
This handler will utilize the associated callback method and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response to Comet.
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run when LLM generates a new token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run when agent is ending.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run when agent ends running.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
flush_tracker(langchain_asset=None, task_type='inference', workspace=None, project_name='comet-langchain-demo', tags=None, name=None, visualizations=None, complexity_metrics=False, custom_metrics=None, finish=False, reset=False)[source]
Flush the tracker and setup the session.
Everything after this will be a new table.
Parameters
name (Optional[str]) – Name of the preformed session so far so it is identifyable
langchain_asset (Any) – The langchain asset to save.
finish (bool) – Whether to finish the run.
Returns – None
task_type (Optional[str]) – 
workspace (Optional[str]) – 
project_name (Optional[str]) – 
tags (Optional[Sequence]) – 
visualizations (Optional[List[str]]) – 
complexity_metrics (bool) – 
custom_metrics (Optional[Callable]) – 
reset (bool) – 
Return type
None
class langchain.callbacks.FileCallbackHandler(filename, mode='a', color=None)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that writes to a file.
Parameters
filename (str) – 
mode (str) – 
color (Optional[str]) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Print out that we are entering a chain.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Print out that we finished a chain.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, color=None, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, color=None, observation_prefix=None, llm_prefix=None, **kwargs)[source]
If not the final action, print out observation.
Parameters
output (str) – 
color (Optional[str]) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_text(text, color=None, end='', **kwargs)[source]
Run when agent ends.
Parameters
text (str) – 
color (Optional[str]) – 
end (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, color=None, **kwargs)[source]
Run on agent end.
Parameters
finish (langchain.schema.AgentFinish) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.FinalStreamingStdOutCallbackHandler(*, answer_prefix_tokens=None, strip_tokens=True, stream_prefix=False)[source]
Bases: langchain.callbacks.streaming_stdout.StreamingStdOutCallbackHandler
Callback handler for streaming in agents.
Only works with agents using LLMs that support streaming.
Only the final output of the agent will be streamed.
Parameters
answer_prefix_tokens (Optional[List[str]]) – 
strip_tokens (bool) – 
stream_prefix (bool) – 
Return type
None
append_to_last_tokens(token)[source]
Parameters
token (str) – 
Return type
None
check_if_answer_reached()[source]
Return type
bool
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts running.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run on new LLM token. Only available when streaming is enabled.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.HumanApprovalCallbackHandler(approve=<function _default_approve>, should_check=<function _default_true>)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback for manually validating values.
Parameters
approve (Callable[[Any], bool]) – 
should_check (Callable[[Dict[str, Any]], bool]) – 
raise_error: bool = True
on_tool_start(serialized, input_str, *, run_id, parent_run_id=None, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
run_id (uuid.UUID) – 
parent_run_id (Optional[uuid.UUID]) – 
kwargs (Any) – 
Return type
Any
class langchain.callbacks.InfinoCallbackHandler(model_id=None, model_version=None, verbose=False)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to Infino.
Parameters
model_id (Optional[str]) – 
model_version (Optional[str]) – 
verbose (bool) – 
Return type
None
on_llm_start(serialized, prompts, **kwargs)[source]
Log the prompts to Infino, and set start time and error flag.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Do nothing when a new token is generated.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Log the latency, error, token usage, and response to Infino.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Set the error flag.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Do nothing when LLM chain starts.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Do nothing when LLM chain ends.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Need to log the error.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Do nothing when tool starts.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Do nothing when agent takes a specific action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, observation_prefix=None, llm_prefix=None, **kwargs)[source]
Do nothing when tool ends.
Parameters
output (str) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Do nothing when tool outputs an error.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Do nothing.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Do nothing.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.MlflowCallbackHandler(name='langchainrun-%', experiment='langchain', tags={}, tracking_uri=None)[source]
Bases: langchain.callbacks.utils.BaseMetadataCallbackHandler, langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs metrics and artifacts to mlflow server.
Parameters
name (str) – Name of the run.
experiment (str) – Name of the experiment.
tags (dict) – Tags to be attached for the run.
tracking_uri (str) – MLflow tracking server uri.
Return type
None
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response to mlflow server.
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run when LLM generates a new token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run when agent is ending.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run when agent ends running.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
flush_tracker(langchain_asset=None, finish=False)[source]
Parameters
langchain_asset (Any) – 
finish (bool) – 
Return type
None
class langchain.callbacks.OpenAICallbackHandler[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that tracks OpenAI info.
total_tokens: int = 0
prompt_tokens: int = 0
completion_tokens: int = 0
successful_requests: int = 0
total_cost: float = 0.0
property always_verbose: bool
Whether to call verbose callbacks even if verbose is False.
on_llm_start(serialized, prompts, **kwargs)[source]
Print out the prompts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Print out the token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Collect token usage.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.StdOutCallbackHandler(color=None)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback Handler that prints to std out.
Parameters
color (Optional[str]) – 
Return type
None
on_llm_start(serialized, prompts, **kwargs)[source]
Print out the prompts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Do nothing.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Do nothing.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Print out that we are entering a chain.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Print out that we finished a chain.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Do nothing.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, color=None, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, color=None, observation_prefix=None, llm_prefix=None, **kwargs)[source]
If not the final action, print out observation.
Parameters
output (str) – 
color (Optional[str]) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, color=None, end='', **kwargs)[source]
Run when agent ends.
Parameters
text (str) – 
color (Optional[str]) – 
end (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, color=None, **kwargs)[source]
Run on agent end.
Parameters
finish (langchain.schema.AgentFinish) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
None
class langchain.callbacks.StreamingStdOutCallbackHandler[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
Callback handler for streaming. Only works with LLMs that support streaming.
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts running.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run on new LLM token. Only available when streaming is enabled.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run on arbitrary text.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run on agent end.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
langchain.callbacks.StreamlitCallbackHandler(parent_container, *, max_thought_containers=4, expand_new_thoughts=True, collapse_completed_thoughts=True, thought_labeler=None)[source]
Construct a new StreamlitCallbackHandler. This CallbackHandler is geared towards
use with a LangChain Agent; it displays the Agent’s LLM and tool-usage “thoughts”
inside a series of Streamlit expanders.
Parameters
parent_container (DeltaGenerator) – The st.container that will contain all the Streamlit elements that the
Handler creates.
max_thought_containers (int) – The max number of completed LLM thought containers to show at once. When this
threshold is reached, a new thought will cause the oldest thoughts to be
collapsed into a “History” expander. Defaults to 4.
expand_new_thoughts (bool) – Each LLM “thought” gets its own st.expander. This param controls whether that
expander is expanded by default. Defaults to True.
collapse_completed_thoughts (bool) – If True, LLM thought expanders will be collapsed when completed.
Defaults to True.
thought_labeler (Optional[LLMThoughtLabeler]) – An optional custom LLMThoughtLabeler instance. If unspecified, the handler
will use the default thought labeling logic. Defaults to None.
Returns
A new StreamlitCallbackHandler instance.
Note that this is an “auto-updating” API (if the installed version of Streamlit)
has a more recent StreamlitCallbackHandler implementation, an instance of that class
will be used.
Return type
BaseCallbackHandler
class langchain.callbacks.LLMThoughtLabeler[source]
Bases: object
Generates markdown labels for LLMThought containers. Pass a custom
subclass of this to StreamlitCallbackHandler to override its default
labeling logic.
get_initial_label()[source]
Return the markdown label for a new LLMThought that doesn’t have
an associated tool yet.
Return type
str
get_tool_label(tool, is_complete)[source]
Return the label for an LLMThought that has an associated
tool.
Parameters
tool (langchain.callbacks.streamlit.streamlit_callback_handler.ToolRecord) – The tool’s ToolRecord
is_complete (bool) – True if the thought is complete; False if the thought
is still receiving input.
Return type
The markdown label for the thought’s container.
get_history_label()[source]
Return a markdown label for the special ‘history’ container
that contains overflow thoughts.
Return type
str
get_final_agent_thought_label()[source]
Return the markdown label for the agent’s final thought -
the “Now I have the answer” thought, that doesn’t involve
a tool.
Return type
str
class langchain.callbacks.WandbCallbackHandler(job_type=None, project='langchain_callback_demo', entity=None, tags=None, group=None, name=None, notes=None, visualize=False, complexity_metrics=False, stream_logs=False)[source]
Bases: langchain.callbacks.utils.BaseMetadataCallbackHandler, langchain.callbacks.base.BaseCallbackHandler
Callback Handler that logs to Weights and Biases.
Parameters
job_type (str) – The type of job.
project (str) – The project to log to.
entity (str) – The entity to log to.
tags (list) – The tags to log.
group (str) – The group to log to.
name (str) – The name of the run.
notes (str) – The notes to log.
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics.
stream_logs (bool) – Whether to stream callback actions to W&B
Return type
None
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response using the run.log() method to Weights and Biases.
on_llm_start(serialized, prompts, **kwargs)[source]
Run when LLM starts.
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Run when LLM generates a new token.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Run when LLM ends running.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Run when LLM errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Run when chain ends running.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Run when chain errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Run when tool starts running.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_tool_end(output, **kwargs)[source]
Run when tool ends running.
Parameters
output (str) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Run when tool errors.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Run when agent is ending.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, **kwargs)[source]
Run when agent ends running.
Parameters
finish (langchain.schema.AgentFinish) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, **kwargs)[source]
Run on agent action.
Parameters
action (langchain.schema.AgentAction) – 
kwargs (Any) – 
Return type
Any
flush_tracker(langchain_asset=None, reset=True, finish=False, job_type=None, project=None, entity=None, tags=None, group=None, name=None, notes=None, visualize=None, complexity_metrics=None)[source]
Flush the tracker and reset the session.
Parameters
langchain_asset (Any) – The langchain asset to save.
reset (bool) – Whether to reset the session.
finish (bool) – Whether to finish the run.
job_type (Optional[str]) – The job type.
project (Optional[str]) – The project.
entity (Optional[str]) – The entity.
tags (Optional[Sequence]) – The tags.
group (Optional[str]) – The group.
name (Optional[str]) – The name.
notes (Optional[str]) – The notes.
visualize (Optional[bool]) – Whether to visualize.
complexity_metrics (Optional[bool]) – Whether to compute complexity metrics.
Returns – None
Return type
None
class langchain.callbacks.WhyLabsCallbackHandler(logger)[source]
Bases: langchain.callbacks.base.BaseCallbackHandler
WhyLabs CallbackHandler.
Parameters
logger (Logger) – 
on_llm_start(serialized, prompts, **kwargs)[source]
Pass the input prompts to the logger
Parameters
serialized (Dict[str, Any]) – 
prompts (List[str]) – 
kwargs (Any) – 
Return type
None
on_llm_end(response, **kwargs)[source]
Pass the generated response to the logger.
Parameters
response (langchain.schema.LLMResult) – 
kwargs (Any) – 
Return type
None
on_llm_new_token(token, **kwargs)[source]
Do nothing.
Parameters
token (str) – 
kwargs (Any) – 
Return type
None
on_llm_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_chain_start(serialized, inputs, **kwargs)[source]
Do nothing.
Parameters
serialized (Dict[str, Any]) – 
inputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_end(outputs, **kwargs)[source]
Do nothing.
Parameters
outputs (Dict[str, Any]) – 
kwargs (Any) – 
Return type
None
on_chain_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_tool_start(serialized, input_str, **kwargs)[source]
Do nothing.
Parameters
serialized (Dict[str, Any]) – 
input_str (str) – 
kwargs (Any) – 
Return type
None
on_agent_action(action, color=None, **kwargs)[source]
Do nothing.
Parameters
action (langchain.schema.AgentAction) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
Any
on_tool_end(output, color=None, observation_prefix=None, llm_prefix=None, **kwargs)[source]
Do nothing.
Parameters
output (str) – 
color (Optional[str]) – 
observation_prefix (Optional[str]) – 
llm_prefix (Optional[str]) – 
kwargs (Any) – 
Return type
None
on_tool_error(error, **kwargs)[source]
Do nothing.
Parameters
error (Union[Exception, KeyboardInterrupt]) – 
kwargs (Any) – 
Return type
None
on_text(text, **kwargs)[source]
Do nothing.
Parameters
text (str) – 
kwargs (Any) – 
Return type
None
on_agent_finish(finish, color=None, **kwargs)[source]
Run on agent end.
Parameters
finish (langchain.schema.AgentFinish) – 
color (Optional[str]) – 
kwargs (Any) – 
Return type
None
flush()[source]
Return type
None
close()[source]
Return type
None
classmethod from_params(*, api_key=None, org_id=None, dataset_id=None, sentiment=False, toxicity=False, themes=False)[source]
Instantiate whylogs Logger from params.
Parameters
api_key (Optional[str]) – WhyLabs API key. Optional because the preferred
way to specify the API key is with environment variable
WHYLABS_API_KEY.
org_id (Optional[str]) – WhyLabs organization id to write profiles to.
If not set must be specified in environment variable
WHYLABS_DEFAULT_ORG_ID.
dataset_id (Optional[str]) – The model or dataset this callback is gathering
telemetry for. If not set must be specified in environment variable
WHYLABS_DEFAULT_DATASET_ID.
sentiment (bool) – If True will initialize a model to perform
sentiment analysis compound score. Defaults to False and will not gather
this metric.
toxicity (bool) – If True will initialize a model to score
toxicity. Defaults to False and will not gather this metric.
themes (bool) – If True will initialize a model to calculate
distance to configured themes. Defaults to None and will not gather this
metric.
Return type
Logger
langchain.callbacks.get_openai_callback()[source]
Get the OpenAI callback handler in a context manager.
which conveniently exposes token and cost information.
Returns
The OpenAI callback handler.
Return type
OpenAICallbackHandler
Example
>>> with get_openai_callback() as cb:
...     # Use the OpenAI callback handler
langchain.callbacks.tracing_enabled(session_name='default')[source]
Get the Deprecated LangChainTracer in a context manager.
Parameters
session_name (str, optional) – The name of the session.
Defaults to “default”.
Returns
The LangChainTracer session.
Return type
TracerSessionV1
Example
>>> with tracing_enabled() as session:
...     # Use the LangChainTracer session
langchain.callbacks.wandb_tracing_enabled(session_name='default')[source]
Get the WandbTracer in a context manager.
Parameters
session_name (str, optional) – The name of the session.
Defaults to “default”.
Returns
None
Return type
Generator[None, None, None]
Example
>>> with wandb_tracing_enabled() as session:
...     # Use the WandbTracer session