langchain.callbacks.base.BaseCallbackHandler¶
class langchain.callbacks.base.BaseCallbackHandler[source]¶
Bases: LLMManagerMixin, ChainManagerMixin, ToolManagerMixin, RetrieverManagerMixin, CallbackManagerMixin, RunManagerMixin
Base callback handler that can be used to handle callbacks from langchain.
Methods
__init__()
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id[, parent_run_id])
Run when chain ends running.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, *, run_id)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id[, parent_run_id])
Run when LLM ends running.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id[, parent_run_id])
Run when tool ends running.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶


Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.manager.CallbackManagerForToolRun¶
class langchain.callbacks.manager.CallbackManagerForToolRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, inheritable_metadata: Optional[Dict[str, Any]] = None)[source]¶
Bases: ParentRunManager, ToolManagerMixin
Callback manager for tool run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
metadata (Optional[Dict[str, Any]]) – The metadata.
inheritable_metadata (Optional[Dict[str, Any]]) – The inheritable metadata.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_child([tag])
Get a child callback manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_text(text, **kwargs)
Run when text is received.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
get_child(tag: Optional[str] = None) → CallbackManager¶
Get a child callback manager.
Parameters


Get a child callback manager.
Parameters
tag (str, optional) – The tag for the child callback manager.
Defaults to None.
Returns
The child callback manager.
Return type
CallbackManager
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
Parameters
output (str) – The output of the tool.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
Parameters
error (Exception or KeyboardInterrupt) – The error.


langchain.callbacks.whylabs_callback.WhyLabsCallbackHandler¶
class langchain.callbacks.whylabs_callback.WhyLabsCallbackHandler(logger: Logger)[source]¶
Bases: BaseCallbackHandler
Callback Handler for logging to WhyLabs. This callback handler utilizes
langkit to extract features from the prompts & responses when interacting with
an LLM. These features can be used to guardrail, evaluate, and observe interactions
over time to detect issues relating to hallucinations, prompt engineering,
or output validation. LangKit is an LLM monitoring toolkit developed by WhyLabs.
Here are some examples of what can be monitored with LangKit:
* Text Quality
readability score
complexity and grade scores
Text Relevance
- Similarity scores between prompt/responses
- Similarity scores against user-defined themes
- Topic classification
Security and Privacy
- patterns - count of strings matching a user-defined regex pattern group
- jailbreaks - similarity scores with respect to known jailbreak attempts
- prompt injection - similarity scores with respect to known prompt attacks
- refusals - similarity scores with respect to known LLM refusal responses
Sentiment and Toxicity
- sentiment analysis
- toxicity analysis
For more information, see https://docs.whylabs.ai/docs/language-model-monitoring
or check out the LangKit repo here: https://github.com/whylabs/langkit
—
:param api_key: WhyLabs API key. Optional because the preferred
way to specify the API key is with environment variable
WHYLABS_API_KEY.
Parameters
org_id (Optional[str]) – WhyLabs organization id to write profiles to.
Optional because the preferred way to specify the organization id is
with environment variable WHYLABS_DEFAULT_ORG_ID.
dataset_id (Optional[str]) – WhyLabs dataset id to write profiles to.


dataset_id (Optional[str]) – WhyLabs dataset id to write profiles to.
Optional because the preferred way to specify the dataset id is
with environment variable WHYLABS_DEFAULT_DATASET_ID.
sentiment (bool) – Whether to enable sentiment analysis. Defaults to False.
toxicity (bool) – Whether to enable toxicity analysis. Defaults to False.
themes (bool) – Whether to enable theme analysis. Defaults to False.
Initiate the rolling logger
Methods
__init__(logger)
Initiate the rolling logger
close()
flush()
from_params(*[, api_key, org_id, ...])
Instantiate whylogs Logger from params.
on_agent_action(action[, color])
Do nothing.
on_agent_finish(finish[, color])
Run on agent end.
on_chain_end(outputs, **kwargs)
Do nothing.
on_chain_error(error, **kwargs)
Do nothing.
on_chain_start(serialized, inputs, **kwargs)
Do nothing.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Pass the generated response to the logger.
on_llm_error(error, **kwargs)
Do nothing.
on_llm_new_token(token, **kwargs)
Do nothing.
on_llm_start(serialized, prompts, **kwargs)
Pass the input prompts to the logger
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text, **kwargs)
Do nothing.
on_tool_end(output[, color, ...])
Do nothing.
on_tool_error(error, **kwargs)
Do nothing.
on_tool_start(serialized, input_str, **kwargs)
Do nothing.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
close() → None[source]¶
flush() → None[source]¶
classmethod from_params(*, api_key: Optional[str] = None, org_id: Optional[str] = None, dataset_id: Optional[str] = None, sentiment: bool = False, toxicity: bool = False, themes: bool = False) → Logger[source]¶
Instantiate whylogs Logger from params.
Parameters
api_key (Optional[str]) – WhyLabs API key. Optional because the preferred
way to specify the API key is with environment variable
WHYLABS_API_KEY.
org_id (Optional[str]) – WhyLabs organization id to write profiles to.
If not set must be specified in environment variable
WHYLABS_DEFAULT_ORG_ID.
dataset_id (Optional[str]) – The model or dataset this callback is gathering
telemetry for. If not set must be specified in environment variable
WHYLABS_DEFAULT_DATASET_ID.
sentiment (bool) – If True will initialize a model to perform
sentiment analysis compound score. Defaults to False and will not gather
this metric.
toxicity (bool) – If True will initialize a model to score
toxicity. Defaults to False and will not gather this metric.


toxicity. Defaults to False and will not gather this metric.
themes (bool) – If True will initialize a model to calculate
distance to configured themes. Defaults to None and will not gather this
metric.
on_agent_action(action: AgentAction, color: Optional[str] = None, **kwargs: Any) → Any[source]¶
Do nothing.
on_agent_finish(finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) → None[source]¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Do nothing.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Do nothing.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Pass the generated response to the logger.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Do nothing.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Pass the input prompts to the logger


Pass the input prompts to the logger
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Do nothing.
on_tool_end(output: str, color: Optional[str] = None, observation_prefix: Optional[str] = None, llm_prefix: Optional[str] = None, **kwargs: Any) → None[source]¶
Do nothing.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Do nothing.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.tracers.schemas.TracerSessionBase¶
class langchain.callbacks.tracers.schemas.TracerSessionBase(*, start_time: datetime = None, name: Optional[str] = None, extra: Optional[Dict[str, Any]] = None, tenant_id: UUID)[source]¶
Bases: TracerSessionV1Base
A creation class for TracerSession.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param extra: Optional[Dict[str, Any]] = None¶
param name: Optional[str] = None¶
param start_time: datetime.datetime [Optional]¶
param tenant_id: uuid.UUID [Required]¶


langchain.callbacks.manager.AsyncCallbackManagerForLLMRun¶
class langchain.callbacks.manager.AsyncCallbackManagerForLLMRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, inheritable_metadata: Optional[Dict[str, Any]] = None)[source]¶
Bases: AsyncRunManager, LLMManagerMixin
Async callback manager for LLM run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
metadata (Optional[Dict[str, Any]]) – The metadata.
inheritable_metadata (Optional[Dict[str, Any]]) – The inheritable metadata.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_text(text, **kwargs)
Run when text is received.
classmethod get_noop_manager() → BRM¶


Run when text is received.
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
async on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
Parameters
response (LLMResult) – The LLM result.
async on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
Parameters
error (Exception or KeyboardInterrupt) – The error.
async on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
Parameters
token (str) – The new token.
async on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any


langchain.callbacks.tracers.evaluation.EvaluatorCallbackHandler¶
class langchain.callbacks.tracers.evaluation.EvaluatorCallbackHandler(evaluators: Sequence[RunEvaluator], max_workers: Optional[int] = None, client: Optional[LangChainPlusClient] = None, example_id: Optional[Union[UUID, str]] = None, skip_unfinished: bool = True, project_name: Optional[str] = None, **kwargs: Any)[source]¶
Bases: BaseTracer
A tracer that runs a run evaluator whenever a run is persisted.
Parameters
evaluators (Sequence[RunEvaluator]) – The run evaluators to apply to all top level runs.
max_workers (int, optional) – The maximum number of worker threads to use for running the evaluators.
If not specified, it will default to the number of evaluators.
client (LangChainPlusClient, optional) – The LangChainPlusClient instance to use for evaluating the runs.
If not specified, a new instance will be created.
example_id (Union[UUID, str], optional) – The example ID to be associated with the runs.
project_name (str, optional) – The LangSmith project name to be organize eval chain runs under.
example_id¶
The example ID associated with the runs.
Type
Union[UUID, None]
client¶
The LangChainPlusClient instance used for evaluating the runs.
Type
LangChainPlusClient
evaluators¶
The sequence of run evaluators to be executed.
Type
Sequence[RunEvaluator]
executor¶
The thread pool executor used for running the evaluators.
Type
ThreadPoolExecutor
futures¶
The set of futures representing the running evaluators.
Type
Set[Future]
skip_unfinished¶
Whether to skip runs that are not finished or raised
an error.
Type
bool
project_name¶


an error.
Type
bool
project_name¶
The LangSmith project name to be organize eval chain runs under.
Type
Optional[str]
Methods
__init__(evaluators[, max_workers, client, ...])
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])


on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
wait_for_futures()
Wait for all futures to complete.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
name
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶


Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
wait_for_futures() → None[source]¶
Wait for all futures to complete.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
name = 'evaluator_callback_handler'¶
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, Run]¶


langchain.callbacks.openai_info.standardize_model_name¶
langchain.callbacks.openai_info.standardize_model_name(model_name: str, is_completion: bool = False) → str[source]¶
Standardize the model name to a format that can be used in the OpenAI API.
:param model_name: Model name to standardize.
:param is_completion: Whether the model is used for completion or not.
Defaults to False.
Returns
Standardized model name.


langchain.callbacks.streaming_aiter_final_only.AsyncFinalIteratorCallbackHandler¶
class langchain.callbacks.streaming_aiter_final_only.AsyncFinalIteratorCallbackHandler(*, answer_prefix_tokens: Optional[List[str]] = None, strip_tokens: bool = True, stream_prefix: bool = False)[source]¶
Bases: AsyncIteratorCallbackHandler
Callback handler that returns an async iterator.
Only the final output of the agent will be iterated.
Instantiate AsyncFinalIteratorCallbackHandler.
Parameters
answer_prefix_tokens – Token sequence that prefixes the answer.
Default is [“Final”, “Answer”, “:”]
strip_tokens – Ignore white spaces and new lines when comparing
answer_prefix_tokens to last tokens? (to determine if answer has been
reached)
stream_prefix – Should answer prefix itself also be streamed?
Methods
__init__(*[, answer_prefix_tokens, ...])
Instantiate AsyncFinalIteratorCallbackHandler.
aiter()
append_to_last_tokens(token)
check_if_answer_reached()
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id[, ...])
Run when chain ends running.
on_chain_error(error, *, run_id[, ...])
Run when chain errors.
on_chain_start(serialized, inputs, *, run_id)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)


Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run on new LLM token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run on retriever end.
on_retriever_error(error, *, run_id[, ...])
Run on retriever error.
on_retriever_start(serialized, query, *, run_id)
Run on retriever start.
on_text(text, *, run_id[, parent_run_id, tags])
Run on arbitrary text.
on_tool_end(output, *, run_id[, ...])
Run when tool ends running.
on_tool_error(error, *, run_id[, ...])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
always_verbose
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
async aiter() → AsyncIterator[str]¶
append_to_last_tokens(token: str) → None[source]¶
check_if_answer_reached() → bool[source]¶
async on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run on agent action.


Run on agent action.
async on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run on agent end.
async on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run when chain ends running.
async on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run when chain errors.
async on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run when chain starts running.
async on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
async on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
async on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None¶
Run when LLM errors.
async on_llm_new_token(token: str, **kwargs: Any) → None[source]¶


Run on new LLM token. Only available when streaming is enabled.
async on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts running.
async on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run on retriever end.
async on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run on retriever error.
async on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run on retriever start.
async on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run on arbitrary text.
async on_tool_end(output: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run when tool ends running.
async on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → None¶
Run when tool errors.


Run when tool errors.
async on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run when tool starts running.
property always_verbose: bool¶
done: asyncio.Event¶
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
queue: asyncio.Queue[str]¶
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.whylabs_callback.import_langkit¶
langchain.callbacks.whylabs_callback.import_langkit(sentiment: bool = False, toxicity: bool = False, themes: bool = False) → Any[source]¶
Import the langkit python package and raise an error if it is not installed.
Parameters
sentiment – Whether to import the langkit.sentiment module. Defaults to False.
toxicity – Whether to import the langkit.toxicity module. Defaults to False.
themes – Whether to import the langkit.themes module. Defaults to False.
Returns
The imported langkit module.


langchain.callbacks.tracers.base.BaseTracer¶
class langchain.callbacks.tracers.base.BaseTracer(**kwargs: Any)[source]¶
Bases: BaseCallbackHandler, ABC
Base interface for tracers.
Methods
__init__(**kwargs)
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])


on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None[source]¶
Start a trace for a chain run.


Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None[source]¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None[source]¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None[source]¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None[source]¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None[source]¶
Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.manager.wandb_tracing_enabled¶
langchain.callbacks.manager.wandb_tracing_enabled(session_name: str = 'default') → Generator[None, None, None][source]¶
Get the WandbTracer in a context manager.
Parameters
session_name (str, optional) – The name of the session.
Defaults to “default”.
Returns
None
Example
>>> with wandb_tracing_enabled() as session:
...     # Use the WandbTracer session


langchain.callbacks.wandb_callback.WandbCallbackHandler¶
class langchain.callbacks.wandb_callback.WandbCallbackHandler(job_type: Optional[str] = None, project: Optional[str] = 'langchain_callback_demo', entity: Optional[str] = None, tags: Optional[Sequence] = None, group: Optional[str] = None, name: Optional[str] = None, notes: Optional[str] = None, visualize: bool = False, complexity_metrics: bool = False, stream_logs: bool = False)[source]¶
Bases: BaseMetadataCallbackHandler, BaseCallbackHandler
Callback Handler that logs to Weights and Biases.
Parameters
job_type (str) – The type of job.
project (str) – The project to log to.
entity (str) – The entity to log to.
tags (list) – The tags to log.
group (str) – The group to log to.
name (str) – The name of the run.
notes (str) – The notes to log.
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics.
stream_logs (bool) – Whether to stream callback actions to W&B
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response using the run.log() method to Weights and Biases.
Initialize callback handler.
Methods
__init__([job_type, project, entity, tags, ...])
Initialize callback handler.
flush_tracker([langchain_asset, reset, ...])
Flush the tracker and reset the session.
get_custom_callback_meta()


Flush the tracker and reset the session.
get_custom_callback_meta()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run when agent ends running.
on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, **kwargs)
Run when agent is ending.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
reset_callback_meta()
Reset the callback metadata.
Attributes
always_verbose
Whether to call verbose callbacks even if verbose is False.
ignore_agent
Whether to ignore agent callbacks.
ignore_chain


ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
flush_tracker(langchain_asset: Any = None, reset: bool = True, finish: bool = False, job_type: Optional[str] = None, project: Optional[str] = None, entity: Optional[str] = None, tags: Optional[Sequence] = None, group: Optional[str] = None, name: Optional[str] = None, notes: Optional[str] = None, visualize: Optional[bool] = None, complexity_metrics: Optional[bool] = None) → None[source]¶
Flush the tracker and reset the session.
Parameters
langchain_asset – The langchain asset to save.
reset – Whether to reset the session.
finish – Whether to finish the run.
job_type – The job type.
project – The project.
entity – The entity.
tags – The tags.
group – The group.
name – The name.
notes – The notes.
visualize – Whether to visualize.
complexity_metrics – Whether to compute complexity metrics.
Returns – None
get_custom_callback_meta() → Dict[str, Any]¶
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run when agent ends running.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.


Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶


Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run when agent is ending.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
reset_callback_meta() → None¶
Reset the callback metadata.
property always_verbose: bool¶
Whether to call verbose callbacks even if verbose is False.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.human.HumanRejectedException¶
class langchain.callbacks.human.HumanRejectedException[source]¶
Bases: Exception
Exception to raise when a person manually review and rejects a value.
add_note()¶
Exception.add_note(note) –
add a note to the exception
with_traceback()¶
Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.
args¶


langchain.callbacks.mlflow_callback.analyze_text¶
langchain.callbacks.mlflow_callback.analyze_text(text: str, nlp: Any = None) → dict[source]¶
Analyze text using textstat and spacy.
Parameters
text (str) – The text to analyze.
nlp (spacy.lang) – The spacy language model to use for visualization.
Returns
A dictionary containing the complexity metrics and visualizationfiles serialized to  HTML string.
Return type
(dict)


langchain.callbacks.tracers.schemas.TracerSessionV1¶
class langchain.callbacks.tracers.schemas.TracerSessionV1(*, start_time: datetime = None, name: Optional[str] = None, extra: Optional[Dict[str, Any]] = None, id: int)[source]¶
Bases: TracerSessionV1Base
TracerSessionV1 schema.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param extra: Optional[Dict[str, Any]] = None¶
param id: int [Required]¶
param name: Optional[str] = None¶
param start_time: datetime.datetime [Optional]¶


langchain.callbacks.comet_ml_callback.CometCallbackHandler¶
class langchain.callbacks.comet_ml_callback.CometCallbackHandler(task_type: Optional[str] = 'inference', workspace: Optional[str] = None, project_name: Optional[str] = None, tags: Optional[Sequence] = None, name: Optional[str] = None, visualizations: Optional[List[str]] = None, complexity_metrics: bool = False, custom_metrics: Optional[Callable] = None, stream_logs: bool = True)[source]¶
Bases: BaseMetadataCallbackHandler, BaseCallbackHandler
Callback Handler that logs to Comet.
Parameters
job_type (str) – The type of comet_ml task such as “inference”,
“testing” or “qc”
project_name (str) – The comet_ml project name
tags (list) – Tags to add to the task
task_name (str) – Name of the comet_ml task
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics
stream_logs (bool) – Whether to stream callback actions to Comet
This handler will utilize the associated callback method and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response to Comet.
Initialize callback handler.
Methods
__init__([task_type, workspace, ...])
Initialize callback handler.
flush_tracker([langchain_asset, task_type, ...])
Flush the tracker and setup the session.
get_custom_callback_meta()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run when agent ends running.
on_chain_end(outputs, **kwargs)
Run when chain ends running.


on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, **kwargs)
Run when agent is ending.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
reset_callback_meta()
Reset the callback metadata.
Attributes
always_verbose
Whether to call verbose callbacks even if verbose is False.
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error


ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
flush_tracker(langchain_asset: Any = None, task_type: Optional[str] = 'inference', workspace: Optional[str] = None, project_name: Optional[str] = 'comet-langchain-demo', tags: Optional[Sequence] = None, name: Optional[str] = None, visualizations: Optional[List[str]] = None, complexity_metrics: bool = False, custom_metrics: Optional[Callable] = None, finish: bool = False, reset: bool = False) → None[source]¶
Flush the tracker and setup the session.
Everything after this will be a new table.
Parameters
name – Name of the preformed session so far so it is identifyable
langchain_asset – The langchain asset to save.
finish – Whether to finish the run.
Returns – None
get_custom_callback_meta() → Dict[str, Any]¶
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run when agent ends running.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.


Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run when agent is ending.


Run when agent is ending.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
reset_callback_meta() → None¶
Reset the callback metadata.
property always_verbose: bool¶
Whether to call verbose callbacks even if verbose is False.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.tracers.wandb.WandbTracer¶
class langchain.callbacks.tracers.wandb.WandbTracer(run_args: Optional[WandbRunArgs] = None, **kwargs: Any)[source]¶
Bases: BaseTracer
Callback Handler that logs to Weights and Biases.
This handler will log the model architecture and run traces to Weights and Biases.
This will ensure that all LangChain activity is logged to W&B.
Initializes the WandbTracer.
Parameters
run_args – (dict, optional) Arguments to pass to wandb.init(). If not
provided, wandb.init() will be called with no arguments. Please
refer to the wandb.init for more details.
To use W&B to monitor all LangChain activity, add this tracer like any other
LangChain callback:
```
from wandb.integration.langchain import WandbTracer
tracer = WandbTracer()
chain = LLMChain(llm, callbacks=[tracer])
# …end of notebook / script:
tracer.finish()
```
Methods
__init__([run_args])
Initializes the WandbTracer.
finish()
Waits for all asynchronous processes to finish and data to upload.
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)


on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
finish() → None[source]¶
Waits for all asynchronous processes to finish and data to upload.


Waits for all asynchronous processes to finish and data to upload.
Proxy for wandb.finish().
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.


Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.


Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, Run]¶


langchain.callbacks.argilla_callback.ArgillaCallbackHandler¶
class langchain.callbacks.argilla_callback.ArgillaCallbackHandler(dataset_name: str, workspace_name: Optional[str] = None, api_url: Optional[str] = None, api_key: Optional[str] = None)[source]¶
Bases: BaseCallbackHandler
Callback Handler that logs into Argilla.
Parameters
dataset_name – name of the FeedbackDataset in Argilla. Note that it must
exist in advance. If you need help on how to create a FeedbackDataset in
Argilla, please visit
https://docs.argilla.io/en/latest/guides/llms/practical_guides/use_argilla_callback_in_langchain.html.
workspace_name – name of the workspace in Argilla where the specified
FeedbackDataset lives in. Defaults to None, which means that the
default workspace will be used.
api_url – URL of the Argilla Server that we want to use, and where the
FeedbackDataset lives in. Defaults to None, which means that either
ARGILLA_API_URL environment variable or the default http://localhost:6900
will be used.
api_key – API Key to connect to the Argilla Server. Defaults to None, which
means that either ARGILLA_API_KEY environment variable or the default
argilla.apikey will be used.
Raises
ImportError – if the argilla package is not installed.
ConnectionError – if the connection to Argilla fails.
FileNotFoundError – if the FeedbackDataset retrieval from Argilla fails.
Examples
>>> from langchain.llms import OpenAI
>>> from langchain.callbacks import ArgillaCallbackHandler
>>> argilla_callback = ArgillaCallbackHandler(
...     dataset_name="my-dataset",
...     workspace_name="my-workspace",
...     api_url="http://localhost:6900",
...     api_key="argilla.apikey",
... )


...     api_key="argilla.apikey",
... )
>>> llm = OpenAI(
...     temperature=0,
...     callbacks=[argilla_callback],
...     verbose=True,
...     openai_api_key="API_KEY_HERE",
... )
>>> llm.generate([
...     "What is the best NLP-annotation tool out there? (no bias at all)",
... ])
"Argilla, no doubt about it."
Initializes the ArgillaCallbackHandler.
Parameters
dataset_name – name of the FeedbackDataset in Argilla. Note that it must
exist in advance. If you need help on how to create a FeedbackDataset
in Argilla, please visit
https://docs.argilla.io/en/latest/guides/llms/practical_guides/use_argilla_callback_in_langchain.html.
workspace_name – name of the workspace in Argilla where the specified
FeedbackDataset lives in. Defaults to None, which means that the
default workspace will be used.
api_url – URL of the Argilla Server that we want to use, and where the
FeedbackDataset lives in. Defaults to None, which means that either
ARGILLA_API_URL environment variable or the default
http://localhost:6900 will be used.
api_key – API Key to connect to the Argilla Server. Defaults to None, which
means that either ARGILLA_API_KEY environment variable or the default
argilla.apikey will be used.
Raises
ImportError – if the argilla package is not installed.
ConnectionError – if the connection to Argilla fails.
FileNotFoundError – if the FeedbackDataset retrieval from Argilla fails.
Methods
__init__(dataset_name[, workspace_name, ...])
Initializes the ArgillaCallbackHandler.
on_agent_action(action, **kwargs)
Do nothing when agent takes a specific action.


on_agent_action(action, **kwargs)
Do nothing when agent takes a specific action.
on_agent_finish(finish, **kwargs)
Do nothing
on_chain_end(outputs, **kwargs)
If either the parent_run_id or the run_id is in self.prompts, then log the outputs to Argilla, and pop the run from self.prompts.
on_chain_error(error, **kwargs)
Do nothing when LLM chain outputs an error.
on_chain_start(serialized, inputs, **kwargs)
If the key input is in inputs, then save it in self.prompts using either the parent_run_id or the run_id as the key.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Log records to Argilla when an LLM ends.
on_llm_error(error, **kwargs)
Do nothing when LLM outputs an error.
on_llm_new_token(token, **kwargs)
Do nothing when a new token is generated.
on_llm_start(serialized, prompts, **kwargs)
Save the prompts in memory when an LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, **kwargs)
Do nothing
on_tool_end(output[, observation_prefix, ...])
Do nothing when tool ends.
on_tool_error(error, **kwargs)
Do nothing when tool outputs an error.


on_tool_error(error, **kwargs)
Do nothing when tool outputs an error.
on_tool_start(serialized, input_str, **kwargs)
Do nothing when tool starts.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Do nothing when agent takes a specific action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Do nothing
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
If either the parent_run_id or the run_id is in self.prompts, then
log the outputs to Argilla, and pop the run from self.prompts. The behavior
differs if the output is a list or not.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing when LLM chain outputs an error.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
If the key input is in inputs, then save it in self.prompts using
either the parent_run_id or the run_id as the key. This is done so that
we don’t log the same input prompt twice, once when the LLM starts and once
when the chain starts.


when the chain starts.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Log records to Argilla when an LLM ends.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing when LLM outputs an error.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Do nothing when a new token is generated.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Save the prompts in memory when an LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Do nothing
on_tool_end(output: str, observation_prefix: Optional[str] = None, llm_prefix: Optional[str] = None, **kwargs: Any) → None[source]¶
Do nothing when tool ends.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing when tool outputs an error.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Do nothing when tool starts.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.wandb_callback.construct_html_from_prompt_and_generation¶
langchain.callbacks.wandb_callback.construct_html_from_prompt_and_generation(prompt: str, generation: str) → Any[source]¶
Construct an html element from a prompt and a generation.
Parameters
prompt (str) – The prompt.
generation (str) – The generation.
Returns
The html element.
Return type
(wandb.Html)


langchain.callbacks.manager.CallbackManagerForRetrieverRun¶
class langchain.callbacks.manager.CallbackManagerForRetrieverRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, inheritable_metadata: Optional[Dict[str, Any]] = None)[source]¶
Bases: ParentRunManager, RetrieverManagerMixin
Callback manager for retriever run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
metadata (Optional[Dict[str, Any]]) – The metadata.
inheritable_metadata (Optional[Dict[str, Any]]) – The inheritable metadata.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_child([tag])
Get a child callback manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_retriever_end(documents, **kwargs)
Run when retriever ends running.
on_retriever_error(error, **kwargs)
Run when retriever errors.
on_text(text, **kwargs)
Run when text is received.
get_child(tag: Optional[str] = None) → CallbackManager¶


get_child(tag: Optional[str] = None) → CallbackManager¶
Get a child callback manager.
Parameters
tag (str, optional) – The tag for the child callback manager.
Defaults to None.
Returns
The child callback manager.
Return type
CallbackManager
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
on_retriever_end(documents: Sequence[Document], **kwargs: Any) → None[source]¶
Run when retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when retriever errors.
on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any


langchain.callbacks.tracers.langchain.LangChainTracer¶
class langchain.callbacks.tracers.langchain.LangChainTracer(example_id: Optional[Union[UUID, str]] = None, project_name: Optional[str] = None, client: Optional[LangChainPlusClient] = None, tags: Optional[List[str]] = None, **kwargs: Any)[source]¶
Bases: BaseTracer
An implementation of the SharedTracer that POSTS to the langchain endpoint.
Initialize the LangChain tracer.
Methods
__init__([example_id, project_name, client, ...])
Initialize the LangChain tracer.
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Start a trace for an LLM run.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.


Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
wait_for_futures()
Wait for the given futures to complete.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.


End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None[source]¶
Start a trace for an LLM run.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.


Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
wait_for_futures() → None[source]¶
Wait for the given futures to complete.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶


Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, langchain.callbacks.tracers.schemas.Run]¶


langchain.callbacks.context_callback.ContextCallbackHandler¶
class langchain.callbacks.context_callback.ContextCallbackHandler(token: str = '', verbose: bool = False, **kwargs: Any)[source]¶
Bases: BaseCallbackHandler
Callback Handler that records transcripts to Context (https://getcontext.ai).
Keyword Arguments
token (optional) – The token with which to authenticate requests to Context.
Visit https://go.getcontext.ai/settings to generate a token.
If not provided, the value of the CONTEXT_TOKEN environment
variable will be used.
Raises
ImportError – if the context-python package is not installed.
Chat Example:>>> from langchain.llms import ChatOpenAI
>>> from langchain.callbacks import ContextCallbackHandler
>>> context_callback = ContextCallbackHandler(
...     token="<CONTEXT_TOKEN_HERE>",
... )
>>> chat = ChatOpenAI(
...     temperature=0,
...     headers={"user_id": "123"},
...     callbacks=[context_callback],
...     openai_api_key="API_KEY_HERE",
... )
>>> messages = [
...     SystemMessage(content="You translate English to French."),
...     HumanMessage(content="I love programming with LangChain."),
... ]
>>> chat(messages)
Chain Example:>>> from langchain import LLMChain
>>> from langchain.llms import ChatOpenAI
>>> from langchain.callbacks import ContextCallbackHandler
>>> context_callback = ContextCallbackHandler(
...     token="<CONTEXT_TOKEN_HERE>",
... )
>>> human_message_prompt = HumanMessagePromptTemplate(
...     prompt=PromptTemplate(
...         template="What is a good name for a company that makes {product}?",
...         input_variables=["product"],
...    ),
... )
>>> chat_prompt_template = ChatPromptTemplate.from_messages(
...   [human_message_prompt]
... )
>>> callback = ContextCallbackHandler(token)


...   [human_message_prompt]
... )
>>> callback = ContextCallbackHandler(token)
>>> # Note: the same callback object must be shared between the
...   LLM and the chain.
>>> chat = ChatOpenAI(temperature=0.9, callbacks=[callback])
>>> chain = LLMChain(
...   llm=chat,
...   prompt=chat_prompt_template,
...   callbacks=[callback]
... )
>>> chain.run("colorful socks")
Methods
__init__([token, verbose])
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, **kwargs)
Run when chain ends.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts.
on_chat_model_start(serialized, messages, *, ...)
Run when the chat model is started.
on_llm_end(response, **kwargs)
Run when LLM ends.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id[, parent_run_id])
Run when tool ends running.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts.


Run when chain starts.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, **kwargs: Any) → Any[source]¶
Run when the chat model is started.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.utils.load_json¶
langchain.callbacks.utils.load_json(json_path: Union[str, Path]) → str[source]¶
Load json file to a string.
Parameters
json_path (str) – The path to the json file.
Returns
The string representation of the json file.
Return type
(str)


langchain.callbacks.tracers.schemas.TracerSessionV1Create¶
class langchain.callbacks.tracers.schemas.TracerSessionV1Create(*, start_time: datetime = None, name: Optional[str] = None, extra: Optional[Dict[str, Any]] = None)[source]¶
Bases: TracerSessionV1Base
Create class for TracerSessionV1.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param extra: Optional[Dict[str, Any]] = None¶
param name: Optional[str] = None¶
param start_time: datetime.datetime [Optional]¶


langchain.callbacks.manager.AsyncCallbackManagerForToolRun¶
class langchain.callbacks.manager.AsyncCallbackManagerForToolRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, inheritable_metadata: Optional[Dict[str, Any]] = None)[source]¶
Bases: AsyncParentRunManager, ToolManagerMixin
Async callback manager for tool run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
metadata (Optional[Dict[str, Any]]) – The metadata.
inheritable_metadata (Optional[Dict[str, Any]]) – The inheritable metadata.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_child([tag])
Get a child callback manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_text(text, **kwargs)
Run when text is received.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
get_child(tag: Optional[str] = None) → AsyncCallbackManager¶
Get a child callback manager.
Parameters


Get a child callback manager.
Parameters
tag (str, optional) – The tag for the child callback manager.
Defaults to None.
Returns
The child callback manager.
Return type
AsyncCallbackManager
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
async on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any
async on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
Parameters
output (str) – The output of the tool.
async on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
Parameters
error (Exception or KeyboardInterrupt) – The error.


langchain.callbacks.manager.CallbackManagerForLLMRun¶
class langchain.callbacks.manager.CallbackManagerForLLMRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, inheritable_metadata: Optional[Dict[str, Any]] = None)[source]¶
Bases: RunManager, LLMManagerMixin
Callback manager for LLM run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
metadata (Optional[Dict[str, Any]]) – The metadata.
inheritable_metadata (Optional[Dict[str, Any]]) – The inheritable metadata.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_text(text, **kwargs)
Run when text is received.
classmethod get_noop_manager() → BRM¶


Run when text is received.
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
Parameters
response (LLMResult) – The LLM result.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
Parameters
error (Exception or KeyboardInterrupt) – The error.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
Parameters
token (str) – The new token.
on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any


langchain.callbacks.file.FileCallbackHandler¶
class langchain.callbacks.file.FileCallbackHandler(filename: str, mode: str = 'a', color: Optional[str] = None)[source]¶
Bases: BaseCallbackHandler
Callback Handler that writes to a file.
Initialize callback handler.
Methods
__init__(filename[, mode, color])
Initialize callback handler.
on_agent_action(action[, color])
Run on agent action.
on_agent_finish(finish[, color])
Run on agent end.
on_chain_end(outputs, **kwargs)
Print out that we finished a chain.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Print out that we are entering a chain.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id[, parent_run_id])
Run when LLM ends running.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text[, color, end])
Run when agent ends.


on_text(text[, color, end])
Run when agent ends.
on_tool_end(output[, color, ...])
If not the final action, print out observation.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, color: Optional[str] = None, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) → None[source]¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Print out that we finished a chain.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Print out that we are entering a chain.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶


Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text: str, color: Optional[str] = None, end: str = '', **kwargs: Any) → None[source]¶
Run when agent ends.
on_tool_end(output: str, color: Optional[str] = None, observation_prefix: Optional[str] = None, llm_prefix: Optional[str] = None, **kwargs: Any) → None[source]¶
If not the final action, print out observation.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.aim_callback.AimCallbackHandler¶
class langchain.callbacks.aim_callback.AimCallbackHandler(repo: Optional[str] = None, experiment_name: Optional[str] = None, system_tracking_interval: Optional[int] = 10, log_system_params: bool = True)[source]¶
Bases: BaseMetadataCallbackHandler, BaseCallbackHandler
Callback Handler that logs to Aim.
Parameters
repo (str, optional) – Aim repository path or Repo object to which
Run object is bound. If skipped, default Repo is used.
experiment_name (str, optional) – Sets Run’s experiment property.
‘default’ if not specified. Can be used later to query runs/sequences.
system_tracking_interval (int, optional) – Sets the tracking interval
in seconds for system usage metrics (CPU, Memory, etc.). Set to None
to disable system metrics tracking.
log_system_params (bool, optional) – Enable/Disable logging of system
params such as installed packages, git info, environment variables, etc.
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run
and then logs the response to Aim.
Initialize callback handler.
Methods
__init__([repo, experiment_name, ...])
Initialize callback handler.
flush_tracker([repo, experiment_name, ...])
Flush the tracker and reset the session.
get_custom_callback_meta()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run when agent ends running.
on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.


Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(serialized, query, *, run_id)
Run when Retriever starts running.
on_text(text, **kwargs)
Run when agent is ending.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
reset_callback_meta()
Reset the callback metadata.
setup(**kwargs)
Attributes
always_verbose
Whether to call verbose callbacks even if verbose is False.
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline


ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
flush_tracker(repo: Optional[str] = None, experiment_name: Optional[str] = None, system_tracking_interval: Optional[int] = 10, log_system_params: bool = True, langchain_asset: Any = None, reset: bool = True, finish: bool = False) → None[source]¶
Flush the tracker and reset the session.
Parameters
repo (str, optional) – Aim repository path or Repo object to which
Run object is bound. If skipped, default Repo is used.
experiment_name (str, optional) – Sets Run’s experiment property.
‘default’ if not specified. Can be used later to query runs/sequences.
system_tracking_interval (int, optional) – Sets the tracking interval
in seconds for system usage metrics (CPU, Memory, etc.). Set to None
to disable system metrics tracking.
log_system_params (bool, optional) – Enable/Disable logging of system
params such as installed packages, git info, environment variables, etc.
langchain_asset – The langchain asset to save.
reset – Whether to reset the session.
finish – Whether to finish the run.
Returns – None
get_custom_callback_meta() → Dict[str, Any]¶
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run when agent ends running.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.


Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(serialized: Dict[str, Any], query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any¶


Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run when agent is ending.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
reset_callback_meta() → None¶
Reset the callback metadata.
setup(**kwargs: Any) → None[source]¶
property always_verbose: bool¶
Whether to call verbose callbacks even if verbose is False.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.manager.tracing_enabled¶
langchain.callbacks.manager.tracing_enabled(session_name: str = 'default') → Generator[TracerSessionV1, None, None][source]¶
Get the Deprecated LangChainTracer in a context manager.
Parameters
session_name (str, optional) – The name of the session.
Defaults to “default”.
Returns
The LangChainTracer session.
Return type
TracerSessionV1
Example
>>> with tracing_enabled() as session:
...     # Use the LangChainTracer session


langchain.callbacks.mlflow_callback.construct_html_from_prompt_and_generation¶
langchain.callbacks.mlflow_callback.construct_html_from_prompt_and_generation(prompt: str, generation: str) → Any[source]¶
Construct an html element from a prompt and a generation.
Parameters
prompt (str) – The prompt.
generation (str) – The generation.
Returns
The html string.
Return type
(str)