Agents & Tools

Transformers Agent is an experimental API which is subject to change at any time. Results returned by the agents can vary as the APIs or underlying models are prone to change.

To learn more about agents and tools make sure to read the [introductory guide](../transformers_agents). This page contains the API docs for the underlying classes.

## [](#agents)Agents

We provide three types of agents: [HfAgent](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.HfAgent) uses inference endpoints for opensource models, [LocalAgent](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.LocalAgent) uses a model of your choice locally and [OpenAiAgent](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.OpenAiAgent) uses OpenAI closed models.

### [](#transformers.HfAgent)HfAgent

### class transformers.HfAgent

[](#transformers.HfAgent)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L582)

( url\_endpointtoken = Nonechat\_prompt\_template = Nonerun\_prompt\_template = Noneadditional\_tools = None )

Parameters

-   [](#transformers.HfAgent.url_endpoint)**url\_endpoint** (`str`) — The name of the url endpoint to use.
-   [](#transformers.HfAgent.token)**token** (`str`, _optional_) — The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
-   [](#transformers.HfAgent.chat_prompt_template)**chat\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case.
-   [](#transformers.HfAgent.run_prompt_template)**run\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case.
-   [](#transformers.HfAgent.additional_tools)**additional\_tools** ([Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool), list of tools or dictionary with tool values, _optional_) — Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden.

Agent that uses an inference endpoint to generate code.

Example:

```
from transformers import HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!")```

### [](#transformers.LocalAgent)LocalAgent

### class transformers.LocalAgent

[](#transformers.LocalAgent)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L653)

( modeltokenizerchat\_prompt\_template = Nonerun\_prompt\_template = Noneadditional\_tools = None )

Parameters

-   [](#transformers.LocalAgent.model)**model** ([PreTrainedModel](/docs/transformers/v4.30.0/en/main_classes/model#transformers.PreTrainedModel)) — The model to use for the agent.
-   [](#transformers.LocalAgent.tokenizer)**tokenizer** ([PreTrainedTokenizer](/docs/transformers/v4.30.0/en/main_classes/tokenizer#transformers.PreTrainedTokenizer)) — The tokenizer to use for the agent.
-   [](#transformers.LocalAgent.chat_prompt_template)**chat\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case.
-   [](#transformers.LocalAgent.run_prompt_template)**run\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case.
-   [](#transformers.LocalAgent.additional_tools)**additional\_tools** ([Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool), list of tools or dictionary with tool values, _optional_) — Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden.

Agent that uses a local model and tokenizer to generate code.

Example:

```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, LocalAgent

checkpoint = "bigcode/starcoder"
model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

agent = LocalAgent(model, tokenizer)
agent.run("Draw me a picture of rivers and lakes.")```

#### from\_pretrained

[](#transformers.LocalAgent.from_pretrained)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L698)

( pretrained\_model\_name\_or\_path\*\*kwargs )

Parameters

-   [](#transformers.LocalAgent.from_pretrained.pretrained_model_name_or_path)**pretrained\_model\_name\_or\_path** (`str` or `os.PathLike`) — The name of a repo on the Hub or a local path to a folder containing both model and tokenizer. kwargs — Keyword arguments passed along to [from\_pretrained()](/docs/transformers/v4.30.0/en/main_classes/model#transformers.PreTrainedModel.from_pretrained).

Convenience method to build a `LocalAgent` from a pretrained checkpoint.

Example:

```
import torch
from transformers import LocalAgent

agent = LocalAgent.from_pretrained("bigcode/starcoder", device_map="auto", torch_dtype=torch.bfloat16)
agent.run("Draw me a picture of rivers and lakes.")```

### [](#transformers.OpenAiAgent)OpenAiAgent

### class transformers.OpenAiAgent

[](#transformers.OpenAiAgent)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L358)

( model = 'text-davinci-003'api\_key = Nonechat\_prompt\_template = Nonerun\_prompt\_template = Noneadditional\_tools = None )

Parameters

-   [](#transformers.OpenAiAgent.model)**model** (`str`, _optional_, defaults to `"text-davinci-003"`) — The name of the OpenAI model to use.
-   [](#transformers.OpenAiAgent.api_key)**api\_key** (`str`, _optional_) — The API key to use. If unset, will look for the environment variable `"OPENAI_API_KEY"`.
-   [](#transformers.OpenAiAgent.chat_prompt_template)**chat\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case.
-   [](#transformers.OpenAiAgent.run_prompt_template)**run\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case.
-   [](#transformers.OpenAiAgent.additional_tools)**additional\_tools** ([Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool), list of tools or dictionary with tool values, _optional_) — Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden.

Agent that uses the openai API to generate code.

The openAI models are used in generation mode, so even for the `chat()` API, it’s better to use models like `"text-davinci-003"` over the chat-GPT variant. Proper support for chat-GPT models will come in a next version.

Example:

```
from transformers import OpenAiAgent

agent = OpenAiAgent(model="text-davinci-003", api_key=xxx)
agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!")```

### [](#transformers.AzureOpenAiAgent)AzureOpenAiAgent

### class transformers.AzureOpenAiAgent

[](#transformers.AzureOpenAiAgent)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L456)

( deployment\_idapi\_key = Noneresource\_name = Noneapi\_version = '2022-12-01'is\_chat\_model = Nonechat\_prompt\_template = Nonerun\_prompt\_template = Noneadditional\_tools = None )

Agent that uses Azure OpenAI to generate code. See the [official documentation](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/) to learn how to deploy an openAI model on Azure

The openAI models are used in generation mode, so even for the `chat()` API, it’s better to use models like `"text-davinci-003"` over the chat-GPT variant. Proper support for chat-GPT models will come in a next version.

Example:

```
from transformers import AzureOpenAiAgent

agent = AzureAiAgent(deployment_id="Davinci-003", api_key=xxx, resource_name=yyy)
agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!")```

### [](#transformers.Agent)Agent

### class transformers.Agent

[](#transformers.Agent)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L190)

( chat\_prompt\_template = Nonerun\_prompt\_template = Noneadditional\_tools = None )

Parameters

-   [](#transformers.Agent.chat_prompt_template)**chat\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case.
-   [](#transformers.Agent.run_prompt_template)**run\_prompt\_template** (`str`, _optional_) — Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case.
-   [](#transformers.Agent.additional_tools)**additional\_tools** ([Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool), list of tools or dictionary with tool values, _optional_) — Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden.

Base class for all agents which contains the main API methods.

#### chat

[](#transformers.Agent.chat)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L262)

( taskreturn\_code = Falseremote = False\*\*kwargs )

Parameters

-   [](#transformers.Agent.chat.task)**task** (`str`) — The task to perform
-   [](#transformers.Agent.chat.return_code)**return\_code** (`bool`, _optional_, defaults to `False`) — Whether to just return code and not evaluate it.
-   [](#transformers.Agent.chat.remote)**remote** (`bool`, _optional_, defaults to `False`) — Whether or not to use remote tools (inference endpoints) instead of local ones.
-   [](#transformers.Agent.chat.kwargs)**kwargs** (additional keyword arguments, _optional_) — Any keyword argument to send to the agent when evaluating the code.

Sends a new request to the agent in a chat. Will use the previous ones in its history.

Example:

```
from transformers import HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
agent.chat("Draw me a picture of rivers and lakes")

agent.chat("Transform the picture so that there is a rock in there")```

#### run

[](#transformers.Agent.run)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/agents.py#L312)

( taskreturn\_code = Falseremote = False\*\*kwargs )

Parameters

-   [](#transformers.Agent.run.task)**task** (`str`) — The task to perform
-   [](#transformers.Agent.run.return_code)**return\_code** (`bool`, _optional_, defaults to `False`) — Whether to just return code and not evaluate it.
-   [](#transformers.Agent.run.remote)**remote** (`bool`, _optional_, defaults to `False`) — Whether or not to use remote tools (inference endpoints) instead of local ones.
-   [](#transformers.Agent.run.kwargs)**kwargs** (additional keyword arguments, _optional_) — Any keyword argument to send to the agent when evaluating the code.

Sends a request to the agent.

Example:

```
from transformers import HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
agent.run("Draw me a picture of rivers and lakes")```

Clears the history of prior calls to [chat()](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Agent.chat).

## [](#tools)Tools

### [](#transformers.load_tool)load\_tool

#### transformers.load\_tool

[](#transformers.load_tool)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L629)

( task\_or\_repo\_idmodel\_repo\_id = Noneremote = Falsetoken = None\*\*kwargs )

Main function to quickly load a tool, be it on the Hub or in the Transformers library.

### [](#transformers.Tool)Tool

### class transformers.Tool

[](#transformers.Tool)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L80)

( \*args\*\*kwargs )

A base class for the functions used by the agent. Subclass this and implement the `__call__` method as well as the following class attributes:

-   **description** (`str`) — A short description of what your tool does, the inputs it expects and the output(s) it will return. For instance ‘This is a tool that downloads a file from a `url`. It takes the `url` as input, and returns the text contained in the file’.
-   **name** (`str`) — A performative name that will be used for your tool in the prompt to the agent. For instance `"text-classifier"` or `"image_generator"`.
-   **inputs** (`List[str]`) — The list of modalities expected for the inputs (in the same order as in the call). Modalitiies should be `"text"`, `"image"` or `"audio"`. This is only used by `launch_gradio_demo` or to make a nice space from your tool.
-   **outputs** (`List[str]`) — The list of modalities returned but the tool (in the same order as the return of the call method). Modalitiies should be `"text"`, `"image"` or `"audio"`. This is only used by `launch_gradio_demo` or to make a nice space from your tool.

You can also override the method [setup()](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool.setup) if your tool as an expensive operation to perform before being usable (such as loading a model). [setup()](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool.setup) will be called the first time you use your tool, but not at instantiation.

Creates a [Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool) from a gradio tool.

#### from\_hub

[](#transformers.Tool.from_hub)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L175)

( repo\_id: strmodel\_repo\_id: typing.Optional\[str\] = Nonetoken: typing.Optional\[str\] = Noneremote: bool = False\*\*kwargs )

Parameters

-   [](#transformers.Tool.from_hub.repo_id)**repo\_id** (`str`) — The name of the repo on the Hub where your tool is defined.
-   [](#transformers.Tool.from_hub.model_repo_id)**model\_repo\_id** (`str`, _optional_) — If your tool uses a model and you want to use a different model than the default, you can pass a second repo ID or an endpoint url to this argument.
-   [](#transformers.Tool.from_hub.token)**token** (`str`, _optional_) — The token to identify you on hf.co. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
-   [](#transformers.Tool.from_hub.remote)**remote** (`bool`, _optional_, defaults to `False`) — Whether to use your tool by downloading the model or (if it is available) with an inference endpoint.
-   [](#transformers.Tool.from_hub.kwargs)**kwargs** (additional keyword arguments, _optional_) — Additional keyword arguments that will be split in two: all arguments relevant to the Hub (such as `cache_dir`, `revision`, `subfolder`) will be used when downloading the files for your tool, and the others will be passed along to its init.

Loads a tool defined on the Hub.

#### push\_to\_hub

[](#transformers.Tool.push_to_hub)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L285)

( repo\_id: strcommit\_message: str = 'Upload tool'private: typing.Optional\[bool\] = Nonetoken: typing.Union\[bool, str, NoneType\] = Nonecreate\_pr: bool = False )

Parameters

-   [](#transformers.Tool.push_to_hub.repo_id)**repo\_id** (`str`) — The name of the repository you want to push your tool to. It should contain your organization name when pushing to a given organization.
-   [](#transformers.Tool.push_to_hub.commit_message)**commit\_message** (`str`, _optional_, defaults to `"Upload tool"`) — Message to commit while pushing.
-   [](#transformers.Tool.push_to_hub.private)**private** (`bool`, _optional_) — Whether or not the repository created should be private.
-   [](#transformers.Tool.push_to_hub.token)**token** (`bool` or `str`, _optional_) — The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
-   [](#transformers.Tool.push_to_hub.create_pr)**create\_pr** (`bool`, _optional_, defaults to `False`) — Whether or not to create a PR with the uploaded files or directly commit.

Upload the tool to the Hub.

#### save

[](#transformers.Tool.save)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L121)

( output\_dir )

Parameters

-   [](#transformers.Tool.save.output_dir)**output\_dir** (`str`) — The folder in which you want to save your tool.

Saves the relevant code files for your tool so it can be pushed to the Hub. This will copy the code of your tool in `output_dir` as well as autogenerate:

-   a config file named `tool_config.json`
-   an `app.py` file so that your tool can be converted to a space
-   a `requirements.txt` containing the names of the module used by your tool (as detected when inspecting its code)

You should only use this method to save tools that are defined in a separate module (not `__main__`).

Overwrite this method here for any operation that is expensive and needs to be executed before you start using your tool. Such as loading a big model.

### [](#transformers.PipelineTool)PipelineTool

### class transformers.PipelineTool

[](#transformers.PipelineTool)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L427)

( model = Nonepre\_processor = Nonepost\_processor = Nonedevice = Nonedevice\_map = Nonemodel\_kwargs = Nonetoken = None\*\*hub\_kwargs )

A [Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool) tailored towards Transformer models. On top of the class attributes of the base class [Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool), you will need to specify:

-   **model\_class** (`type`) — The class to use to load the model in this tool.
-   **default\_checkpoint** (`str`) — The default checkpoint that should be used when the user doesn’t specify one.
-   **pre\_processor\_class** (`type`, _optional_, defaults to [AutoProcessor](/docs/transformers/v4.30.0/en/model_doc/auto#transformers.AutoProcessor)) — The class to use to load the pre-processor
-   **post\_processor\_class** (`type`, _optional_, defaults to [AutoProcessor](/docs/transformers/v4.30.0/en/model_doc/auto#transformers.AutoProcessor)) — The class to use to load the post-processor (when different from the pre-processor).

Uses the `post_processor` to decode the model output.

Uses the `pre_processor` to prepare the inputs for the `model`.

Sends the inputs through the `model`.

Instantiates the `pre_processor`, `model` and `post_processor` if necessary.

### [](#transformers.RemoteTool)RemoteTool

### class transformers.RemoteTool

[](#transformers.RemoteTool)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L345)

( endpoint\_url = Nonetoken = Nonetool\_class = None )

Parameters

-   [](#transformers.RemoteTool.endpoint_url)**endpoint\_url** (`str`) — The url of the endpoint to use.
-   [](#transformers.RemoteTool.token)**token** (`str`, _optional_) — The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`).
-   [](#transformers.RemoteTool.tool_class)**tool\_class** (`type`, _optional_) — The corresponding `tool_class` if this is a remote version of an existing tool. Will help determine when the output should be converted to another type (like images).

A [Tool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.Tool) that will make requests to an inference endpoint.

You can override this method in your custom class of [RemoteTool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.RemoteTool) to apply some custom post-processing of the outputs of the endpoint.

Prepare the inputs received for the HTTP client sending data to the endpoint. Positional arguments will be matched with the signature of the `tool_class` if it was provided at instantation. Images will be encoded into bytes.

You can override this method in your custom class of [RemoteTool](/docs/transformers/v4.30.0/en/main_classes/agent#transformers.RemoteTool).

### [](#transformers.launch_gradio_demo)launch\_gradio\_demo

#### transformers.launch\_gradio\_demo

[](#transformers.launch_gradio_demo)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/tools/base.py#L563)

( tool\_class: Tool )

Parameters

-   [](#transformers.launch_gradio_demo.tool_class)**tool\_class** (`type`) — The class of the tool for which to launch the demo.

Launches a gradio demo for a tool. The corresponding tool class needs to properly implement the class attributes `inputs` and `outputs`.