Spaces:
Paused
A newer version of the Gradio SDK is available:
4.42.0
Agents & Tools
Transformers Agents is an experimental API which is subject to change at any time. Results returned by the agents can vary as the APIs or underlying models are prone to change.
To learn more about agents and tools make sure to read the introductory guide. This page contains the API docs for the underlying classes.
Agents
We provide three types of agents: [HfAgent
] uses inference endpoints for opensource models, [LocalAgent
] uses a model of your choice locally and [OpenAiAgent
] uses OpenAI closed models.
HfAgent
[[autodoc]] HfAgent
LocalAgent
[[autodoc]] LocalAgent
OpenAiAgent
[[autodoc]] OpenAiAgent
AzureOpenAiAgent
[[autodoc]] AzureOpenAiAgent
Agent
[[autodoc]] Agent - chat - run - prepare_for_new_chat
Tools
load_tool
[[autodoc]] load_tool
Tool
[[autodoc]] Tool
PipelineTool
[[autodoc]] PipelineTool
RemoteTool
[[autodoc]] RemoteTool
launch_gradio_demo
[[autodoc]] launch_gradio_demo
Agent Types
Agents can handle any type of object in-between tools; tools, being completely multimodal, can accept and return text, image, audio, video, among other types. In order to increase compatibility between tools, as well as to correctly render these returns in ipython (jupyter, colab, ipython notebooks, ...), we implement wrapper classes around these types.
The wrapped objects should continue behaving as initially; a text object should still behave as a string, an image
object should still behave as a PIL.Image
.
These types have three specific purposes:
- Calling
to_raw
on the type should return the underlying object - Calling
to_string
on the type should return the object as a string: that can be the string in case of anAgentText
but will be the path of the serialized version of the object in other instances - Displaying it in an ipython kernel should display the object correctly
AgentText
[[autodoc]] transformers.tools.agent_types.AgentText
AgentImage
[[autodoc]] transformers.tools.agent_types.AgentImage
AgentAudio
[[autodoc]] transformers.tools.agent_types.AgentAudio