TASK:

Review the following documentation. Our objective is to build an AI-first Semantic SEO article writer using the no-code tool Bubble.io. We aim to leverage autonomous agents focused on various SEO and content tasks. Your goal is to brainstorm the ideal framework and database structure to implement these concepts effectively.

INSTRUCTIONS:

1. Understand the Project Scope:

- Autonomous Agents:
- The system will consist of multiple autonomous agents performing specific SEO and content-related tasks without human intervention.
- These agents will handle tasks such as keyword research, content generation, optimization, and analysis.
- Agents will run sequentially based on a predefined workflow (crew process) established by the admin.

- Admin Backend Configuration:
- The SaaS admin will have access to an admin dashboard to configure agents, prompts, workflows, and tasks.
- All prompt engineering and agent configurations, including OpenAI's Structured Outputs, will be managed here.
- The admin can define workflows, set agent execution order, and adjust settings as needed.

- User Frontend Interface:
- The frontend will provide users with forms to initiate agents and input required information.
- Users can trigger specific workflows or tasks, supplying any necessary data.
- The frontend will display outputs generated by the agents, such as drafted articles or SEO analysis reports.

2. Architectural Considerations:

- Leverage Bubble.io Best Practices:
- Utilize Bubble.io's capabilities for building scalable and maintainable applications.
- Follow methodologies related to Workflow Units to optimize performance.
- Implement modern no-code development practices to ensure flexibility and ease of updates.

- Workflow Management:
- Design a dynamic workflow system where agents execute based on the crew process defined in the admin settings.
- Ensure the workflow is modular, allowing the admin to add, remove, or reorder agents without significant rework.
- Workflows should be able to handle conditional logic, looping, and parallel processing if necessary.

- Integration of OpenAI's Structured Outputs:
- Incorporate OpenAI's Structured Outputs to ensure agent outputs are formatted consistently.
- This will facilitate easier parsing, display, and further processing of the generated content.
- Structured outputs help in maintaining data integrity and improve the overall efficiency of the system.

3. Database Structure Design:

- Data Models and Fields:

- Users:
- Stores information about each user of the platform.
- Fields:
- User_ID (unique identifier)
- Name
- Email
- Password (hashed)
- Role (e.g., admin, user)
- Date_Created
- Last_Login

- Agents:
- Represents each autonomous agent in the system.
- Fields:
- Agent_ID
- Agent_Name
- Description
- Configuration_Settings (JSON or structured data)
- Status (active/inactive)
- Execution_Order (for workflow sequencing)

- Workflows:
- Defines agent execution sequences.
- Fields:
- Workflow_ID
- Workflow_Name
- Description
- Agent_Sequence (list of Agent_IDs in order)
- Created_By (User_ID of admin)
- Date_Created
- Last_Modified

- Prompts:
- Contains prompt templates and configurations for OpenAI API calls.
- Fields:
- Prompt_ID
- Agent_ID (associated agent)
- Template_Text
- Variables (placeholders to be replaced with dynamic data)
- Structured_Output_Format (definition of expected output)

- Tasks:
- Instances of workflows initiated by users.
- Fields:
- Task_ID
- User_ID (who initiated the task)
- Workflow_ID
- Input_Data (user-provided or agent-generated)
- Output_Data (results from agents)
- Status (pending, in progress, completed, failed)
- Start_Time
- End_Time

- Settings:
- Stores global configurations and preferences.
- Fields:
- Setting_ID
- Key
- Value
- Description

- Relationships:

- Users can initiate multiple Tasks.
- Agents are linked to Prompts and are part of Workflows.
- Workflows consist of an ordered sequence of Agents.
- Tasks are executions of Workflows and generate Output_Data.

4. Step-by-Step Framework Development:

- Step 1: Database Setup in Bubble.io

- Create data types for all the models listed above.
- Define fields with appropriate data types (text, number, date, list, etc.).
- Set up data privacy rules to protect sensitive information.

- Step 2: Admin Dashboard Development

- Agents Management:
- Build interfaces to create, edit, activate/deactivate agents.
- Allow configuration of Configuration_Settings and integration with OpenAI prompts.

- Workflows Management:
- Enable admins to create and modify workflows, arranging agents in desired sequences.
- Provide visualization of workflows for better understanding.

- Prompts Configuration:
- Implement a prompt editor where admins can define Template_Text, Variables, and Structured_Output_Format.
- Include testing capabilities to validate prompts with sample data.

- Step 3: Frontend User Interface

- Task Initiation Forms:
- Design intuitive forms for users to input data required to start tasks.
- Forms should dynamically adjust based on the selected workflow.

- Progress Tracking:
- Provide real-time updates on task status (e.g., in progress, completed).
- Display logs or output data as agents complete their tasks.

- Output Display:
- Present the final generated content in a user-friendly format.
- Allow users to download results or further interact with the content (e.g., editing).

- Step 4: Workflow and Agent Execution Logic

- Workflow Orchestration:
- Utilize Bubble.io's backend workflows to manage agent execution in the defined order.
- Ensure that data flows correctly between agents, passing outputs as inputs when necessary.

- Agent Integration:
- Set up API connections with OpenAI for agents that require AI capabilities.
- Implement error handling and retry mechanisms for API calls.

- Structured Output Handling:
- Parse the structured outputs returned by OpenAI and store them appropriately.
- Use Bubble.io's data manipulation tools to handle JSON or other structured data formats.

- Step 5: Incorporate OpenAI's Structured Outputs

- Prompt Engineering:
- Design prompts to elicit structured responses using OpenAI's guidelines.
- Define clear output formats (e.g., JSON) that simplify parsing and data extraction.

- API Configuration:
- Set up API connectors in Bubble.io with necessary authentication.
- Configure requests to include parameters for structured output.

- Data Processing:
- Create workflows to automatically parse and utilize the structured outputs.
- Map parsed data to the corresponding fields in the database.

- Step 6: Testing and Optimization

- Functional Testing:
- Test each agent individually to ensure it behaves as expected.
- Run end-to-end tests of workflows to validate the entire process.

- Performance Optimization:
- Monitor workflow execution times and optimize queries or operations causing delays.
- Use Bubble.io’s debugging tools to identify and fix issues.

- User Feedback Loop:
- Implement mechanisms for users to provide feedback on outputs.
- Use feedback to refine prompts, agent configurations, and workflows.

5. Adhering to Best Practices:

- Workflow Units Optimization:
- Minimize the number of actions in a single workflow to enhance performance.
- Use custom events and reusable elements to streamline processes.

- Scalability Considerations:
- Design the system to handle increased load, such as more concurrent users or larger data volumes.
- Plan for database indexing and efficient data retrieval methods.

- Security Measures:
- Secure API keys and sensitive configurations using Bubble.io’s secret management.
- Implement user authentication and role-based access control.

- Maintainability:
- Write clear documentation for workflows, data structures, and configurations.
- Use consistent naming conventions for all elements in Bubble.io.

6. Additional Recommendations:

- User Experience Enhancement:
- Provide tutorials or tooltips to guide new users through the process.
- Implement a clean, responsive design that works well on various devices.

- Analytics and Reporting:
- Include dashboards for admins to monitor system usage, agent performance, and other key metrics.
- Collect data that can inform future improvements.

- Continuous Improvement:
- Stay updated with Bubble.io and OpenAI updates to leverage new features.
- Regularly review and refine prompts and agent configurations based on performance data.

- Support and Help Resources:
- Offer a help center or FAQ section to assist users with common issues.
- Provide contact options for user support.

OBJECTIVE:

Develop a comprehensive framework and database structure for an AI-first Semantic SEO article writer utilizing Bubble.io. The system should seamlessly integrate autonomous agents, managed via an admin backend, with a user-friendly frontend interface. By following best practices, leveraging OpenAI's Structured Outputs, and ensuring clarity in design and execution, we'll create a robust, scalable solution that empowers users to generate high-quality SEO content efficiently.
"""
Agents
What is an Agent?¶
What is an Agent?

An agent is an autonomous unit programmed to:

Perform tasks
Make decisions
Communicate with other agents

Think of an agent as a member of a team, with specific skills and a particular job to do. Agents can have different roles like 'Researcher', 'Writer', or 'Customer Support', each contributing to the overall goal of the crew.
Agent Attributes¶
Attribute	Parameter	Description
Role	role	Defines the agent's function within the crew. It determines the kind of tasks the agent is best suited for.
Goal	goal	The individual objective that the agent aims to achieve. It guides the agent's decision-making process.
Backstory	backstory	Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics.
LLM (optional)	llm	Represents the language model that will run the agent. It dynamically fetches the model name from the OPENAI_MODEL_NAME environment variable, defaulting to "gpt-4" if not specified.
Tools (optional)	tools	Set of capabilities or functions that the agent can use to perform tasks. Expected to be instances of custom classes compatible with the agent's execution environment. Tools are initialized with a default value of an empty list.
Function Calling LLM (optional)	function_calling_llm	Specifies the language model that will handle the tool calling for this agent, overriding the crew function calling LLM if passed. Default is None.
Max Iter (optional)	max_iter	Max Iter is the maximum number of iterations the agent can perform before being forced to give its best answer. Default is 25.
Max RPM (optional)	max_rpm	Max RPM is the maximum number of requests per minute the agent can perform to avoid rate limits. It's optional and can be left unspecified, with a default value of None.
Max Execution Time (optional)	max_execution_time	Max Execution Time is the maximum execution time for an agent to execute a task. It's optional and can be left unspecified, with a default value of None, meaning no max execution time.
Verbose (optional)	verbose	Setting this to True configures the internal logger to provide detailed execution logs, aiding in debugging and monitoring. Default is False.
Allow Delegation (optional)	allow_delegation	Agents can delegate tasks or questions to one another, ensuring that each task is handled by the most suitable agent. Default is False.
Step Callback (optional)	step_callback	A function that is called after each step of the agent. This can be used to log the agent's actions or to perform other operations. It will overwrite the crew step_callback.
Cache (optional)	cache	Indicates if the agent should use a cache for tool usage. Default is True.
System Template (optional)	system_template	Specifies the system format for the agent. Default is None.
Prompt Template (optional)	prompt_template	Specifies the prompt format for the agent. Default is None.
Response Template (optional)	response_template	Specifies the response format for the agent. Default is None.
Allow Code Execution (optional)	allow_code_execution	Enable code execution for the agent. Default is False.
Max Retry Limit (optional)	max_retry_limit	Maximum number of retries for an agent to execute a task when an error occurs. Default is 2.
Use Stop Words (optional)	use_stop_words	Adds the ability to not use stop words (to support o1 models). Default is True.
Use System Prompt (optional)	use_system_prompt	Adds the ability to not use system prompt (to support o1 models). Default is True.
Respect Context Window (optional)	respect_context_window	Summary strategy to avoid overflowing the context window. Default is True.
Creating an Agent¶
Agent Interaction

Agents can interact with each other using crewAI's built-in delegation and communication mechanisms. This allows for dynamic task management and problem-solving within the crew.

To create an agent, you would typically initialize an instance of the Agent class with the desired properties. Here's a conceptual example including all attributes:


# Example: Creating an agent with all attributes
from crewai import Agent

agent = Agent(
  role='Data Analyst',
  goal='Extract actionable insights',
  backstory="""You're a data analyst at a large company.
  You're responsible for analyzing data and providing insights
  to the business.
  You're currently working on a project to analyze the
  performance of our marketing campaigns.""",
  tools=[my_tool1, my_tool2],  # Optional, defaults to an empty list
  llm=my_llm,  # Optional
  function_calling_llm=my_llm,  # Optional
  max_iter=15,  # Optional
  max_rpm=None, # Optional
  max_execution_time=None, # Optional
  verbose=True,  # Optional
  allow_delegation=False,  # Optional
  step_callback=my_intermediate_step_callback,  # Optional
  cache=True,  # Optional
  system_template=my_system_template,  # Optional
  prompt_template=my_prompt_template,  # Optional
  response_template=my_response_template,  # Optional
  config=my_config,  # Optional
  crew=my_crew,  # Optional
  tools_handler=my_tools_handler,  # Optional
  cache_handler=my_cache_handler,  # Optional
  callbacks=[callback1, callback2],  # Optional
  allow_code_execution=True,  # Optional
  max_retry_limit=2,  # Optional
  use_stop_words=True,  # Optional
  use_system_prompt=True,  # Optional
  respect_context_window=True,  # Optional
)
Setting prompt templates¶
Prompt templates are used to format the prompt for the agent. You can use to update the system, regular and response templates for the agent. Here's an example of how to set prompt templates:


agent = Agent(
        role="{topic} specialist",
        goal="Figure {goal} out",
        backstory="I am the master of {role}",
        system_template="""
system


{{ .System }}
""",
        prompt_template="""
user


{{ .Prompt }}
""",
        response_template="""
assistant


{{ .Response }}
""",
    )
Bring your Third Party Agents¶
Extend your Third Party Agents like LlamaIndex, Langchain, Autogen or fully custom agents using the the crewai's BaseAgent class.

BaseAgent includes attributes and methods required to integrate with your crews to run and delegate tasks to other agents within your own crew.

CrewAI is a universal multi-agent framework that allows for all agents to work together to automate tasks and solve problems.


from crewai import Agent, Task, Crew
from custom_agent import CustomAgent # You need to build and extend your own agent logic with the CrewAI BaseAgent class then import it here.

from langchain.agents import load_tools

langchain_tools = load_tools(["google-serper"], llm=llm)

agent1 = CustomAgent(
    role="agent role",
    goal="who is {input}?",
    backstory="agent backstory",
    verbose=True,
)

task1 = Task(
    expected_output="a short biography of {input}",
    description="a short biography of {input}",
    agent=agent1,
)

agent2 = Agent(
    role="agent role",
    goal="summarize the short bio for {input} and if needed do more research",
    backstory="agent backstory",
    verbose=True,
)

task2 = Task(
    description="a tldr summary of the short biography",
    expected_output="5 bullet point summary of the biography",
    agent=agent2,
    context=[task1],
)

my_crew = Crew(agents=[agent1, agent2], tasks=[task1, task2])
crew = my_crew.kickoff(inputs={"input": "Mark Twain"})
Conclusion¶
Agents are the building blocks of the CrewAI framework. By understanding how to define and interact with agents, you can create sophisticated AI systems that leverage the power of collaborative intelligence.
"""
Tasks

---
title: crewAI Tasks
description: Detailed guide on managing and creating tasks within the crewAI framework, reflecting the latest codebase updates.
---

## Overview of a Task

!!! note "What is a Task?"
In the crewAI framework, tasks are specific assignments completed by agents. They provide all necessary details for execution, such as a description, the agent responsible, required tools, and more, facilitating a wide range of action complexities.

Tasks within crewAI can be collaborative, requiring multiple agents to work together. This is managed through the task properties and orchestrated by the Crew's process, enhancing teamwork and efficiency.

## Task Attributes

| Attribute                        | Parameters        | Type                          | Description                                                                                                          |
| :------------------------------- | :---------------- | :---------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| **Description**                  | description     | str                         | A clear, concise statement of what the task entails.                                                                 |
| **Agent**                        | agent           | Optional[BaseAgent]         | The agent responsible for the task, assigned either directly or by the crew's process.                               |
| **Expected Output**              | expected_output | str                         | A detailed description of what the task's completion looks like.                                                     |
| **Tools** _(optional)_           | tools           | Optional[List[Any]]         | The functions or capabilities the agent can utilize to perform the task. Defaults to an empty list.                  |
| **Async Execution** _(optional)_ | async_execution | Optional[bool]              | If set, the task executes asynchronously, allowing progression without waiting for completion. Defaults to False.    |
| **Context** _(optional)_         | context         | Optional[List["Task"]]      | Specifies tasks whose outputs are used as context for this task.                                                     |
| **Config** _(optional)_          | config          | Optional[Dict[str, Any]]    | Additional configuration details for the agent executing the task, allowing further customization. Defaults to None. |
| **Output JSON** _(optional)_     | output_json     | Optional[Type[BaseModel]]   | Outputs a JSON object, requiring an OpenAI client. Only one output format can be set.                                |
| **Output Pydantic** _(optional)_ | output_pydantic | Optional[Type[BaseModel]]   | Outputs a Pydantic model object, requiring an OpenAI client. Only one output format can be set.                      |
| **Output File** _(optional)_     | output_file     | Optional[str]               | Saves the task output to a file. If used with Output JSON or Output Pydantic, specifies how the output is saved. |
| **Output** _(optional)_          | output          | Optional[TaskOutput]        | An instance of TaskOutput, containing the raw, JSON, and Pydantic output plus additional details.                  |
| **Callback** _(optional)_        | callback        | Optional[Any]               | A callable that is executed with the task's output upon completion.                                                  |
| **Human Input** _(optional)_     | human_input     | Optional[bool]              | Indicates if the task should involve human review at the end, useful for tasks needing human oversight. Defaults to False.|
| **Converter Class** _(optional)_ | converter_cls   | Optional[Type[Converter]]   | A converter class used to export structured output. Defaults to None.                                                |

## Creating a Task

Creating a task involves defining its scope, responsible agent, and any additional attributes for flexibility:

python
from crewai import Task

task = Task(
    description='Find and summarize the latest and most relevant news on AI',
    agent=sales_agent,
    expected_output='A bullet list summary of the top 5 most important AI news',
)
Task Assignment

Directly specify an agent for assignment or let the hierarchical CrewAI's process decide based on roles, availability, etc.

Task Output¶
Understanding Task Outputs

The output of a task in the crewAI framework is encapsulated within the TaskOutput class. This class provides a structured way to access results of a task, including various formats such as raw output, JSON, and Pydantic models. By default, the TaskOutput will only include the raw output. A TaskOutput will only include the pydantic or json_dict output if the original Task object was configured with output_pydantic or output_json, respectively.

Task Output Attributes¶
Attribute	Parameters	Type	Description
Description	description	str	Description of the task.
Summary	summary	Optional[str]	Summary of the task, auto-generated from the first 10 words of the description.
Raw	raw	str	The raw output of the task. This is the default format for the output.
Pydantic	pydantic	Optional[BaseModel]	A Pydantic model object representing the structured output of the task.
JSON Dict	json_dict	Optional[Dict[str, Any]]	A dictionary representing the JSON output of the task.
Agent	agent	str	The agent that executed the task.
Output Format	output_format	OutputFormat	The format of the task output, with options including RAW, JSON, and Pydantic. The default is RAW.
Task Methods and Properties¶
Method/Property	Description
json	Returns the JSON string representation of the task output if the output format is JSON.
to_dict	Converts the JSON and Pydantic outputs to a dictionary.
str	Returns the string representation of the task output, prioritizing Pydantic, then JSON, then raw.
Accessing Task Outputs¶
Once a task has been executed, its output can be accessed through the output attribute of the Task object. The TaskOutput class provides various ways to interact with and present this output.

Example¶

# Example task
task = Task(
    description='Find and summarize the latest AI news',
    expected_output='A bullet list summary of the top 5 most important AI news',
    agent=research_agent,
    tools=[search_tool]
)

# Execute the crew
crew = Crew(
    agents=[research_agent],
    tasks=[task],
    verbose=True
)

result = crew.kickoff()

# Accessing the task output
task_output = task.output

print(f"Task Description: {task_output.description}")
print(f"Task Summary: {task_output.summary}")
print(f"Raw Output: {task_output.raw}")
if task_output.json_dict:
    print(f"JSON Output: {json.dumps(task_output.json_dict, indent=2)}")
if task_output.pydantic:
    print(f"Pydantic Output: {task_output.pydantic}")
Integrating Tools with Tasks¶
Leverage tools from the crewAI Toolkit and LangChain Tools for enhanced task performance and agent interaction.

Creating a Task with Tools¶

import os
os.environ["OPENAI_API_KEY"] = "Your Key"
os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key

from crewai import Agent, Task, Crew
from crewai_tools import SerperDevTool

research_agent = Agent(
  role='Researcher',
  goal='Find and summarize the latest AI news',
  backstory="""You're a researcher at a large company.
  You're responsible for analyzing data and providing insights
  to the business.""",
  verbose=True
)

# to perform a semantic search for a specified query from a text's content across the internet
search_tool = SerperDevTool()

task = Task(
  description='Find and summarize the latest AI news',
  expected_output='A bullet list summary of the top 5 most important AI news',
  agent=research_agent,
  tools=[search_tool]
)

crew = Crew(
    agents=[research_agent],
    tasks=[task],
    verbose=True
)

result = crew.kickoff()
print(result)
This demonstrates how tasks with specific tools can override an agent's default set for tailored task execution.

Referring to Other Tasks¶
In crewAI, the output of one task is automatically relayed into the next one, but you can specifically define what tasks' output, including multiple, should be used as context for another task.

This is useful when you have a task that depends on the output of another task that is not performed immediately after it. This is done through the context attribute of the task:


# ...

research_ai_task = Task(
    description='Find and summarize the latest AI news',
    expected_output='A bullet list summary of the top 5 most important AI news',
    async_execution=True,
    agent=research_agent,
    tools=[search_tool]
)

research_ops_task = Task(
    description='Find and summarize the latest AI Ops news',
    expected_output='A bullet list summary of the top 5 most important AI Ops news',
    async_execution=True,
    agent=research_agent,
    tools=[search_tool]
)

write_blog_task = Task(
    description="Write a full blog post about the importance of AI and its latest news",
    expected_output='Full blog post that is 4 paragraphs long',
    agent=writer_agent,
    context=[research_ai_task, research_ops_task]
)

#...
Asynchronous Execution¶
You can define a task to be executed asynchronously. This means that the crew will not wait for it to be completed to continue with the next task. This is useful for tasks that take a long time to be completed, or that are not crucial for the next tasks to be performed.

You can then use the context attribute to define in a future task that it should wait for the output of the asynchronous task to be completed.


#...

list_ideas = Task(
    description="List of 5 interesting ideas to explore for an article about AI.",
    expected_output="Bullet point list of 5 ideas for an article.",
    agent=researcher,
    async_execution=True # Will be executed asynchronously
)

list_important_history = Task(
    description="Research the history of AI and give me the 5 most important events.",
    expected_output="Bullet point list of 5 important events.",
    agent=researcher,
    async_execution=True # Will be executed asynchronously
)

write_article = Task(
    description="Write an article about AI, its history, and interesting ideas.",
    expected_output="A 4 paragraph article about AI.",
    agent=writer,
    context=[list_ideas, list_important_history] # Will wait for the output of the two tasks to be completed
)

#...
Callback Mechanism¶
The callback function is executed after the task is completed, allowing for actions or notifications to be triggered based on the task's outcome.


# ...

def callback_function(output: TaskOutput):
    # Do something after the task is completed
    # Example: Send an email to the manager
    print(f"""
        Task completed!
        Task: {output.description}
        Output: {output.raw}
    """)

research_task = Task(
    description='Find and summarize the latest AI news',
    expected_output='A bullet list summary of the top 5 most important AI news',
    agent=research_agent,
    tools=[search_tool],
    callback=callback_function
)

#...
Accessing a Specific Task Output¶
Once a crew finishes running, you can access the output of a specific task by using the output attribute of the task object:


# ...
task1 = Task(
    description='Find and summarize the latest AI news',
    expected_output='A bullet list summary of the top 5 most important AI news',
    agent=research_agent,
    tools=[search_tool]
)

#...

crew = Crew(
    agents=[research_agent],
    tasks=[task1, task2, task3],
    verbose=True
)

result = crew.kickoff()

# Returns a TaskOutput object with the description and results of the task
print(f"""
    Task completed!
    Task: {task1.output.description}
    Output: {task1.output.raw}
""")
Tool Override Mechanism¶
Specifying tools in a task allows for dynamic adaptation of agent capabilities, emphasizing CrewAI's flexibility.

Error Handling and Validation Mechanisms¶
While creating and executing tasks, certain validation mechanisms are in place to ensure the robustness and reliability of task attributes. These include but are not limited to:

Ensuring only one output type is set per task to maintain clear output expectations.
Preventing the manual assignment of the id attribute to uphold the integrity of the unique identifier system.
These validations help in maintaining the consistency and reliability of task executions within the crewAI framework.

Creating Directories when Saving Files¶
You can now specify if a task should create directories when saving its output to a file. This is particularly useful for organizing outputs and ensuring that file paths are correctly structured.


# ...

save_output_task = Task(
    description='Save the summarized AI news to a file',
    expected_output='File saved successfully',
    agent=research_agent,
    tools=[file_save_tool],
    output_file='outputs/ai_news_summary.txt',
    create_directory=True
)

#...
Conclusion¶
Tasks are the driving force behind the actions of agents in crewAI. By properly defining tasks and their outcomes, you set the stage for your AI agents to work effectively, either independently or as a collaborative unit. Equipping tasks with appropriate tools, understanding the execution process, and following robust validation practices are crucial for maximizing CrewAI's potential, ensuring agents are effectively prepared for their assignments and that tasks are executed as intended.
"""
Tools
Introduction¶
CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers. This documentation outlines how to create, integrate, and leverage these tools within the CrewAI framework, including a new focus on collaboration tools.

What is a Tool?¶
Definition

A tool in CrewAI is a skill or function that agents can utilize to perform various actions. This includes tools from the crewAI Toolkit and LangChain Tools, enabling everything from simple searches to complex interactions and effective teamwork among agents.

Key Characteristics of Tools¶
Utility: Crafted for tasks such as web searching, data analysis, content generation, and agent collaboration.
Integration: Boosts agent capabilities by seamlessly integrating tools into their workflow.
Customizability: Provides the flexibility to develop custom tools or utilize existing ones, catering to the specific needs of agents.
Error Handling: Incorporates robust error handling mechanisms to ensure smooth operation.
Caching Mechanism: Features intelligent caching to optimize performance and reduce redundant operations.
Using crewAI Tools¶
To enhance your agents' capabilities with crewAI tools, begin by installing our extra tools package:


pip install 'crewai[tools]'
Here's an example demonstrating their use:


import os
from crewai import Agent, Task, Crew
# Importing crewAI tools
from crewai_tools import (
    DirectoryReadTool,
    FileReadTool,
    SerperDevTool,
    WebsiteSearchTool
)

# Set up API keys
os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
os.environ["OPENAI_API_KEY"] = "Your Key"

# Instantiate tools
docs_tool = DirectoryReadTool(directory='./blog-posts')
file_tool = FileReadTool()
search_tool = SerperDevTool()
web_rag_tool = WebsiteSearchTool()

# Create agents
researcher = Agent(
    role='Market Research Analyst',
    goal='Provide up-to-date market analysis of the AI industry',
    backstory='An expert analyst with a keen eye for market trends.',
    tools=[search_tool, web_rag_tool],
    verbose=True
)

writer = Agent(
    role='Content Writer',
    goal='Craft engaging blog posts about the AI industry',
    backstory='A skilled writer with a passion for technology.',
    tools=[docs_tool, file_tool],
    verbose=True
)

# Define tasks
research = Task(
    description='Research the latest trends in the AI industry and provide a summary.',
    expected_output='A summary of the top 3 trending developments in the AI industry with a unique perspective on their significance.',
    agent=researcher
)

write = Task(
    description='Write an engaging blog post about the AI industry, based on the research analyst’s summary. Draw inspiration from the latest blog posts in the directory.',
    expected_output='A 4-paragraph blog post formatted in markdown with engaging, informative, and accessible content, avoiding complex jargon.',
    agent=writer,
    output_file='blog-posts/new_post.md'  # The final blog post will be saved here
)

# Assemble a crew with planning enabled
crew = Crew(
    agents=[researcher, writer],
    tasks=[research, write],
    verbose=True,
    planning=True,  # Enable planning feature
)

# Execute tasks
crew.kickoff()
Available crewAI Tools¶
Error Handling: All tools are built with error handling capabilities, allowing agents to gracefully manage exceptions and continue their tasks.
Caching Mechanism: All tools support caching, enabling agents to efficiently reuse previously obtained results, reducing the load on external resources and speeding up the execution time. You can also define finer control over the caching mechanism using the cache_function attribute on the tool.
Here is a list of the available tools and their descriptions:

Tool	Description
BrowserbaseLoadTool	A tool for interacting with and extracting data from web browsers.
CodeDocsSearchTool	A RAG tool optimized for searching through code documentation and related technical documents.
CodeInterpreterTool	A tool for interpreting python code.
ComposioTool	Enables use of Composio tools.
CSVSearchTool	A RAG tool designed for searching within CSV files, tailored to handle structured data.
DALL-E Tool	A tool for generating images using the DALL-E API.
DirectorySearchTool	A RAG tool for searching within directories, useful for navigating through file systems.
DOCXSearchTool	A RAG tool aimed at searching within DOCX documents, ideal for processing Word files.
DirectoryReadTool	Facilitates reading and processing of directory structures and their contents.
EXASearchTool	A tool designed for performing exhaustive searches across various data sources.
FileReadTool	Enables reading and extracting data from files, supporting various file formats.
FirecrawlSearchTool	A tool to search webpages using Firecrawl and return the results.
FirecrawlCrawlWebsiteTool	A tool for crawling webpages using Firecrawl.
FirecrawlScrapeWebsiteTool	A tool for scraping webpages URL using Firecrawl and returning its contents.
GithubSearchTool	A RAG tool for searching within GitHub repositories, useful for code and documentation search.
SerperDevTool	A specialized tool for development purposes, with specific functionalities under development.
TXTSearchTool	A RAG tool focused on searching within text (.txt) files, suitable for unstructured data.
JSONSearchTool	A RAG tool designed for searching within JSON files, catering to structured data handling.
LlamaIndexTool	Enables the use of LlamaIndex tools.
MDXSearchTool	A RAG tool tailored for searching within Markdown (MDX) files, useful for documentation.
PDFSearchTool	A RAG tool aimed at searching within PDF documents, ideal for processing scanned documents.
PGSearchTool	A RAG tool optimized for searching within PostgreSQL databases, suitable for database queries.
Vision Tool	A tool for generating images using the DALL-E API.
RagTool	A general-purpose RAG tool capable of handling various data sources and types.
ScrapeElementFromWebsiteTool	Enables scraping specific elements from websites, useful for targeted data extraction.
ScrapeWebsiteTool	Facilitates scraping entire websites, ideal for comprehensive data collection.
WebsiteSearchTool	A RAG tool for searching website content, optimized for web data extraction.
XMLSearchTool	A RAG tool designed for searching within XML files, suitable for structured data formats.
YoutubeChannelSearchTool	A RAG tool for searching within YouTube channels, useful for video content analysis.
YoutubeVideoSearchTool	A RAG tool aimed at searching within YouTube videos, ideal for video data extraction.
Creating your own Tools¶
Custom Tool Creation

Developers can craft custom tools tailored for their agent’s needs or utilize pre-built options:

To create your own crewAI tools you will need to install our extra tools package:


pip install 'crewai[tools]'
Once you do that there are two main ways for one to create a crewAI tool:

Subclassing BaseTool¶

from crewai_tools import BaseTool

class MyCustomTool(BaseTool):
    name: str = "Name of my tool"
    description: str = "Clear description for what this tool is useful for, your agent will need this information to use it."

    def _run(self, argument: str) -> str:
        # Implementation goes here
        return "Result from custom tool"
Utilizing the tool Decorator¶

from crewai_tools import tool
@tool("Name of my tool")
def my_tool(question: str) -> str:
    """Clear description for what this tool is useful for, your agent will need this information to use it."""
    # Function logic here
    return "Result from your custom tool"
Custom Caching Mechanism¶
Caching

Tools can optionally implement a cache_function to fine-tune caching behavior. This function determines when to cache results based on specific conditions, offering granular control over caching logic.


from crewai_tools import tool

@tool
def multiplication_tool(first_number: int, second_number: int) -> str:
    """Useful for when you need to multiply two numbers together."""
    return first_number * second_number

def cache_func(args, result):
    # In this case, we only cache the result if it's a multiple of 2
    cache = result % 2 == 0
    return cache

multiplication_tool.cache_function = cache_func

writer1 = Agent(
        role="Writer",
        goal="You write lessons of math for kids.",
        backstory="You're an expert in writing and you love to teach kids but you know nothing of math.",
        tools=[multiplication_tool],
        allow_delegation=False,
    )
    #...
Conclusion¶
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.
"""
Processes
Understanding Processes¶
Core Concept

In CrewAI, processes orchestrate the execution of tasks by agents, akin to project management in human teams. These processes ensure tasks are distributed and executed efficiently, in alignment with a predefined strategy.

Process Implementations¶
Sequential: Executes tasks sequentially, ensuring tasks are completed in an orderly progression.
Hierarchical: Organizes tasks in a managerial hierarchy, where tasks are delegated and executed based on a structured chain of command. A manager language model (manager_llm) or a custom manager agent (manager_agent) must be specified in the crew to enable the hierarchical process, facilitating the creation and management of tasks by the manager.
Consensual Process (Planned): Aiming for collaborative decision-making among agents on task execution, this process type introduces a democratic approach to task management within CrewAI. It is planned for future development and is not currently implemented in the codebase.
The Role of Processes in Teamwork¶
Processes enable individual agents to operate as a cohesive unit, streamlining their efforts to achieve common objectives with efficiency and coherence.

Assigning Processes to a Crew¶
To assign a process to a crew, specify the process type upon crew creation to set the execution strategy. For a hierarchical process, ensure to define manager_llm or manager_agent for the manager agent.


from crewai import Crew
from crewai.process import Process
from langchain_openai import ChatOpenAI

# Example: Creating a crew with a sequential process
crew = Crew(
    agents=my_agents,
    tasks=my_tasks,
    process=Process.sequential
)

# Example: Creating a crew with a hierarchical process
# Ensure to provide a manager_llm or manager_agent
crew = Crew(
    agents=my_agents,
    tasks=my_tasks,
    process=Process.hierarchical,
    manager_llm=ChatOpenAI(model="gpt-4")
    # or
    # manager_agent=my_manager_agent
)
Note: Ensure my_agents and my_tasks are defined prior to creating a Crew object, and for the hierarchical process, either manager_llm or manager_agent is also required.
Sequential Process¶
This method mirrors dynamic team workflows, progressing through tasks in a thoughtful and systematic manner. Task execution follows the predefined order in the task list, with the output of one task serving as context for the next.

To customize task context, utilize the context parameter in the Task class to specify outputs that should be used as context for subsequent tasks.

Hierarchical Process¶
Emulates a corporate hierarchy, CrewAI allows specifying a custom manager agent or automatically creates one, requiring the specification of a manager language model (manager_llm). This agent oversees task execution, including planning, delegation, and validation. Tasks are not pre-assigned; the manager allocates tasks to agents based on their capabilities, reviews outputs, and assesses task completion.

Process Class: Detailed Overview¶
The Process class is implemented as an enumeration (Enum), ensuring type safety and restricting process values to the defined types (sequential, hierarchical). The consensual process is planned for future inclusion, emphasizing our commitment to continuous development and innovation.

Conclusion¶
The structured collaboration facilitated by processes within CrewAI is crucial for enabling systematic teamwork among agents. This documentation has been updated to reflect the latest features, enhancements, and the planned integration of the Consensual Process, ensuring users have access to the most current and comprehensive information.
"""
Crews
What is a Crew?¶
A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for task execution, agent collaboration, and the overall workflow.

Crew Attributes¶
Attribute	Parameters	Description
Tasks	tasks	A list of tasks assigned to the crew.
Agents	agents	A list of agents that are part of the crew.
Process (optional)	process	The process flow (e.g., sequential, hierarchical) the crew follows. Default is sequential.
Verbose (optional)	verbose	The verbosity level for logging during execution. Defaults to False.
Manager LLM (optional)	manager_llm	The language model used by the manager agent in a hierarchical process. Required when using a hierarchical process.
Function Calling LLM (optional)	function_calling_llm	If passed, the crew will use this LLM to do function calling for tools for all agents in the crew. Each agent can have its own LLM, which overrides the crew's LLM for function calling.
Config (optional)	config	Optional configuration settings for the crew, in Json or Dict[str, Any] format.
Max RPM (optional)	max_rpm	Maximum requests per minute the crew adheres to during execution. Defaults to None.
Language (optional)	language	Language used for the crew, defaults to English.
Language File (optional)	language_file	Path to the language file to be used for the crew.
Memory (optional)	memory	Utilized for storing execution memories (short-term, long-term, entity memory). Defaults to False.
Cache (optional)	cache	Specifies whether to use a cache for storing the results of tools' execution. Defaults to True.
Embedder (optional)	embedder	Configuration for the embedder to be used by the crew. Mostly used by memory for now. Default is {"provider": "openai"}.
Full Output (optional)	full_output	Whether the crew should return the full output with all tasks outputs or just the final output. Defaults to False.
Step Callback (optional)	step_callback	A function that is called after each step of every agent. This can be used to log the agent's actions or to perform other operations; it won't override the agent-specific step_callback.
Task Callback (optional)	task_callback	A function that is called after the completion of each task. Useful for monitoring or additional operations post-task execution.
Share Crew (optional)	share_crew	Whether you want to share the complete crew information and execution with the crewAI team to make the library better, and allow us to train models.
Output Log File (optional)	output_log_file	Whether you want to have a file with the complete crew output and execution. You can set it using True and it will default to the folder you are currently in and it will be called logs.txt or passing a string with the full path and name of the file.
Manager Agent (optional)	manager_agent	manager sets a custom agent that will be used as a manager.
Manager Callbacks (optional)	manager_callbacks	manager_callbacks takes a list of callback handlers to be executed by the manager agent when a hierarchical process is used.
Prompt File (optional)	prompt_file	Path to the prompt JSON file to be used for the crew.
Planning (optional)	planning	Adds planning ability to the Crew. When activated before each Crew iteration, all Crew data is sent to an AgentPlanner that will plan the tasks and this plan will be added to each task description.
Planning LLM (optional)	planning_llm	The language model used by the AgentPlanner in a planning process.
Crew Max RPM

The max_rpm attribute sets the maximum number of requests per minute the crew can perform to avoid rate limits and will override individual agents' max_rpm settings if you set it.

Crew Output¶
Understanding Crew Outputs

The output of a crew in the crewAI framework is encapsulated within the CrewOutput class. This class provides a structured way to access results of the crew's execution, including various formats such as raw strings, JSON, and Pydantic models. The CrewOutput includes the results from the final task output, token usage, and individual task outputs.

Crew Output Attributes¶
Attribute	Parameters	Type	Description
Raw	raw	str	The raw output of the crew. This is the default format for the output.
Pydantic	pydantic	Optional[BaseModel]	A Pydantic model object representing the structured output of the crew.
JSON Dict	json_dict	Optional[Dict[str, Any]]	A dictionary representing the JSON output of the crew.
Tasks Output	tasks_output	List[TaskOutput]	A list of TaskOutput objects, each representing the output of a task in the crew.
Token Usage	token_usage	Dict[str, Any]	A summary of token usage, providing insights into the language model's performance during execution.
Crew Output Methods and Properties¶
Method/Property	Description
json	Returns the JSON string representation of the crew output if the output format is JSON.
to_dict	Converts the JSON and Pydantic outputs to a dictionary.
**str**	Returns the string representation of the crew output, prioritizing Pydantic, then JSON, then raw.
Accessing Crew Outputs¶
Once a crew has been executed, its output can be accessed through the output attribute of the Crew object. The CrewOutput class provides various ways to interact with and present this output.

Example¶

# Example crew execution
crew = Crew(
    agents=[research_agent, writer_agent],
    tasks=[research_task, write_article_task],
    verbose=True
)

crew_output = crew.kickoff()

# Accessing the crew output
print(f"Raw Output: {crew_output.raw}")
if crew_output.json_dict:
    print(f"JSON Output: {json.dumps(crew_output.json_dict, indent=2)}")
if crew_output.pydantic:
    print(f"Pydantic Output: {crew_output.pydantic}")
print(f"Tasks Output: {crew_output.tasks_output}")
print(f"Token Usage: {crew_output.token_usage}")
Memory Utilization¶
Crews can utilize memory (short-term, long-term, and entity memory) to enhance their execution and learning over time. This feature allows crews to store and recall execution memories, aiding in decision-making and task execution strategies.

Cache Utilization¶
Caches can be employed to store the results of tools' execution, making the process more efficient by reducing the need to re-execute identical tasks.

Crew Usage Metrics¶
After the crew execution, you can access the usage_metrics attribute to view the language model (LLM) usage metrics for all tasks executed by the crew. This provides insights into operational efficiency and areas for improvement.


# Access the crew's usage metrics
crew = Crew(agents=[agent1, agent2], tasks=[task1, task2])
crew.kickoff()
print(crew.usage_metrics)
Crew Execution Process¶
Sequential Process: Tasks are executed one after another, allowing for a linear flow of work.
Hierarchical Process: A manager agent coordinates the crew, delegating tasks and validating outcomes before proceeding. Note: A manager_llm or manager_agent is required for this process and it's essential for validating the process flow.
Kicking Off a Crew¶
Once your crew is assembled, initiate the workflow with the kickoff() method. This starts the execution process according to the defined process flow.


# Start the crew's task execution
result = my_crew.kickoff()
print(result)
Different Ways to Kick Off a Crew¶
Once your crew is assembled, initiate the workflow with the appropriate kickoff method. CrewAI provides several methods for better control over the kickoff process: kickoff(), kickoff_for_each(), kickoff_async(), and kickoff_for_each_async().

kickoff(): Starts the execution process according to the defined process flow.
kickoff_for_each(): Executes tasks for each agent individually.
kickoff_async(): Initiates the workflow asynchronously.
kickoff_for_each_async(): Executes tasks for each agent individually in an asynchronous manner.

# Start the crew's task execution
result = my_crew.kickoff()
print(result)

# Example of using kickoff_for_each
inputs_array = [{'topic': 'AI in healthcare'}, {'topic': 'AI in finance'}]
results = my_crew.kickoff_for_each(inputs=inputs_array)
for result in results:
    print(result)

# Example of using kickoff_async
inputs = {'topic': 'AI in healthcare'}
async_result = my_crew.kickoff_async(inputs=inputs)
print(async_result)

# Example of using kickoff_for_each_async
inputs_array = [{'topic': 'AI in healthcare'}, {'topic': 'AI in finance'}]
async_results = my_crew.kickoff_for_each_async(inputs=inputs_array)
for async_result in async_results:
    print(async_result)
These methods provide flexibility in how you manage and execute tasks within your crew, allowing for both synchronous and asynchronous workflows tailored to your needs.

Replaying from a Specific Task¶
You can now replay from a specific task using our CLI command replay.

The replay feature in CrewAI allows you to replay from a specific task using the command-line interface (CLI). By running the command crewai replay -t <task_id>, you can specify the task_id for the replay process.

Kickoffs will now save the latest kickoffs returned task outputs locally for you to be able to replay from.

Replaying from a Specific Task Using the CLI¶
To use the replay feature, follow these steps:

Open your terminal or command prompt.
Navigate to the directory where your CrewAI project is located.
Run the following command:
To view the latest kickoff task IDs, use:


crewai log-tasks-outputs
Then, to replay from a specific task, use:


crewai replay -t <task_id>
These commands let you replay from your latest kickoff tasks, still retaining context from previously executed tasks.
"""
schema.

Request
Output JSON
Request
Output JSON
Request

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
1
POST /v1/chat/completions
2
{
3
  "model": "gpt-4o-2024-08-06",
4
  "messages": [
5
    {
6
      "role": "system",
7
      "content": "You are a helpful math tutor."
8
    },
9
    {
10
      "role": "user",
11
      "content": "solve 8x + 31 = 2"
12
    }
13
  ],
14
  "response_format": {
15
    "type": "json_schema",
16
    "json_schema": {
17
      "name": "math_response",
18
      "strict": true,
19
      "schema": {
20
        "type": "object",
21
        "properties": {
22
          "steps": {
23
            "type": "array",
24
            "items": {
25
              "type": "object",
26
              "properties": {
27
                "explanation": {
28
                  "type": "string"
29
                },
30
                "output": {
31
                  "type": "string"
32
                }
33
              },
34
              "required": ["explanation", "output"],
35
              "additionalProperties": false
36
            }
37
          },
38
          "final_answer": {
39
            "type": "string"
40
          }
41
        },
42
        "required": ["steps", "final_answer"],
43
        "additionalProperties": false
44
      }
45
    }
46
  }
47
}
Output JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
1
{
2
  "steps": [
3
    {
4
      "explanation": "Subtract 31 from both sides to isolate the term with x.",
5
      "output": "8x + 31 - 31 = 2 - 31"
6
    },
7
    {
8
      "explanation": "This simplifies to 8x = -29.",
9
      "output": "8x = -29"
10
    },
11
    {
12
      "explanation": "Divide both sides by 8 to solve for x.",
13
      "output": "x = -29 / 8"
14
    }
15
  ],
16
  "final_answer": "x = -29 / 8"
17
}
Safe Structured Outputs
Safety is a top priority for OpenAI—the new Structured Outputs functionality will abide by our existing safety policies and will still allow the model to refuse an unsafe request. To make development simpler, there is a new refusal string value on API responses which allows developers to programmatically detect if the model has generated a refusal instead of output matching the schema. When the response does not include a refusal and the model’s response has not been prematurely interrupted (as indicated by finish_reason), then the model’s response will reliably produce valid JSON matching the supplied schema.

JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
1
{
2
  "id": "chatcmpl-9nYAG9LPNonX8DAyrkwYfemr3C8HC",
3
  "object": "chat.completion",
4
  "created": 1721596428,
5
  "model": "gpt-4o-2024-08-06",
6
  "choices": [
7
    {
8
      "index": 0,
9
      "message": {
10
        "role": "assistant",
11
        "refusal": "I'm sorry, I cannot assist with that request."
12
      },
13
      "logprobs": null,
14
      "finish_reason": "stop"
15
    }
16
  ],
17
  "usage": {
18
    "prompt_tokens": 81,
19
    "completion_tokens": 11,
20
    "total_tokens": 92
21
  },
22
  "system_fingerprint": "fp_3407719c7f"
23
}
Native SDK support
Our Python and Node SDKs have been updated with native support for Structured Outputs. Supplying a schema for tools or as a response format is as easy as supplying a Pydantic or Zod object, and our SDKs will handle converting the data type to a supported JSON schema, deserializing the JSON response into the typed data structure automatically, and parsing refusals if they arise.

The following examples show native support for Structured Outputs with function calling.

Python
Node
Python
Node
python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
1
from enum import Enum
2
from typing import Union
3

4
from pydantic import BaseModel
5

6
import openai
7
from openai import OpenAI
8

9

10
class Table(str, Enum):
11
    orders = "orders"
12
    customers = "customers"
13
    products = "products"
14

15

16
class Column(str, Enum):
17
    id = "id"
18
    status = "status"
19
    expected_delivery_date = "expected_delivery_date"
20
    delivered_at = "delivered_at"
21
    shipped_at = "shipped_at"
22
    ordered_at = "ordered_at"
23
    canceled_at = "canceled_at"
24

25

26
class Operator(str, Enum):
27
    eq = "="
28
    gt = ">"
29
    lt = "<"
30
    le = "<="
31
    ge = ">="
32
    ne = "!="
33

34

35
class OrderBy(str, Enum):
36
    asc = "asc"
37
    desc = "desc"
38

39

40
class DynamicValue(BaseModel):
41
    column_name: str
42

43

44
class Condition(BaseModel):
45
    column: str
46
    operator: Operator
47
    value: Union[str, int, DynamicValue]
48

49

50
class Query(BaseModel):
51
    table_name: Table
52
    columns: list[Column]
53
    conditions: list[Condition]
54
    order_by: OrderBy
55

56

57
client = OpenAI()
58

59
completion = client.beta.chat.completions.parse(
60
    model="gpt-4o-2024-08-06",
61
    messages=[
62
        {
63
            "role": "system",
64
            "content": "You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function.",
65
        },
66
        {
67
            "role": "user",
68
            "content": "look up all my orders in may of last year that were fulfilled but not delivered on time",
69
        },
70
    ],
71
    tools=[
72
        openai.pydantic_function_tool(Query),
73
    ],
74
)
75

76
print(completion.choices[0].message.tool_calls[0].function.parsed_arguments)
javascript

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
1
import OpenAI from 'openai';
2
import z from 'zod';
3
import { zodFunction } from 'openai/helpers/zod';
4

5
const Table = z.enum(['orders', 'customers', 'products']);
6
const Column = z.enum([
7
    'id',
8
    'status',
9
    'expected_delivery_date',
10
    'delivered_at',
11
    'shipped_at',
12
    'ordered_at',
13
    'canceled_at',
14
]);
15
const Operator = z.enum(['=', '>', '<', '<=', '>=', '!=']);
16
const OrderBy = z.enum(['asc', 'desc']);
17

18
const DynamicValue = z.object({
19
    column_name: z.string(),
20
});
21

22
const Condition = z.object({
23
    column: z.string(),
24
    operator: Operator,
25
    value: z.union([z.string(), z.number(), DynamicValue]),
26
});
27

28
const QueryArgs = z.object({
29
    table_name: Table,
30
    columns: z.array(Column),
31
    conditions: z.array(Condition),
32
    order_by: OrderBy,
33
});
34

35
const client = new OpenAI();
36

37
const completion = await client.beta.chat.completions.parse({
38
    model: 'gpt-4o-2024-08-06',
39
    messages: [
40
        { role: 'system', content: 'You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function.' },
41
        { role: 'user', content: 'look up all my orders in may of last year that were fulfilled but not delivered on time' }
42
    ],
43
    tools: [zodFunction({ name: 'query', parameters: QueryArgs })],
44
});
45
console.log(completion.choices[0].message.tool_calls[0].function.parsed_arguments);
Native Structured Outputs support is also available for response_format.

Python
Node
Python
Node
python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
1
from pydantic import BaseModel
2

3
from openai import OpenAI
4

5

6
class Step(BaseModel):
7
    explanation: str
8
    output: str
9

10

11
class MathResponse(BaseModel):
12
    steps: list[Step]
13
    final_answer: str
14

15

16
client = OpenAI()
17

18
completion = client.beta.chat.completions.parse(
19
    model="gpt-4o-2024-08-06",
20
    messages=[
21
        {"role": "system", "content": "You are a helpful math tutor."},
22
        {"role": "user", "content": "solve 8x + 31 = 2"},
23
    ],
24
    response_format=MathResponse,
25
)
26

27
message = completion.choices[0].message
28
if message.parsed:
29
    print(message.parsed.steps)
30
    print(message.parsed.final_answer)
31
else:
32
    print(message.refusal)
javascript

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
1
import OpenAI from 'openai';
2
import { zodResponseFormat } from 'openai/helpers/zod';
3
import { z } from 'zod';
4

5

6
const Step = z.object({
7
    explanation: z.string(),
8
    output: z.string(),
9
})
10

11
const MathResponse = z.object({
12
    steps: z.array(Step),
13
    final_answer: z.string(),
14
})
15

16

17
const client = new OpenAI();
18

19
const completion = await client.beta.chat.completions.parse({
20
    model: 'gpt-4o-2024-08-06',
21
    messages: [
22
        {
23
            "role": "system",
24
            "content": "You are a helpful math tutor. Only use the schema for math responses.",
25
        },
26
        { "role": "user", "content": "solve 8x + 3 = 21" },
27
    ],
28
    response_format: zodResponseFormat(MathResponse, 'mathResponse'),
29
});
30

31
const message = completion.choices[0]?.message;
32
if (message?.parsed) {
33
    console.log(message.parsed.steps);
34
    console.log(message.parsed.final_answer);
35
} else {
36
    console.log(message.refusal);
37
}
Additional use cases
Developers frequently use OpenAI’s models to generate structured data for various use cases. Some additional examples include:

Dynamically generating user interfaces based on the user’s intent
For example, developers can use Structured Outputs to create code- or UI-generating applications. All of the following examples use the same response_format, and can be used to generate varying UIs based on user input.

System
You are a user interface assistant. Your job is to help users visualize their website and app ideas.
Response format

View JSON schema
Assistant
Landing page for a gardener
Sign up screen for an app
Stock price widget
Landing page for a gardener
Sign up screen for an app
Stock price widget
{
  "type": "div",
  "label": "",
  "children": [
    {
      "type": "header",
      "label": "",
      "children": [
        {
          "type": "div",
          "label": "Green Thumb Gardening",
          "children": [],
          "attributes": [{ "name": "className", "value": "site-title" }]
        },
        {
          "type": "div",
          "label": "Bringing Life to Your Garden",
          "children": [],
          "attributes": [{ "name": "className", "value": "site-tagline" }]
        }
      ],
      "attributes": [{ "name": "className", "value": "header" }]
    },
    {
      "type": "section",
      "label": "",
      "children": [
        {
          "type": "div",
          "label": "",
          "children": [
            {
              "type": "div",
              "label": "About Us",
              "children": [
                {
                  "type": "div",
                  "label": "At Green Thumb Gardening, we specialize in transforming your outdoor spaces into beautiful, thriving gardens. Our team has decades of experience in horticulture and landscape design.",
                  "children": [],
                  "attributes": [
                    { "name": "className", "value": "about-description" }
                  ]
                }
              ],
              "attributes": [{ "name": "className", "value": "about-section" }]
            }
          ],
          "attributes": [{ "name": "className", "value": "content" }]
        }
      ],
      "attributes": [{ "name": "className", "value": "about-container" }]
    },
    {
      "type": "section",
      "label": "",
      "children": [
        {
          "type": "div",
          "label": "",
          "children": [
            {
              "type": "div",
              "label": "Our Services",
              "children": [
                {
                  "type": "div",
                  "label": "Garden Design",
                  "children": [],
                  "attributes": [
                    { "name": "className", "value": "service-item" }
                  ]
                },
                {
                  "type": "div",
                  "label": "Plant Care & Maintenance",
                  "children": [],
                  "attributes": [
                    { "name": "className", "value": "service-item" }
                  ]
                },
                {
                  "type": "div",
                  "label": "Seasonal Cleanup",
                  "children": [],
                  "attributes": [
                    { "name": "className", "value": "service-item" }
                  ]
                },
                {
                  "type": "div",
                  "label": "Custom Landscaping",
                  "children": [],
                  "attributes": [
                    { "name": "className", "value": "service-item" }
                  ]
                }
              ],
              "attributes": [{ "name": "className", "value": "services-list" }]
            }
          ],
          "attributes": [{ "name": "className", "value": "content" }]
        }
      ],
      "attributes": [{ "name": "className", "value": "services-container" }]
    }
  ],
  "attributes": [{ "name": "className", "value": "landing-page" }]
}

Welcome to Green Thumb Gardening
Bringing Life to Your Garden
At Green Thumb Gardening, we specialize in transforming your outdoor spaces into beautiful, thriving gardens. Our team has decades of experience in horticulture and landscape design.
Our services
Garden Design
Plant Care & Maintenance
Seasonal Cleanup
Custom Landscaping
{
  "type": "form",
  "label": "Sign Up Form",
  "children": [
    {
      "type": "header",
      "label": "Sign Up for MovieReviews",
      "children": [],
      "attributes": [{ "name": "className", "value": "signup-header" }]
    },
    {
      "type": "field",
      "label": "Username",
      "attributes": [
        { "name": "type", "value": "text" },
        { "name": "placeholder", "value": "Enter your username" },
        { "name": "name", "value": "username" },
        { "name": "required", "value": "true" },
        { "name": "className", "value": "input-text" }
      ],
      "children": []
    },
    {
      "type": "field",
      "label": "Email",
      "attributes": [
        { "name": "type", "value": "email" },
        { "name": "placeholder", "value": "Enter your email" },
        { "name": "name", "value": "email" },
        { "name": "required", "value": "true" },
        { "name": "className", "value": "input-email" }
      ],
      "children": []
    },
    {
      "type": "field",
      "label": "Password",
      "attributes": [
        { "name": "type", "value": "password" },
        { "name": "placeholder", "value": "Create a password" },
        { "name": "name", "value": "password" },
        { "name": "required", "value": "true" },
        { "name": "className", "value": "input-password" }
      ],
      "children": []
    },
    {
      "type": "field",
      "label": "Confirm Password",
      "attributes": [
        { "name": "type", "value": "password" },
        { "name": "placeholder", "value": "Confirm your password" },
        { "name": "name", "value": "confirm_password" },
        { "name": "required", "value": "true" },
        { "name": "className", "value": "input-password-confirm" }
      ],
      "children": []
    },
    {
      "type": "button",
      "label": "Sign Up",
      "attributes": [
        { "name": "type", "value": "submit" },
        { "name": "className", "value": "submit-button" }
      ],
      "children": []
    }
  ],
  "attributes": [
    { "name": "action", "value": "/signup" },
    { "name": "method", "value": "POST" },
    { "name": "className", "value": "signup-form" }
  ]
}
Sign up to MovieReviews
Enter your username
Enter your email
Create a password
Confirm your password
Submit
{
  "type": "div",
  "label": "",
  "children": [
    {
      "type": "header",
      "label": "Stock App",
      "children": [],
      "attributes": [
        {
          "name": "style",
          "value": "font-size: 24px; font-weight: bold; text-align: center; margin-bottom: 20px;"
        }
      ]
    },
    {
      "type": "div",
      "label": "",
      "children": [
        {
          "type": "div",
          "label": "Company Name: XYZ Corp",
          "children": [],
          "attributes": [
            {
              "name": "style",
              "value": "font-size: 20px; font-weight: bold; margin-bottom: 10px;"
            }
          ]
        },
        {
          "type": "div",
          "label": "Chart Area",
          "children": [],
          "attributes": [
            {
              "name": "style",
              "value": "width: 100%; height: 300px; border: 1px solid #ccc; margin-bottom: 20px;"
            }
          ]
        }
      ],
      "attributes": [
        {
          "name": "style",
          "value": "padding: 20px; border: 1px solid #ddd; margin-bottom: 20px;"
        }
      ]
    },
    {
      "type": "div",
      "label": "",
      "children": [
        {
          "type": "button",
          "label": "Refresh",
          "children": [],
          "attributes": [
            {
              "name": "style",
              "value": "margin-right: 10px; padding: 10px 20px; background-color: #4CAF50; color: white; border: none; font-size: 16px;"
            },
            { "name": "type", "value": "button" }
          ]
        },
        {
          "type": "button",
          "label": "Change Time Scale",
          "children": [],
          "attributes": [
            {
              "name": "style",
              "value": "margin-right: 10px; padding: 10px 20px; background-color: #2196F3; color: white; border: none; font-size: 16px;"
            },
            { "name": "type", "value": "button" }
          ]
        },
        {
          "type": "button",
          "label": "Export Data",
          "children": [],
          "attributes": [
            {
              "name": "style",
              "value": "padding: 10px 20px; background-color: #f44336; color: white; border: none; font-size: 16px;"
            },
            { "name": "type", "value": "button" }
          ]
        }
      ],
      "attributes": [{ "name": "style", "value": "text-align: center;" }]
    }
  ],
  "attributes": [
    {
      "name": "style",
      "value": "max-width: 600px; margin: 0 auto; font-family: Arial, sans-serif;"
    }
  ]
}
Stock app
Company Name: XYZ Corp
Jan
Feb
Mar
Apr
May
June
July
Refresh
Change time scale
Export data
Separating a final answer from supporting reasoning or additional commentary
It can be useful to give the model a separate field for chain of thought to improve the final quality of the response.

Request
Structured Output
Request
Structured Output
JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
1
{
2
  "model": "gpt-4o-2024-08-06",
3
  "messages": [
4
    {
5
      "role": "system",
6
      "content": "You are a helpful assistant"
7
    },
8
    {
9
      "role": "user",
10
      "content": "9.11 and 9.9 -- which is bigger?"
11
    }
12
  ],
13
  "response_format": {
14
    "type": "json_schema",
15
    "json_schema": {
16
      "name": "reasoning_schema",
17
      "strict": true,
18
      "schema": {
19
        "type": "object",
20
        "properties": {
21
          "reasoning_steps": {
22
            "type": "array",
23
            "items": {
24
              "type": "string"
25
            },
26
            "description": "The reasoning steps leading to the final conclusion."
27
          },
28
          "answer": {
29
            "type": "string",
30
            "description": "The final answer, taking into account the reasoning steps."
31
          }
32
        },
33
        "required": ["reasoning_steps", "answer"],
34
        "additionalProperties": false
35
      }
36
    }
37
  }
38
}
JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
1
{
2
  "reasoning_steps": [
3
    "First step is to compare the numbers 9.11 and 9.9.",
4
    "Both numbers have the same whole number part, which is 9.",
5
    "To compare the decimal parts, convert them to the same number of decimal places.",
6
    "9.11 has two decimal places: it is 9.11.",
7
    "9.9 has one decimal place: it can be rewritten as 9.90.",
8
    "Now, compare 9.11 and 9.90 by looking at the decimal parts.",
9
    "Compare 11 with 90.",
10
    "90 is greater than 11, so 9.90 is greater than 9.11."
11
  ],
12
  "answer": "9.9 is bigger than 9.11."
13
}
Extracting structured data from unstructured data
For example, instructing the model to extract things like to-dos, due dates, and assignments from meeting notes.

Request
Structured Output
Request
Structured Output
JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
1
POST /v1/chat/completions
2
{
3
  "model": "gpt-4o-2024-08-06",
4
  "messages": [
5
    {
6
      "role": "system",
7
      "content": "Extract action items, due dates, and owners from meeting notes."
8
    },
9
    {
10
      "role": "user",
11
      "content": "...meeting notes go here..."
12
    }
13
  ],
14
  "response_format": {
15
    "type": "json_schema",
16
    "json_schema": {
17
      "name": "action_items",
18
      "strict": true,
19
      "schema": {
20
        "type": "object",
21
        "properties": {
22
          "action_items": {
23
            "type": "array",
24
            "items": {
25
              "type": "object",
26
              "properties": {
27
                "description": {
28
                  "type": "string",
29
                  "description": "Description of the action item."
30
                },
31
                "due_date": {
32
                  "type": ["string", "null"],
33
                  "description": "Due date for the action item, can be null if not specified."
34
                },
35
                "owner": {
36
                  "type": ["string", "null"],
37
                  "description": "Owner responsible for the action item, can be null if not specified."
38
                }
39
              },
40
              "required": ["description", "due_date", "owner"],
41
              "additionalProperties": false
42
            },
43
            "description": "List of action items from the meeting."
44
          }
45
        },
46
        "required": ["action_items"],
47
        "additionalProperties": false
48
      }
49
    }
50
  }
51
}
JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
1
{
2
  "action_items": [
3
    {
4
      "description": "Collaborate on optimizing the path planning algorithm",
5
      "due_date": "2024-06-30",
6
      "owner": "Jason Li"
7
    },
8
    {
9
      "description": "Reach out to industry partners for additional datasets",
10
      "due_date": "2024-06-25",
11
      "owner": "Aisha Patel"
12
    },
13
    {
14
      "description": "Explore alternative LIDAR sensor configurations and report findings",
15
      "due_date": "2024-06-27",
16
      "owner": "Kevin Nguyen"
17
    },
18
    {
19
      "description": "Schedule extended stress tests for the integrated navigation system",
20
      "due_date": "2024-06-28",
21
      "owner": "Emily Chen"
22
    },
23
    {
24
      "description": "Retest the system after bug fixes and update the team",
25
      "due_date": "2024-07-01",
26
      "owner": "David Park"
27
    }
28
  ]
29
}
Under the hood
We took a two part approach to improving reliability for model outputs that match JSON Schema. First, we trained our newest model gpt-4o-2024-08-06 to understand complicated schemas and how best to produce outputs that match them. However, model behavior is inherently non-deterministic—despite this model’s performance improvements (93% on our benchmark), it still did not meet the reliability that developers need to build robust applications. So we also took a deterministic, engineering-based approach to constrain the model’s outputs to achieve 100% reliability.

Constrained decoding
Our approach is based on a technique known as constrained sampling or constrained decoding. By default, when models are sampled to produce outputs, they are entirely unconstrained and can select any token from the vocabulary as the next output. This flexibility is what allows models to make mistakes; for example, they are generally free to sample a curly brace token at any time, even when that would not produce valid JSON. In order to force valid outputs, we constrain our models to only tokens that would be valid according to the supplied schema, rather than all available tokens.

It can be challenging to implement this constraining in practice, since the tokens that are valid differ throughout a model’s output. Let’s say we have the following schema:

JSON

1
2
3
4
5
6
7
8
1
{
2
  "type": "object",
3
  "properties": {
4
    "value": { "type": "number" }
5
  },
6
  "required": ["value"],
7
  "additionalProperties": false
8
}
The tokens that are valid at the beginning of the output include things like {, {“, {\n, etc. However, once the model has already sampled {“val, then { is no longer a valid token. Thus we need to implement dynamic constrained decoding, and determine which tokens are valid after each token is generated, rather than upfront at the beginning of the response.

To do this, we convert the supplied JSON Schema into a context-free grammar (CFG). A grammar is a set of rules that defines a language, and a context-free grammar is a grammar that conforms to specific rules. You can think of JSON and JSON Schema as particular languages with rules to define what is valid within the language. Just as it’s not valid in English to have a sentence with no verb, it is not valid in JSON to have a trailing comma.

Thus, for each JSON Schema, we compute a grammar that represents that schema, and pre-process its components to make it easily accessible during model sampling. This is why the first request with a new schema incurs a latency penalty—we must preprocess the schema to generate this artifact that we can use efficiently during sampling.

While sampling, after every token, our inference engine will determine which tokens are valid to be produced next based on the previously generated tokens and the rules within the grammar that indicate which tokens are valid next. We then use this list of tokens to mask the next sampling step, which effectively lowers the probability of invalid tokens to 0. Because we have preprocessed the schema, we can use a cached data structure to do this efficiently, with minimal latency overhead.

Alternate approaches
Alternate approaches to this problem often use finite state machines (FSMs) or regexes (generally implemented with FSMs) for constrained decoding. These function similarly in that they dynamically update which tokens are valid after each token is produced, but they have some key differences from the CFG approach. Notably, CFGs can express a broader class of languages than FSMs. In practice, this doesn’t matter for very simple schemas like the value schema shown above. However, we find that the difference is meaningful for more complex schemas that involve nested or recursive data structures. As an example, FSMs cannot generally express recursive types, which means FSM based approaches may struggle to match parentheses in deeply nested JSON. The following is a sample recursive schema that is supported on the OpenAI API with Structured Outputs but would not be possible to express with a FSM.

JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
1
{
2
  "name": "ui",
3
  "description": "Dynamically generated UI",
4
  "strict": true,
5
  "schema": {
6
    "type": "object",
7
    "properties": {
8
      "type": {
9
        "type": "string",
10
        "description": "The type of the UI component",
11
        "enum": ["div", "button", "header", "section", "field", "form"]
12
      },
13
      "label": {
14
        "type": "string",
15
        "description": "The label of the UI component, used for buttons or form fields"
16
      },
17
      "children": {
18
        "type": "array",
19
        "description": "Nested UI components",
20
        "items": {
21
          "$ref": "#"
22
        }
23
      },
24
      "attributes": {
25
        "type": "array",
26
        "description": "Arbitrary attributes for the UI component, suitable for any element",
27
        "items": {
28
          "type": "object",
29
          "properties": {
30
            "name": {
31
              "type": "string",
32
              "description": "The name of the attribute, for example onClick or className"
33
            },
34
            "value": {
35
              "type": "string",
36
              "description": "The value of the attribute"
37
            }
38
          }
39
        }
40
      }
41
    },
42
    "required": ["type", "label", "children", "attributes"],
43
    "additionalProperties": false
44
  }
45
}
Note that each UI element can have arbitrary children which reference the root schema recursively. This flexibility is something that the CFG approach affords.

Limitations and restrictions
There are a few limitations to keep in mind when using Structured Outputs:

Structured Outputs allows only a subset of JSON Schema, detailed in our docs(opens in a new window). This helps us ensure the best possible performance.

The first API response with a new schema will incur additional latency, but subsequent responses will be fast with no latency penalty. This is because during the first request, we process the schema as indicated above and then cache these artifacts for fast reuse later on. Typical schemas take under 10 seconds to process on the first request, but more complex schemas may take up to a minute.

The model can fail to follow the schema if the model chooses to refuse an unsafe request. If it chooses to refuse, the return message will have the refusal boolean set to true to indicate this. 

The model can fail to follow the schema if the generation reaches max_tokens or another stop condition before finishing. 

Structured Outputs doesn’t prevent all kinds of model mistakes. For example, the model may still make mistakes within the values of the JSON object (e.g., getting a step wrong in a mathematical equation). If developers find mistakes, we recommend providing examples in the system instructions or splitting tasks into simpler subtasks.

Structured Outputs is not compatible with parallel function calls. When a parallel function call is generated, it may not match supplied schemas. Set parallel_tool_calls: false to disable parallel function calling.

JSON Schemas supplied with Structured Outputs aren’t Zero Data Retention(opens in a new window) (ZDR) eligible.

Availability
Structured Outputs is generally available today in the API. 

Structured Outputs with function calling is available on all models that support function calling in the API. This includes our newest models (gpt-4o, gpt-4o-mini), all models after and including gpt-4-0613 and gpt-3.5-turbo-0613, and any fine-tuned models that support function calling. This functionality is available on the Chat Completions API, Assistants API, and Batch API. Structured Outputs with function calling is also compatible with vision inputs.

Structured Outputs with response formats is available on gpt-4o-mini and gpt-4o-2024-08-06 and any fine tunes based on these models. This functionality is available on the Chat Completions API, Assistants API, and Batch API. Structured Outputs with response formats is also compatible with vision inputs. 

By switching to the new gpt-4o-2024-08-06, developers save 50% on inputs ($2.50/1M input tokens) and 33% on outputs ($10.00/1M output tokens) compared to gpt-4o-2024-05-13.

To start using Structured Outputs, check out our docs(opens in a new window). 
"""