Repository: run-llama/llama_index
Pull request: Add notebook with workaround for lengthy tool descriptions
# Description

OpenAI tools have a limit on the length of the description provided, which is currently 1024 characters. In `QueryPlanTool`, we ultimately concatenate a number of constituent `QueryEngineTools` into a single OpenAI tool. Obviously this character limit will be hit after adding enough engine tools, and probably pretty quickly. I've made some small changes and added a notebook to demonstrate an alternative solution: moving the engine tools to the prompt.

I made a small opinionated change to allow manual overriding of the `QueryPlanTool` metadata, which will enable the separation of these tool components. Feel free to suggest another approach if it aligns better (e.g., we could have added an optional parameter to `QueryToolPlan` that allows users to omit the tool descriptions in the metadata getter).

Fixes #13159

## New Package?

Did I fill in the `tool.llamahub` section in the `pyproject.toml` and provide a detailed README.md for my new integration or package?

- [ ] Yes
- [x] No

## Version Bump?

Did I bump the version in the `pyproject.toml` file of the package I am updating? (Except for the `llama-index-core` package)

- [ ] Yes
- [x] No

## Type of Change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

- [x] Added new unit/integration tests
- [ ] Added new notebook (that tests end-to-end)
- [x] I stared at the code and made sure it makes sense

## Suggested Checklist:

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added Google Colab support for the newly added notebooks.
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I ran `make format; make lint` to appease the lint gods


Problem statement:
[Bug]: QueryPlanTool --> function description too long
### Bug Description

In the current implementation of the QueryPlanTool, function descriptions are concatenated directly when more than one function is used in a query plan. This results in a single, lengthy function description in the generated query plan. This way it is impractical to use more than two tools. There should be a better way than just concatenating the functions and descriptions.

### Version

0.10.33

### Steps to Reproduce

Use more than two tools in the query planning tool.

### Relevant Logs/Tracbacks

```shell
BadRequestError: Error code: 400 - {'error': {'message': "' This is a query plan tool that takes in a list of tools and executes a query plan over these tools to answer a query. The query plan is a DAG of query nodes.\\n\\nGiven a list of tool names and the query plan schema, you can choose to generate a query plan to answer a question.\\n\\nThe tool names and descriptions are as follows:\\n\\n\\n\\n Tool Name:  ..... is too long - 'tools.0.function.description'", 'type': 'invalid_request_error', 'param': None, 'code': None}}
```

Hints from issue discussion:
<!-- Greeting -->
Hey @WoutDeRijck! 👋 I'm [Dosu](https://dosu.dev), a bot here to lend a hand with bugs, answer your queries, and guide you on contributing while you're waiting for a human maintainer to jump in. Just wanted you to know I'm diving into your issue about the QueryPlanTool's lengthy descriptions and will get back to you with a full answer soon. Stay tuned! 🤖🛠



Feel free to make a PR or suggest a change :) 
We made a workaround by putting the function names and descriptions in the prompt of the agent calling the query_plan_tool, but I am not sure if that's the way to go.
<!-- Answer -->
To address the issue with lengthy function descriptions in the `QueryPlanTool`, consider implementing a function to truncate or summarize descriptions before they are used. This can help manage the descriptions more effectively and avoid errors related to exceeding length limits. Here's a concise example of how you could implement a truncation function:

```python
def truncate_description(description: str, max_length: int = 100) -> str:
    """Truncate the description to a maximum length, appending '...' if truncated."""
    if len(description) > max_length:
        return description[:max_length].rstrip() + "..."
    return description

class QueryPlanTool(BaseTool):
    # Existing implementation...

    @property
    def metadata(self) -> ToolMetadata:
        """Metadata with truncated tool descriptions."""
        tools_description = "\n\n".join(
            [
                f"Tool Name: {tool.metadata.name}\n"
                + f"Tool Description: {truncate_description(tool.metadata.description)}"
                for tool in self._query_tools_dict.values()
            ]
        )
        description = f"""\
        {self._description_prefix}\n\n
        {tools_description}
        """
        return ToolMetadata(description, self._name, fn_schema=QueryPlan)
```

This approach ensures that all tool descriptions are truncated to a manageable length before being included in the tool's metadata, preventing the descriptions from becoming too lengthy and unwieldy. This solution should help mitigate the issue you're experiencing with the `QueryPlanTool`.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [llama-index-legacy/llama_index/legacy/tools/query_plan.py](https://github.com/run-llama/llama_index/blob/main/llama-index-legacy/llama_index/legacy/tools/query_plan.py)
- [llama-index-core/llama_index/core/tools/query_plan.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/tools/query_plan.py)
- [llama-index-core/CONTRIBUTING.md](https://github.com/run-llama/llama_index/blob/main/llama-index-core/CONTRIBUTING.md)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*

@dosu This does not solve the problem when more tools are added. The descriptions of the functions the query plan tool can use should not be described in the queryplantool description.
<!-- Answer -->
To effectively manage the issue with lengthy function descriptions in the `QueryPlanTool` without overloading the tool's description, especially as more tools are added, consider the following approach:

1. **Externalize Descriptions**: Keep lengthy descriptions outside the main tool code, such as in a configuration file or database. This separation keeps the code focused on functionality.

2. **Dynamic Description Generation**: Implement a method within the `QueryPlanTool` to dynamically generate descriptions based on the tools being used. This method can tailor the overall description to the specific context, ensuring relevance and conciseness.

3. **Leverage Tool Metadata**: Use the `metadata` property of each tool for essential information, allowing for a concise summary that can be detailed upon demand.

Here's a simplified example:

```python
class QueryPlanTool(BaseTool):
    ...
    def generate_dynamic_description(self) -> str:
        """Generates a dynamic description based on the tools in the query plan."""
        tool_descriptions = [self._query_tools_dict[tool_name].metadata.description for tool_name in self._query_tools_dict]
        return "This query plan involves the following tools:\n" + "\n".join(tool_descriptions)

    @property
    def metadata(self) -> ToolMetadata:
        """Metadata with dynamic description."""
        return ToolMetadata(f"{self._description_prefix}\n\n{self.generate_dynamic_description()}", self._name, fn_schema=QueryPlan)
```

This implementation dynamically creates a description based on the tools available, which can be further enhanced to fetch descriptions from an external source. This strategy should help manage function descriptions effectively, keeping the overall tool description concise and relevant.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [llama-index-core/llama_index/core/tools/query_plan.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/tools/query_plan.py)
- [llama-index-core/llama_index/core/agent/function_calling/step.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/function_calling/step.py)
- [llama-index-legacy/llama_index/legacy/tools/query_plan.py](https://github.com/run-llama/llama_index/blob/main/llama-index-legacy/llama_index/legacy/tools/query_plan.py)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*

Commit message: Add notebook with workaround for lengthy tool descriptions

Patch to review:
diff --git a/docs/docs/examples/agent/openai_agent_lengthy_tools.ipynb b/docs/docs/examples/agent/openai_agent_lengthy_tools.ipynb
new file mode 100644
index 0000000000..1bd6c45ef4
--- /dev/null
+++ b/docs/docs/examples/agent/openai_agent_lengthy_tools.ipynb
@@ -0,0 +1,589 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "fa593487",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/agent/openai_agent_query_plan.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "034e355d-83a0-4bd2-877e-0f493c5f713d",
+   "metadata": {},
+   "source": [
+    "# OpenAI Agent Workarounds for Lengthy Tool Descriptions\n",
+    "In this demo, we illustrate a workaround for defining an OpenAI tool\n",
+    "whose description exceeds OpenAI's current limit of 1024 characters.\n",
+    "For simplicity, we will build upon the `QueryPlanTool` notebook\n",
+    "example."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "dc72e6f9",
+   "metadata": {},
+   "source": [
+    "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "32a42109",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install llama-index-agent-openai\n",
+    "%pip install llama-index-llms-openai"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4df080d3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install llama-index"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c225e522-1ebd-436f-9c47-8738eb513880",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%load_ext autoreload\n",
+    "%autoreload 2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1c1e705a-36f2-4272-8f98-7f3785e76e8c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core import SimpleDirectoryReader, VectorStoreIndex\n",
+    "from llama_index.llms.openai import OpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "35364259-f1c3-4df0-b8c9-79e0afca7436",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0, model=\"gpt-4\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "26f545cd",
+   "metadata": {},
+   "source": [
+    "## Download Data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e6385d12",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--2024-05-23 13:36:24--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_march_2022.pdf\n",
+      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...\n",
+      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 1260185 (1.2M) [application/octet-stream]\n",
+      "Saving to: ‘data/10q/uber_10q_march_2022.pdf’\n",
+      "\n",
+      "data/10q/uber_10q_m 100%[===================>]   1.20M  --.-KB/s    in 0.04s   \n",
+      "\n",
+      "2024-05-23 13:36:24 (29.0 MB/s) - ‘data/10q/uber_10q_march_2022.pdf’ saved [1260185/1260185]\n",
+      "\n",
+      "--2024-05-23 13:36:24--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_june_2022.pdf\n",
+      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...\n",
+      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 1238483 (1.2M) [application/octet-stream]\n",
+      "Saving to: ‘data/10q/uber_10q_june_2022.pdf’\n",
+      "\n",
+      "data/10q/uber_10q_j 100%[===================>]   1.18M  --.-KB/s    in 0.04s   \n",
+      "\n",
+      "2024-05-23 13:36:24 (26.4 MB/s) - ‘data/10q/uber_10q_june_2022.pdf’ saved [1238483/1238483]\n",
+      "\n",
+      "--2024-05-23 13:36:24--  https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_sept_2022.pdf\n",
+      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...\n",
+      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 1178622 (1.1M) [application/octet-stream]\n",
+      "Saving to: ‘data/10q/uber_10q_sept_2022.pdf’\n",
+      "\n",
+      "data/10q/uber_10q_s 100%[===================>]   1.12M  --.-KB/s    in 0.05s   \n",
+      "\n",
+      "2024-05-23 13:36:25 (22.7 MB/s) - ‘data/10q/uber_10q_sept_2022.pdf’ saved [1178622/1178622]\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "!mkdir -p 'data/10q/'\n",
+    "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_march_2022.pdf' -O 'data/10q/uber_10q_march_2022.pdf'\n",
+    "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_june_2022.pdf' -O 'data/10q/uber_10q_june_2022.pdf'\n",
+    "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10q/uber_10q_sept_2022.pdf' -O 'data/10q/uber_10q_sept_2022.pdf'"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "e7113434-0e41-46b6-a74e-284ce211fd38",
+   "metadata": {},
+   "source": [
+    "## Load data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d036819a-754e-460b-8734-8af7071287e5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "march_2022 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./data/10q/uber_10q_march_2022.pdf\"]\n",
+    ").load_data()\n",
+    "june_2022 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./data/10q/uber_10q_june_2022.pdf\"]\n",
+    ").load_data()\n",
+    "sept_2022 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./data/10q/uber_10q_sept_2022.pdf\"]\n",
+    ").load_data()"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "fd541b68-c67f-4cbf-b579-5437d48e5b8f",
+   "metadata": {},
+   "source": [
+    "## Build indices\n",
+    "\n",
+    "We build a vector index / query engine over each of the documents (March, June, September)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4e6c3178-6aab-4fdc-99f6-c820661e7a73",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "march_index = VectorStoreIndex.from_documents(march_2022)\n",
+    "june_index = VectorStoreIndex.from_documents(june_2022)\n",
+    "sept_index = VectorStoreIndex.from_documents(sept_2022)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3af29f86-9a18-4b8e-af38-8ddc24b550e8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "march_engine = march_index.as_query_engine(similarity_top_k=3, llm=llm)\n",
+    "june_engine = june_index.as_query_engine(similarity_top_k=3, llm=llm)\n",
+    "sept_engine = sept_index.as_query_engine(similarity_top_k=3, llm=llm)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "2d6471ed-5645-4bb0-b8db-1b964ff7cd23",
+   "metadata": {},
+   "source": [
+    "## Defining an Excessively Lengthy Query Plan\n",
+    "\n",
+    "Although a `QueryPlanTool` may be composed from many `QueryEngineTools`,\n",
+    "a single OpenAI tool is ultimately created from the `QueryPlanTool`\n",
+    "when the OpenAI API call is made. The description of this tool begins with\n",
+    "general instructions about the query plan approach, followed by the\n",
+    "descriptions of each constituent `QueryEngineTool`.\n",
+    "\n",
+    "Currently, each OpenAI tool description has a maximum length of 1024 characters.\n",
+    "As you add more `QueryEngineTools` to your `QueryPlanTool`, you may exceed\n",
+    "this limit. If the limit is exceeded, LlamaIndex will raise an error when it\n",
+    "attempts to construct the OpenAI tool.\n",
+    "\n",
+    "Let's demonstrate this scenario with an exaggerated example, where we will\n",
+    "give each query engine tool a very lengthy and redundant description."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ce32fa42",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "description_10q_general = \"\"\"\\\n",
+    "A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\n",
+    "providing an overview of the company's financial performance for the quarter.\n",
+    "It includes unaudited financial statements (income statement, balance sheet,\n",
+    "and cash flow statement) and the Management's Discussion and Analysis (MD&A),\n",
+    "where management explains significant changes and future expectations.\n",
+    "The 10-Q also discloses significant legal proceedings, updates on risk factors,\n",
+    "and information on the company's internal controls. Its primary purpose is to keep\n",
+    "investors informed about the company's financial status and operations,\n",
+    "enabling informed investment decisions.\"\"\"\n",
+    "\n",
+    "description_10q_specific = (\n",
+    "    \"This 10-Q provides Uber quarterly financials ending\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "316db046",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core.tools import QueryEngineTool\n",
+    "from llama_index.core.tools import QueryPlanTool\n",
+    "from llama_index.core import get_response_synthesizer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a89972cc-f7b8-4ebd-9c39-935e8a3671ba",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "730\n",
+      "725\n",
+      "726\n"
+     ]
+    }
+   ],
+   "source": [
+    "query_tool_sept = QueryEngineTool.from_defaults(\n",
+    "    query_engine=sept_engine,\n",
+    "    name=\"sept_2022\",\n",
+    "    description=f\"{description_10q_general} {description_10q_specific} September 2022\",\n",
+    ")\n",
+    "query_tool_june = QueryEngineTool.from_defaults(\n",
+    "    query_engine=june_engine,\n",
+    "    name=\"june_2022\",\n",
+    "    description=f\"{description_10q_general} {description_10q_specific} June 2022\",\n",
+    ")\n",
+    "query_tool_march = QueryEngineTool.from_defaults(\n",
+    "    query_engine=march_engine,\n",
+    "    name=\"march_2022\",\n",
+    "    description=f\"{description_10q_general} {description_10q_specific} March 2022\",\n",
+    ")\n",
+    "\n",
+    "print(len(query_tool_sept.metadata.description))\n",
+    "print(len(query_tool_june.metadata.description))\n",
+    "print(len(query_tool_march.metadata.description))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1557d64c",
+   "metadata": {},
+   "source": [
+    "From the print statements above, we see that we will easily exceed the\n",
+    "maximum character limit of 1024 when composing these tools into the `QueryPlanTool`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8ae6d0bd-bd85-4f99-8363-96e203156933",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query_engine_tools = [query_tool_sept, query_tool_june, query_tool_march]\n",
+    "\n",
+    "response_synthesizer = get_response_synthesizer()\n",
+    "query_plan_tool = QueryPlanTool.from_defaults(\n",
+    "    query_engine_tools=query_engine_tools,\n",
+    "    response_synthesizer=response_synthesizer,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "320e8bde-6afc-4896-94bf-7a186f94fa49",
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ValueError",
+     "evalue": "Tool description exceeds maximum length of 1024 characters. Please shorten your description or move it to the prompt.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[12], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m openai_tool \u001b[38;5;241m=\u001b[39m \u001b[43mquery_plan_tool\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmetadata\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mto_openai_tool\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Code/run-llama/llama_index/llama-index-core/llama_index/core/tools/types.py:74\u001b[0m, in \u001b[0;36mToolMetadata.to_openai_tool\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     72\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"To OpenAI tool.\"\"\"\u001b[39;00m\n\u001b[1;32m     73\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdescription) \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m1024\u001b[39m:\n\u001b[0;32m---> 74\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m     75\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mTool description exceeds maximum length of 1024 characters. \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m     76\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPlease shorten your description or move it to the prompt.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m     77\u001b[0m     )\n\u001b[1;32m     78\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m {\n\u001b[1;32m     79\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtype\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfunction\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m     80\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfunction\u001b[39m\u001b[38;5;124m\"\u001b[39m: {\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     84\u001b[0m     },\n\u001b[1;32m     85\u001b[0m }\n",
+      "\u001b[0;31mValueError\u001b[0m: Tool description exceeds maximum length of 1024 characters. Please shorten your description or move it to the prompt."
+     ]
+    }
+   ],
+   "source": [
+    "openai_tool = query_plan_tool.metadata.to_openai_tool()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0dba5958",
+   "metadata": {},
+   "source": [
+    "## Moving Tool Descriptions to the Prompt\n",
+    "\n",
+    "One obvious solution to this problem would be to shorten the tool\n",
+    "descriptions themselves, however with sufficiently many tools,\n",
+    "we will still eventually exceed the character limit.\n",
+    "\n",
+    "A more scalable solution would be to move the tool descriptions to the prompt.\n",
+    "This solves the character limit issue, since without the descriptions\n",
+    "of the query engine tools, the query plan description will remain fixed\n",
+    "in size. Of course, token limits imposed by the selected LLM will still\n",
+    "bound the tool descriptions, however these limits are far larger than the\n",
+    "1024 character limit."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f96da5d3",
+   "metadata": {},
+   "source": [
+    "There are two steps involved in moving these tool descriptions to the\n",
+    "prompt. First, we must modify the metadata property of the `QueryPlanTool`\n",
+    "to omit the `QueryEngineTool` descriptions, and make a slight modification\n",
+    "to the default query planning instructions (telling the LLM to look for the\n",
+    "tool names and descriptions in the prompt.)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "90a3f429",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "ToolMetadata(description='This is a query plan tool that takes in a list of tools and executes a query plan over these tools to answer a query. The query plan is a DAG of query nodes.\\n\\nGiven a list of tool names and the query plan schema, you can choose to generate a query plan to answer a question.\\n\\nThe tool names and descriptions will be given alongside the query.\\n', name='query_plan_tool', fn_schema=<class 'llama_index.core.tools.query_plan.QueryPlan'>, return_direct=False)"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from llama_index.core.tools.types import ToolMetadata\n",
+    "\n",
+    "introductory_tool_description_prefix = \"\"\"\\\n",
+    "This is a query plan tool that takes in a list of tools and executes a \\\n",
+    "query plan over these tools to answer a query. The query plan is a DAG of query nodes.\n",
+    "\n",
+    "Given a list of tool names and the query plan schema, you \\\n",
+    "can choose to generate a query plan to answer a question.\n",
+    "\n",
+    "The tool names and descriptions will be given alongside the query.\n",
+    "\"\"\"\n",
+    "\n",
+    "# Modify metadata to only include the general query plan instructions\n",
+    "new_metadata = ToolMetadata(\n",
+    "    introductory_tool_description_prefix,\n",
+    "    query_plan_tool.metadata.name,\n",
+    "    query_plan_tool.metadata.fn_schema,\n",
+    ")\n",
+    "query_plan_tool.metadata = new_metadata\n",
+    "query_plan_tool.metadata"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7d8d906e",
+   "metadata": {},
+   "source": [
+    "Second, we must concatenate our tool names and descriptions alongside\n",
+    "the query being posed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6466d4bf-59bd-4916-bf28-c1eb71a0a650",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.agent.openai import OpenAIAgent\n",
+    "from llama_index.llms.openai import OpenAI\n",
+    "\n",
+    "agent = OpenAIAgent.from_tools(\n",
+    "    [query_plan_tool],\n",
+    "    max_function_calls=10,\n",
+    "    llm=OpenAI(temperature=0, model=\"gpt-4-0613\"),\n",
+    "    verbose=True,\n",
+    ")\n",
+    "\n",
+    "query = \"What were the risk factors in sept 2022?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "51cb85a5-8539-4218-b337-521ba77771a3",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"Tool Name: sept_2022\\nTool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\\nproviding an overview of the company's financial performance for the quarter.\\nIt includes unaudited financial statements (income statement, balance sheet,\\nand cash flow statement) and the Management's Discussion and Analysis (MD&A),\\nwhere management explains significant changes and future expectations.\\nThe 10-Q also discloses significant legal proceedings, updates on risk factors,\\nand information on the company's internal controls. Its primary purpose is to keep\\ninvestors informed about the company's financial status and operations,\\nenabling informed investment decisions. This 10-Q provides Uber quarterly financials ending September 2022 \\n\\nTool Name: june_2022\\nTool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\\nproviding an overview of the company's financial performance for the quarter.\\nIt includes unaudited financial statements (income statement, balance sheet,\\nand cash flow statement) and the Management's Discussion and Analysis (MD&A),\\nwhere management explains significant changes and future expectations.\\nThe 10-Q also discloses significant legal proceedings, updates on risk factors,\\nand information on the company's internal controls. Its primary purpose is to keep\\ninvestors informed about the company's financial status and operations,\\nenabling informed investment decisions. This 10-Q provides Uber quarterly financials ending June 2022 \\n\\nTool Name: march_2022\\nTool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\\nproviding an overview of the company's financial performance for the quarter.\\nIt includes unaudited financial statements (income statement, balance sheet,\\nand cash flow statement) and the Management's Discussion and Analysis (MD&A),\\nwhere management explains significant changes and future expectations.\\nThe 10-Q also discloses significant legal proceedings, updates on risk factors,\\nand information on the company's internal controls. Its primary purpose is to keep\\ninvestors informed about the company's financial status and operations,\\nenabling informed investment decisions. This 10-Q provides Uber quarterly financials ending March 2022 \\n\\nQuery: What were the risk factors in sept 2022?\""
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Reconstruct concatenated query engine tool descriptions\n",
+    "tools_description = \"\\n\\n\".join(\n",
+    "    [\n",
+    "        f\"Tool Name: {tool.metadata.name}\\n\"\n",
+    "        + f\"Tool Description: {tool.metadata.description} \"\n",
+    "        for tool in query_engine_tools\n",
+    "    ]\n",
+    ")\n",
+    "\n",
+    "# Concatenate tool descriptions and query\n",
+    "query_planned_query = f\"{tools_description}\\n\\nQuery: {query}\"\n",
+    "query_planned_query"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e916d809",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Added user message to memory: Tool Name: sept_2022\n",
+      "Tool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\n",
+      "providing an overview of the company's financial performance for the quarter.\n",
+      "It includes unaudited financial statements (income statement, balance sheet,\n",
+      "and cash flow statement) and the Management's Discussion and Analysis (MD&A),\n",
+      "where management explains significant changes and future expectations.\n",
+      "The 10-Q also discloses significant legal proceedings, updates on risk factors,\n",
+      "and information on the company's internal controls. Its primary purpose is to keep\n",
+      "investors informed about the company's financial status and operations,\n",
+      "enabling informed investment decisions. This 10-Q provides Uber quarterly financials ending September 2022 \n",
+      "\n",
+      "Tool Name: june_2022\n",
+      "Tool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\n",
+      "providing an overview of the company's financial performance for the quarter.\n",
+      "It includes unaudited financial statements (income statement, balance sheet,\n",
+      "and cash flow statement) and the Management's Discussion and Analysis (MD&A),\n",
+      "where management explains significant changes and future expectations.\n",
+      "The 10-Q also discloses significant legal proceedings, updates on risk factors,\n",
+      "and information on the company's internal controls. Its primary purpose is to keep\n",
+      "investors informed about the company's financial status and operations,\n",
+      "enabling informed investment decisions. This 10-Q provides Uber quarterly financials ending June 2022 \n",
+      "\n",
+      "Tool Name: march_2022\n",
+      "Tool Description: A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\n",
+      "providing an overview of the company's financial performance for the quarter.\n",
+      "It includes unaudited financial statements (income statement, balance sheet,\n",
+      "and cash flow statement) and the Management's Discussion and Analysis (MD&A),\n",
+      "where management explains significant changes and future expectations.\n",
+      "The 10-Q also discloses significant legal proceedings, updates on risk factors,\n",
+      "and information on the company's internal controls. Its primary purpose is to keep\n",
+      "investors informed about the company's financial status and operations,\n",
+      "enabling informed investment decisions. This 10-Q provides Uber quarterly financials ending March 2022 \n",
+      "\n",
+      "Query: What were the risk factors in sept 2022?\n",
+      "=== Calling Function ===\n",
+      "Calling function: query_plan_tool with args: {\n",
+      "\"nodes\": [\n",
+      "  {\n",
+      "    \"id\": 1,\n",
+      "    \"query_str\": \"What were the risk factors in sept 2022?\",\n",
+      "    \"tool_name\": \"sept_2022\",\n",
+      "    \"dependencies\": []\n",
+      "  }\n",
+      "]\n",
+      "}\n",
+      "\u001b[1;3;34mExecuting node {\"id\": 1, \"query_str\": \"What were the risk factors in sept 2022?\", \"tool_name\": \"sept_2022\", \"dependencies\": []}\n",
+      "\u001b[0m\u001b[1;3;38;5;200mSelected Tool: ToolMetadata(description=\"A Form 10-Q is a quarterly report required by the SEC for publicly traded companies,\\nproviding an overview of the company's financial performance for the quarter.\\nIt includes unaudited financial statements (income statement, balance sheet,\\nand cash flow statement) and the Management's Discussion and Analysis (MD&A),\\nwhere management explains significant changes and future expectations.\\nThe 10-Q also discloses significant legal proceedings, updates on risk factors,\\nand information on the company's internal controls. Its primary purpose is to keep\\ninvestors informed about the company's financial status and operations,\\nenabling informed investment decisions. This 10-Q provides Uber quarterly financials ending September 2022\", name='sept_2022', fn_schema=<class 'llama_index.core.tools.types.DefaultToolFnSchema'>, return_direct=False)\n",
+      "\u001b[0m\u001b[1;3;34mExecuted query, got response.\n",
+      "Query: What were the risk factors in sept 2022?\n",
+      "Response: The risk factors in September 2022 included failure to meet regulatory requirements related to climate change or to meet stated climate change commitments, which could impact costs, operations, brand, and reputation. The ongoing COVID-19 pandemic and responses to it were also a risk, as they had an adverse impact on business and operations, including reducing the demand for Mobility offerings globally and affecting travel behavior and demand. Catastrophic events such as disease, weather events, war, or terrorist attacks could also adversely impact the business, financial condition, and results of operation. Other risks included errors, bugs, or vulnerabilities in the platform's code or systems, inappropriate or controversial data practices, and the growing use of artificial intelligence. Climate change related physical and transition risks, such as market shifts toward electric vehicles and lower carbon business models, and risks related to extreme weather events or natural disasters, were also a concern.\n",
+      "\u001b[0mGot output: The risk factors in September 2022 included failure to meet regulatory requirements related to climate change or to meet stated climate change commitments, which could impact costs, operations, brand, and reputation. The ongoing COVID-19 pandemic and responses to it were also a risk, as they had an adverse impact on business and operations, including reducing the demand for Mobility offerings globally and affecting travel behavior and demand. Catastrophic events such as disease, weather events, war, or terrorist attacks could also adversely impact the business, financial condition, and results of operation. Other risks included errors, bugs, or vulnerabilities in the platform's code or systems, inappropriate or controversial data practices, and the growing use of artificial intelligence. Climate change related physical and transition risks, such as market shifts toward electric vehicles and lower carbon business models, and risks related to extreme weather events or natural disasters, were also a concern.\n",
+      "========================\n",
+      "\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "Response(response=\"The risk factors for Uber in September 2022 included:\\n\\n1. Failure to meet regulatory requirements related to climate change or to meet stated climate change commitments, which could impact costs, operations, brand, and reputation.\\n2. The ongoing COVID-19 pandemic and responses to it were also a risk, as they had an adverse impact on business and operations, including reducing the demand for Mobility offerings globally and affecting travel behavior and demand.\\n3. Catastrophic events such as disease, weather events, war, or terrorist attacks could also adversely impact the business, financial condition, and results of operation.\\n4. Other risks included errors, bugs, or vulnerabilities in the platform's code or systems, inappropriate or controversial data practices, and the growing use of artificial intelligence.\\n5. Climate change related physical and transition risks, such as market shifts toward electric vehicles and lower carbon business models, and risks related to extreme weather events or natural disasters, were also a concern.\", source_nodes=[NodeWithScore(node=TextNode(id_='a92c1e5e-6285-4225-8c87-b9dbd2b07d89', embedding=None, metadata={'page_label': '74', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='b5e99044-59e9-439a-9e53-802a517b287d', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '74', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, hash='edddd9bda362411ae2e4d36144b5049c8b2ce5ec26047fa7c04003a9265aa87d'), <NodeRelationship.PREVIOUS: '2'>: RelatedNodeInfo(node_id='04ae9351-0136-491a-8756-41dc2b8071a1', node_type=<ObjectType.TEXT: '1'>, metadata={'page_label': '74', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, hash='bdc5ab49a54e18f73a2687096148f9e567a053ea2d3bd3c051956fb359078f5e')}, text='Any failure to\\nmeet regulatory requirements related to climate change, or to meet our stated climate change commitments on the timeframe we committed to, or at all, could have\\nan adverse impact on our costs and ability to operate, as well as harm our brand, reputation, and consequently, our business.\\nGeneral Economic Risks\\nOutbreaks of contagious disease, such as the COVID-19 pandemic and the impact of actions to mitigate the such disease or pandemic, have adversely impacted\\nand could continue to adversely impact our business, financial condition and results of operations.\\nOccurrence of a catastrophic event, including but not limited to disease, a weather event, war, or terrorist attack, could adversely impact our business, financial\\ncondition and results of operation. We also face risks related to health epidemics, outbreaks of contagious disease, and other adverse health developments. For\\nexample, the ongoing COVID-19 pandemic and responses to it have had, and may continue to have, an adverse impact on our business and operations, including,\\nfor example, by reducing the demand for our Mobility offerings globally, and affecting travel behavior and demand. Even as COVID-related restrictions have been\\nlifted and many regions around the world are making progress in their recovery from the pandemic, we have experienced and may continue to experience Driver\\nsupply constraints, and we are observing that consumer demand for Mobility is recovering faster than driver availability, as such supply constraints have been and\\nmay continue to be impacted by concerns regarding the COVID-19 pandemic. Furthermore, to support social distancing, we temporarily suspended our shared\\nrides offering globally, and recently re-launched our shared rides offering in certain regions.\\n73', start_char_idx=4469, end_char_idx=6258, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.8039095664957979), NodeWithScore(node=TextNode(id_='04ae9351-0136-491a-8756-41dc2b8071a1', embedding=None, metadata={'page_label': '74', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='b5e99044-59e9-439a-9e53-802a517b287d', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '74', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, hash='edddd9bda362411ae2e4d36144b5049c8b2ce5ec26047fa7c04003a9265aa87d'), <NodeRelationship.NEXT: '3'>: RelatedNodeInfo(node_id='a92c1e5e-6285-4225-8c87-b9dbd2b07d89', node_type=<ObjectType.TEXT: '1'>, metadata={}, hash='4862564636269739d75565c6c9dc166659cf52e29ec337fe9b85802beeb2ce7d')}, text='platform to platform users. In addition, our release of new software in the past has inadvertently caused, and may in the future cause, interruptions in the\\navailability or functionality of our platform. Any errors, bugs, or vulnerabilities discovered in our code or systems after release could result in an interruption in the\\navailability of our platform or a negative experience for Drivers, consumers, merchants, Shippers, and Carriers, and could also result in negative publicity and\\nunfavorable media coverage, damage to our reputation, loss of platform users, loss of revenue or liability for damages, regulatory inquiries, or other proceedings,\\nany of which could adversely affect our business and financial results. In addition, our growing use of artificial intelligence (“AI”) (including machine learning) in\\nour offerings presents additional risks. AI algorithms or automated processing of data may be flawed and datasets may be insufficient or contain biased information.\\nInappropriate or controversial data practices by us or others could impair the acceptance of AI solutions or subject us to lawsuits and regulatory investigations.\\nThese deficiencies could undermine the decisions, predictions or analysis AI applications produce, or lead to unintentional bias and discrimination, subjecting us to\\ncompetitive harm, legal liability, and brand or reputational harm.\\nWe are subject to climate change risks, including physical and transitional risks, and if we are unable to manage such risks, our business may be adversely\\nimpacted.\\nWe face climate change related physical and transition risks, which include the risk of market shifts toward electric vehicles (“EVs”) and lower carbon\\nbusiness models and risks related to extreme weather events or natural disasters. Climate-related events, including the increasing frequency, severity and duration\\nof extreme weather events and their impact on critical infrastructure in the United States and elsewhere, have the potential to disrupt our business, our third-party\\nsuppliers, and the business of merchants, Shippers, Carriers and Drivers using our platform, and may cause us to experience higher losses and additional costs to\\nmaintain or resume operations. Additionally, we are subject to emerging climate policies such as a regulation adopted in California in May 2021 requiring 90% of\\nvehicle miles traveled by rideshare fleets in California to have been in zero emission vehicles by 2030, with interim targets beginning in 2023. In addition, Drivers\\nmay be subject to climate-related policies that indirectly impact our business, such as the Congestion Charge Zone and Ultra Low Emission Zone schemes adopted\\nin London that impose fees on drivers in fossil-fueled vehicles, which may impact our ability to attract and maintain Drivers on our platform, and to the extent we\\nexperience Driver supply constraints in a given market, we may need to increase Driver incentives.\\nWe have made climate related commitments that require us to invest significant effort, resources, and management time and circumstances may arise,\\nincluding those beyond our control, that may require us to revise the contemplated timeframes for implementing these commitments.\\nWe have made climate related commitments, including our commitment to 100% renewable electricity for our U.S. offices by 2025, our commitment to net\\nzero climate emissions from corporate operations by 2030, and our commitment to be a net zero company by 2040. In addition, our Supplier Code of Conduct sets\\nenvironmental standards for our supply chain, and we recognize that there are inherent climate-related risks wherever business is conducted. Progressing towards\\nour climate commitments requires us to invest significant effort, resources, and management time, and circumstances may arise, including those beyond our\\ncontrol, that may require us to revise our timelines and/or climate commitments. For example, the COVID-19 pandemic has negatively impacted our ability to\\ndedicate resources to make the progress on our climate commitments that we initially anticipated. In addition, our ability to meet our climate commitments is\\ndependent on external factors such as rapidly changing regulations, policies and related interpretation, advances in technology such as battery storage, as well the\\navailability, cost and accessibility of EVs to Drivers, and the availability of EV charging infrastructure that can be efficiently accessed by Drivers. Any failure to\\nmeet regulatory requirements related to climate change, or to meet our stated climate change commitments on the timeframe we committed to, or at all, could have\\nan adverse impact on our costs and ability to operate, as well as harm our brand, reputation, and consequently, our business.\\nGeneral Economic Risks\\nOutbreaks of contagious disease, such as the COVID-19 pandemic and the impact of actions to mitigate the such disease or pandemic, have adversely impacted\\nand could continue to adversely impact our business, financial condition and results of operations.\\nOccurrence of a catastrophic event, including but not limited to disease, a weather event, war, or terrorist attack, could adversely impact our business, financial\\ncondition and results of operation.', start_char_idx=0, end_char_idx=5248, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.7967699969539317), NodeWithScore(node=TextNode(id_='474572f4-866f-4efa-aa5f-f898d4ba831a', embedding=None, metadata={'page_label': '13', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='2ecd357c-acd3-4398-a0ae-223bf9980f34', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '13', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, hash='8da53f7e83f88f63304dfcf8186b0ce111a5e161a317d243451266b863e9f2af'), <NodeRelationship.PREVIOUS: '2'>: RelatedNodeInfo(node_id='e2e88da4-92e0-4185-96c6-010a35d94287', node_type=<ObjectType.TEXT: '1'>, metadata={'page_label': '13', 'file_name': 'uber_10q_sept_2022.pdf', 'file_path': 'data/10q/uber_10q_sept_2022.pdf', 'file_type': 'application/pdf', 'file_size': 1178622, 'creation_date': '2024-05-23', 'last_modified_date': '2024-05-23'}, hash='373d1a92181565c8fb2ae2831069e904f4f4d2a9fdc70486b91f102cdac9d442')}, text='Estimates are based on historical experience, where\\napplicable, and other assumptions which management believes are reasonable under the circumstances. Additionally, we considered the impacts of the coronavirus\\npandemic (“COVID-19”) on the assumptions and inputs (including market data) supporting certain of these estimates, assumptions and judgments. On an ongoing\\nbasis, management evaluates estimates, including, but not limited to: fair values of investments and other financial instruments (including the measurement of\\ncredit or impairment losses); useful lives of amortizable long-lived assets; fair value of acquired intangible assets and related impairment assessments; impairment\\nof goodwill; stock-based compensation; income taxes and non-income tax reserves; certain deferred tax assets and tax liabilities; insurance reserves; and other\\ncontingent liabilities. These estimates are inherently subject to judgment and actual results could differ from those estimates.\\nCertain Significant Risks and Uncertainties - COVID-19\\nCOVID-19 restrictions have had an adverse impact on our business and operations by reducing, in particular, the global demand for Mobility offerings. It is\\nnot possible to predict COVID-19’s cumulative and ultimate impact on our future business operations, results of operations, financial position, liquidity, and cash\\nflows. The extent of the impact of COVID-19 on our business and financial results will depend largely on future developments, including: outbreaks or variants of\\nthe virus, both globally and within the United\\n12', start_char_idx=4007, end_char_idx=5573, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.7911034852964277)], metadata=None)"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "response = agent.query(query_planned_query)\n",
+    "response"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/llama-index-core/llama_index/core/tools/query_plan.py b/llama-index-core/llama_index/core/tools/query_plan.py
index c78aa73966..c8d75706df 100644
--- a/llama-index-core/llama_index/core/tools/query_plan.py
+++ b/llama-index-core/llama_index/core/tools/query_plan.py
@@ -98,6 +98,7 @@ def __init__(
         self._response_synthesizer = response_synthesizer
         self._name = name
         self._description_prefix = description_prefix
+        self._custom_metadata = None
 
     @classmethod
     def from_defaults(
@@ -122,6 +123,9 @@ def from_defaults(
     @property
     def metadata(self) -> ToolMetadata:
         """Metadata."""
+        if self._custom_metadata is not None:
+            return self._custom_metadata
+
         tools_description = "\n\n".join(
             [
                 f"Tool Name: {tool.metadata.name}\n"
@@ -136,6 +140,10 @@ def metadata(self) -> ToolMetadata:
         """
         return ToolMetadata(description, self._name, fn_schema=QueryPlan)
 
+    @metadata.setter
+    def metadata(self, value: ToolMetadata):
+        self._custom_metadata = value
+
     def _execute_node(
         self, node: QueryNode, nodes_dict: Dict[int, QueryNode]
     ) -> ToolOutput:
diff --git a/llama-index-core/llama_index/core/tools/types.py b/llama-index-core/llama_index/core/tools/types.py
index 66207ae44f..18e05a23b6 100644
--- a/llama-index-core/llama_index/core/tools/types.py
+++ b/llama-index-core/llama_index/core/tools/types.py
@@ -70,6 +70,11 @@ def to_openai_function(self) -> Dict[str, Any]:
 
     def to_openai_tool(self) -> Dict[str, Any]:
         """To OpenAI tool."""
+        if len(self.description) > 1024:
+            raise ValueError(
+                "Tool description exceeds maximum length of 1024 characters. "
+                "Please shorten your description or move it to the prompt."
+            )
         return {
             "type": "function",
             "function": {
diff --git a/llama-index-core/tests/tools/test_types.py b/llama-index-core/tests/tools/test_types.py
new file mode 100644
index 0000000000..f74c9d027b
--- /dev/null
+++ b/llama-index-core/tests/tools/test_types.py
@@ -0,0 +1,12 @@
+import pytest
+from llama_index.core.tools.types import ToolMetadata
+
+
+def test_toolmetadata_openai_tool_description_max_length() -> None:
+    openai_tool_description_limit = 1024
+    valid_description = "a" * openai_tool_description_limit
+    invalid_description = "a" * (1 + openai_tool_description_limit)
+
+    ToolMetadata(valid_description).to_openai_tool()
+    with pytest.raises(ValueError):
+        ToolMetadata(invalid_description).to_openai_tool()


Code review comments:
diff_hunk: @@ -70,6 +70,11 @@ def to_openai_function(self) -> Dict[str, Any]:
 
     def to_openai_tool(self) -> Dict[str, Any]:
         """To OpenAI tool."""
+        if len(self.description) > 1024:
path: llama-index-core/llama_index/core/tools/types.py
text: @user1:
my one comment is that (sadly) this function is used in a few things outside the scope of openai (namely, inside the huggingface llm integration, and the mistral llm integration.

Should we have an optional flag to disable this check for these cases?

@author:
Ohh interesting, I was wondering why it appears in core. I can definitely add a flag, should we default to skipping the check?

@user1:
Since its specific to openai (or supposed to be lol), we should probably default to checking, and use the flag to disable

@author:
Okay cool, should I update any of these other calls to set this flag? Will any break if we don't?

@user1:
yea lets update them to set the flag. They won't break per-say without it, but I'm not 100% sure if they have this limit

@author:
Added the flag, let me know if I missed anything!