## 4 Experiments
### Tool Learning
#### 4.1.3 Ablation Study of Tool Learning  
In order to assess the impact of specific strategies or team members in the CoReAAgents framework, we perform ablation studies on tool learning datasets to develop three variants of our framework. Note that task planning by the Plan Agent is an indispensable component for solving complex reasoning tasks, making it impossible to remove the Plan Agent from our framework. The settings for the ablation study are as follows:  
* The CoReAAgents framework + w/o TA: the Tool Agent is removed from the CoReAAgents framework, meaning that no tool augmentation and subtask execution are conducted when handling complex tasks.
* The CoReAAgents framework + w/o RA: similarly, the task division of the Reflect Agent, including subtask reflection and plan reflection, is removed from the process.
* The CoReAAgents framework + w/o TA&RA: the Reflect Agent and the Tool Agent are removed from the processing of complex tasks in the CoReAAgents framework, only utilizing the Plan Agent to complete tasks.  
For tool learning, it is often necessary to use external tools to obtain the information required to complete the task, and LLMs are expected to make timely adjustments to task planning based on the acquired information. Table 2 shows the results of the ablation study. The results show that removing the Reflect Agent has the greatest impact on the CoReAAgents framework, followed by the removal of the Tool Agent. The main reason for this is that timely adjustments to task planning are more critical in quite complex scenarios. The improvement in effectiveness provided by the Reflect Agent confirms the importance of reflecting on subtasks and overall planning during the process. The impact of removing the Tool Agent confirms that our strategy aids in selecting more appropriate tools and using them effectively.  
Whereas removing the Reflect Agent and Tool Agent degrades the performance of the CoReAAgents framework, this strongly supports the idea that collaboration among multiple agents leads to better results than relying solely on a single agent to complete tool learning tasks. The ablation results on the API-Bank dataset suggest that when the number and variety of tools are minor, removing either the Reflect Agent or the Tool Agent has a similar impact on the CoReAAgents framework's performance.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**ToolBench (Pass Rate)**} & \multicolumn{4}{c}{**API-Bank (EM)**} \\
**Methods** & **I2-Ins** & **I2-Cat** & **I3-Ins** & **Average** & **Lv1 + Lv2** & **Tokens** \\ \hline ReAct & 42.5 & 46.5 & 22.0 & 37.0 & 70.0 5100 & \\ Reflexion & 58.0 & 66.0 & 55.0 & 59.7 & 77.0 & 8750 \\ DFSDT & 75.0 & 71.5 & 62.0 & 69.5 & 72.0 & 25,000 \\ DFSDT-EASYTOOL & 77.5 & 74.5 & 65.0 & 72.3 & 76.4 & - \\ ToolNet & - & - & - & - & 75.0 & 4112 \\ ToolNet + Reflexion & - & - & - & - & 83.0 & 7950 \\ Tool-Planner & 75.5 & 78.0 & 66.0 & 73.2 & - & - \\ CoReAAgents & **80.5** & **79.0** & **72.0** & **77.2** & **84.0** & 15,000 \\ \hline w/o TA & 75.0 & 74.0 & 68.0 & 72.3 & 79.8 & \\ w/o RA & 73.0 & 72.5 & 65.0 & 70.2 & 79.5 & \\ w/o TA\&RA & 70.5 & 70.0 & 61.0 & 67.2 & 74.5 & \\ \hline \hline \end{tabular}
\end{table}
Table 2: The results of different baselines and the ablation study in terms of Pass Rate (%) and EM (%). The table highlights the highest scores in bold, with underscores indicating the highest scores achieved by prior methods. - indicates that the method is not publicly available. The rightmost column in the table shows the number of tokens completing this task for each baseline.