Paper: ToolTalk: Evaluating Tool Usage in a Conversational Setting (ICLR-2024)

Reviewer comment (Novelty):
The scope covered by the proposed benchmark is too narrow, compared to recent work on benchmarking the tool-using ability. Although the authors may want to have a focused contribution to this specialized direction (i.e., conversation), the difference with other work is not that clear. For example, a lot of recent work has tried to evaluate the tool-using ability to solve real-world tasks. In this case, real-world tasks can always happen during the conversation. See [1] for an example.

Author response:
As you pointed out, we focus on contributing to a specialized direction, conversation. We chose to do so since we are not trying to replace existing benchmarks for tool-using LLMs, but rather fill a gap apparent in existing work. Specifically, existing benchmarks fail to measure performance on complex tasks that require multiple utterances from a human to specify such as in a conversation. While this gap may seem narrow, it is of particular importance since we can expect most humans to interact with tool-using LLMs in this way, and indeed many already do such as ChatGPT, Anthropic's Claude, or Microsoft's Bing Chat.

While it is true real-world tasks often do occur during a conversation, a single real-world task does not necessarily encapsulate the entire goal of a user for the conversation. For example, [1] makes use of some popular mathematical reasoning datasets. We can imagine a scenario where a user wishes to answer a single math question in the conversation, in which case existing mathematical reasoning benchmarks might be sufficient. But we can also imagine scenarios where a user wishes to answer several, potentially dependent, math questions like a student, or even combine domains such as a user trying to balance dieting and a grocery budget. In this case, measuring performance on each individual instance outside of a conversation is less meaningful as failure to succeed at one or more instances would mean failure to accomplish the user’s end goal. Furthermore, specifying these user tasks in a conversation allows us to create complex tasks without having potentially long and unwieldy task descriptions. We find existing benchmarks do not capture these nuances of specifying tasks in a conversation resulting in the creation of ToolTalk.

There are also several differences when comparing ToolTalk to [1] in particular. For one, [1] improves the realism of existing benchmarks by simulating a human in the loop when an incorrect answer is provided. However, the benchmarks re-used still specify their tasks in a single utterance, and thus the “dialogue” provided by [1] is not used to specify additional tasks in a conversation instead allowing for the LLM to “retry” at a single task. Thus, the concept and usage of dialogue is completely different between ToolTalk and [1].

Reviewer comment (Experiments):
The evaluation is too limited. This is a fundamental drawback for this kind of benchmark paper. Only two models, namely GPT-3.5-turbo and GPT-4, are considered in the experiments. It would be more interesting to see the results of some open-sourced models, like Vicuna, and Alpaca.

Author response:
Thank you for your feedback, we were unable to evaluate more models due to time constraints. We are doing so for future revisions to our paper.

Reviewer comment (Presentation):
The overall presentation needs to be improved. For example, the two algorithm blocks on Page-5 do not provide too much useful information.

Author response:
Thank you for your feedback, what did you find confusing and additional information would be useful?

Reviewer comment (Evaluation):
The number of involved tools is small, which may lead to biased evaluation for the LLMs measurement.

Author response:
While the number of tools is small compared to recent works, we believe it is sufficient to show the shortcomings of GPT-4 and GPT-3.5. For example, both models tend to over-trigger, calling tools before all relevant information from the user has been supplied. This issue generally occurs for all tools though it is more consequential for action tools. While the occurrence of this problem might vary for any new, unseen tools, it seems safe to theorize that over-triggering is an issue not specific to any particular tool.

We also generally chose the tools to reflect conversations we might expect for personal productivity, a topic that is likely to receive a lot of use for tool-augmented LLMs. So even in the unlikely event many of the issues we found in our analysis are only specific to these kinds of tools, we can expect many people to still experience these issues when conversing with a tool-augmented LLM.

Ultimately, the current number of tools seems sufficient to demonstrate that evaluating tool-augmented LLMs in a conversational setting is a new task with unique characteristics that even some of the most state-of-the-art LLMs struggle to perform well on. However, adding additional tools is a clear target for future versions of ToolTalk,

Reviewer comment (Experiments):
There is a lack of experiments on open-sourced LLMs, and how to enhance the ability of LLMs on their proposed benchmark is missing.

Author response:
Thanks for your feedback. We plan on evaluating more LLMs for future revisions.

Reviewer comment (Writing):
It's not clear whether the user is always requesting that specific tools are invoked (e.g., "search my email", "check the weather") or whether there are situations wherein the assistant is expected to infer tool usage from a passive statement (e.g., "I forgot who emailed me about X" --> search_email(X), or "John asked me to do X by tomorrow" --> add_reminder(X, tomorrow)). Additional examples would help.

Author response:
We plan on adding additional example visualizations in the revision. To clarify now, the assistant is generally expected to infer what tool to use. The creation of ToolTalk was done under the assumption that the user does not necessarily know the exact tools enabled by any particular plugin, and thus will not say things like “Use AddReminder to create a reminder to do X by tomorrow”.

Reviewer comment (Evaluation):
Will the proposed methodology generalize to a tool such a search engine or general-purpose retrieval? I imagine the tool correctness metric would need to change.

Author response:
Yes, the methodology should extend to search. We already implement some basic search tools such as searching for emails or messages. These tools have a tool correctness metric that simply checks that all relevant emails or messages have been found regardless of the input to the tool. Given a static search database, a similar tool correctness metric could be implemented for search where we check for a few key search results and maybe additionally check that the search query is relatively similar to the ground truth.