On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering (2024)

Abstract: This study introduces a new long-form database question answering dataset designed to evaluate how Large Language Models (LLMs) interact with a SQL interpreter. The task necessitates LLMs to strategically generate multiple SQL queries to retrieve sufficient data from a database, to reason with the acquired context, and to synthesize them into a comprehensive analytical narrative. Our findings highlight that this task poses great challenges even for the state-of-the-art **GPT-4** model. We propose and evaluate two interaction strategies, and provide a fine-grained analysis of the individual stages within the interaction. A key discovery is the identification of two primary bottlenecks hindering effective interaction: the capacity for planning and the ability to generate multiple SQL queries. To address the challenge of accurately assessing answer quality, we introduce a multi-agent evaluation framework that simulates the academic peer-review process, enhancing the precision and reliability of our evaluations. This framework allows for a more nuanced understanding of the strengths and limitations of current LLMs in complex retrieval and reasoning tasks.

## introduction significant advancements in natural language processing have been driven by the development of large language models (llms) @xcite @xcite @xcite , which have become fundamental components of numerous products used by millions, reshaping people's habits on accessing information. despite their widespread adoption and impact, llms face intrinsic limitations due to their design, including limited context window, stochastic nature which makes them less suited for tasks requiring high standards of precision, and extensive computations @xcite @xcite . many studies have explored ways to mitigate these constraints by augmenting llms with modules/tools of complementary features @xcite @xcite @xcite . in our study, we focus on augmenting llms with a symbolic module -a sql code interpreter -and assess their performance using the long-form database question-answering task that we introduce, illustrated in figure 1 . such augmentation is inevitable for tasks involving databases, as they often far exceed the size of llms' context windows foot_0 , making information retrieval through any means other than sql inefficient. additionally, the use of sql queries brings transparency to the reasoning process of llm agents, providing a means to validate the accuracy of their generated responses. llms augmented with external modules/tools possess two primary abilities: the capacity to act, which involves the use of tools, and the capability to reason, which encompasses planning and analyzing the outcomes of actions @xcite @xcite @xcite . while numerous studies have evaluated these abilities in different contexts, we contend that some of them focus more on evaluating tool selection and tool employment with less focus on evaluating how llm agents reflect or synthesize the action results @xcite @xcite . other research @xcite @xcite does examine both the action and reasoning capacities of llm agents, yet the actions' complexity is not as demanding as in studies with a stronger focus on the action aspect. our goals are twofold: firstly, to introduce a task that places figure 1 : illustration of our long-form database question answering task. the llm agent is expected to perform a series of tasks requiring reasoning and actions to interact with the database module. property value evaluation dataset size 200 -# conclusive questions 98 -# interpretive questions 102 reference answer length -conclusive questions (avg.) 132 -interpretive questions (avg.) 209 database size -# tables (med.) 4 -# columns (med.) 4 -# data records (med.) 11 ## data collection in constructing our evaluation dataset, we prioritize a robust set of desiderata. these include the intensive retrieval of diverse information from the database, the application of rigorous reasoning over the information retrieved, and the synthesis of facts and inferences into a coherent and comprehensive long-form answer. our methodology employs a hybrid annotation framework: we leverage the capabilities of gpt-4 to generate preliminary annotations, then these annotations are selected and refined through manual intervention to ensure quality and relevance. the specifics and quantitative details of our evaluation dataset are presented in ## methods we aim to evaluate how effective llms are at performing a complex task that necessitate working with external modules. we explore five main aspects: (1) the proficiency of llms in completing our proposed task through interaction with external modules; (2) the extent of improvement llms gain from engaging with external modules; (3) the impact of various interaction strategies on llm performance and the identification of the most effective one; (4) the challenges that hinder effective interaction; (5) the generalizability of our findings across diverse llms and the performance disparities attributed to the usage of different llms. we can address the first, second and last aspects by directly evaluating the quality of the final answer generated by llms. to delve into the third and fourth aspects, we need to first dissect the "interaction" process within our task into its constituent components. we propose to decompose the llms' expected workflow for our task into three distinct sub-tasks: interaction planning, tool employment, and information synthesis. interaction planning involves the llm determining its interaction strategy with the external module, considering the question and past interactions. tool employment is the phase where the llm generates module-specific commands for the actual interaction. information synthesis requires the llm to review the interaction history and any newly acquired context to compile the key information for the final answer. this framework allows us to refine our second objective into assessing how different compositions of these sub-tasks affect the quality of the final answer, and also to define the most effective composition. the third objective can be addressed by evaluating llm's execution within each sub-task. while the potential configurations of these subtasks are vast, this study will narrow its focus to two primary interaction strategies for feasibility: equipped with these strategies, we proceed to empirically explore our central questions. ## conclusion in conclusion, our investigation reveals the current limitations of llms in complex retrieval and reasoning tasks. augmentation with a sql module proved beneficial, particularly for conclusive questions, and pointed to the necessity of strategic interaction planning and proficient tool employment. our findings stress the need for improvement in these areas to enhance llm effectiveness. despite the challenge of varying performance across different models, our multi-agent evaluation framework provides a scalable and rigorous method for assessing agent capabilities. we hope that our proposed task and findings will encourage further investigations in llms' capabilities of interacting with external modules, inching towards llms capable of handling complex tasks with enhanced precision.