I Need Help! Evaluating LLM ’s Ability to Ask for Users’ Support: A Case Study on Text-to- SQL Generation (2024)

Abstract: This study explores the proactive ability of LLMs to seek user support. We propose metrics to evaluate the trade-off between performance improvements and user burden, and investigate whether LLMs can determine when to request help under varying information availability. Our experiments show that without external feedback, many LLMs struggle to recognize their need for user support. The findings highlight the importance of external signals and provide insights for future research on improving support-seeking strategies. Source code: https://github.com/appier-research/i-need-help

## introduction the impressive instruction-following @xcite abilities of large language models (llms) have enabled their out-of-the-box usage to solve problems. however, these models generate hallucinated content @xcite or incorrect predictions in their efforts to fulfill user instructions, which undermines their reliability. when llms generate incorrect outputs for a given instruction, the issue can be examined from multiple perspectives. one is that the model simply lacks the competence to satisfy the instruction, suggesting a straightforward solution: enhancing the model's capabilities, which is the focus of most previous research. another is that the model could actually solve the task with additional support. for instance, @xcite found that models often fail due to underspecified natural language queries. similarly, @xcite showed that while gpt-4 struggles initially, its performance can improve by up to 20.01% with human-annotated external knowledge. in such cases, models should proactively seek help rather than attempting to satisfy instructions with insufficient information. ## main results using the formulation in section 2.2, we quantify the performance of different methods with the area under delta-burden curve (audbc) in visualized dbcs are available in the leftmost subplots in figure 2 . note that audbc should only be compared between methods under the same llm, as it is normalized to the range of [0, 1] by dividing the area under the curve by the maximum square area, which depends on the scale of ∆ex and differs across llms, as shown in there are three major findings: (1) execution then ask consistently improves the performanceburden trade-off for llms, although llama-3-70bchat fails to outperform the random baseline. (2) the leftmost four llms in ## related work the ability of llms to identify the need for support relies on their well-calibratedness @xcite , which refers to their capacity to recognize uncertainty. previous studies focus on enhancing the calibration of predictions @xcite , or using verbalized token probabilities to achieve better calibration @xcite . our work extends this line of research by exploring how llms can effectively seek user support by leveraging their well-calibrated property. the major distinction between this and existing calibration studies lies in extending the focus from identifying the uncertainty to utilizing support. ## conclusion we propose a framework for llms to seek support, and evaluate methods on text-to-sql generation. our findings suggest the importance of external signals, such as sql execution results, in helping llms better manage performance-burden trade-off. we further decompose dbc into the ability of identify the need for support and the ability to utilize the support. future works may explore a broader range of tasks or develop methods to improve both the identification and utilization of support.