Challenging Large Language Models with New Tasks: A Study on their Adaptability and Robustness (2024)

Abstract: Recent progress in large language models (LLMs) has marked a notable milestone in the field of artificial intelligence. The conventional evaluation of LLMs primarily relies on existing tasks and benchmarks, raising concerns about test set contamination and the genuine comprehension abilities of LLMs. To address these concerns, we propose to evaluate LLMs by designing new tasks, automatically generating evaluation datasets for the tasks, and conducting detailed error analyses to scrutinize LLMs’ adaptability to new tasks, their sensitivity to prompt variations, and their error tendencies. We investigate the capacity of LLMs to adapt to new but simple tasks, especially when they diverge from the models’ pre-existing knowledge. Our methodology emphasizes the creation of straightforward tasks, facilitating a precise error analysis to uncover the underlying causes of LLM failures. This strategic approach also aims to uncover effective strategies for enhancing LLM performance based on the detailed error analysis of system output.

## introduction llms have produced impressive results on many nlp tasks @xcite @xcite , and existing studies for evaluating llms mainly leverage benchmark datasets for existing tasks, such as summarization @xcite , question answering @xcite @xcite , reasoning @xcite , and math word problems @xcite @xcite . these evaluation approaches face the test set contamination issue @xcite . it is also difficult to determine to what degree the * equal contribution. † corresponding author. 1 the code and processed datasets are available at https: //github.com/clineek/elagent . another challenge with prior research lies in the increasing complexity of benchmark tasks. paired with the frequent absence of thorough error analysis, the resulting performance metrics often amount to mere numbers. they lack the substantive insights necessary for understanding why models struggle with certain tasks, which is essential for improving llms in the future. in this study, we focus on the following research questions: (q1) how effectively can llms adapt to new tasks, particularly when instructions in the prompts conflict with the model's prior knowledge? (q2) how robust are llms to minor variations of prompts? (q3) what are the common error types made by llms in our designed tasks? (q4) drawing from the insights gained in the preceding questions, what strategies can be employed to enhance llms' performance on specific tasks? to answer these questions, we propose a new paradigm of evaluating llms with automatically generated evaluation datasets for new tasks (elagent) -we create new tasks and new datasets instead of exploiting current benchmarks -to reduce the risk of data contamination. more importantly, our tasks are designed to be novel but simple to facilitate identifying the source of llms' deficiencies during error analysis. in addition, based on detailed error analysis, we improve system performance via techniques such as chainof-thought (cot) @xcite . ## experiment setting we run the experiments on gpt4-turbo foot_1 (openai, 2023), llama-2-chat-70b (llama2) @xcite , llama-3-70b-instruct (llama3) (ai@meta, 2024), and flan-t5-xxl (t5) @xcite . for comparison, we also randomly select some instances from each task's dataset and ask human annotators to answer the questions. ## gptresults in all the tables in this section that report system performance, each cell reports accuracy on 200 test instances except for task 5, which uses 250 test instances modified from the original babi dataset. for task 1-3, we experiment with chain of thought (cot) with 0-shot and 1-shot. in this section, we include only the results with 0-shot without cot (denoted as 0-s) and 1-shot with cot (denoted as 1-s-c). more results including the ones under the settings of 0-shot-with-cot and 1-shotwithout-cot are in appendix b. the full prompts and some system output for task 1-3 are in appendix c-e. ## discussion we have conducted extensive error analysis across tasks. in this section, we synthesize the findings from the error analysis to answer our research questions. ## related work numerous studies have been conducted to assess llms. many have extensively examined a diverse range of llm abilities on existing benchmarks @xcite @xcite , but the results may be influenced by the data contamination problem @xcite . several studies have integrated strategies to mitigate such risks in dataset development. for instance, @xcite devised counterfactual tasks that are less likely to have been encountered by llms, while saparov and he (2022) constructed datasets featuring false and fictional ontologies, which are potentially unfamiliar to llms. our work differs from theirs in that we designed six new tasks and conducted a comprehensive error analysis to identify common error types made by llms on these tasks. in terms of the robustness of llms, much attention is focused on evaluating their resistance to adversarial attacks @xcite . a related study explores the effect of reordering answer choices in multiple-choice questions on model performance @xcite . we test the robustness of llms in several ways, such as randomizing the spellings of words (in task 3-6) and altering the placement of key information (in task 4-5). ## conclusion in this study, we propose a new paradigm for evaluating llms, which includes three crucial components: designing simple tasks, automatically generating the evaluation dataset, improving system performance based on thorough error analysis. our experiments show llms such as gpt4 struggle with simple tasks when instructions conflict with prior knowledge. our research also highlights llms' robustness to some types of noise but vulnerability to other variations, such as the position of key information in the prompts. additionally, we have shown that extensive error analysis can guide the design of cot by aiming at eliminating common errors and improving model performance. the six tasks used in this study cover a small portion of the spectrum in assessing llms. therefore, we intend to develop additional tasks to explore other facets of llm capabilities. also, while our study focuses on analyzing the output of gpt4, we aim to extend our analysis to include other models. by conducting detailed error analyses on the outputs of these models, we hope to identify common patterns in errors across different llms. this approach could potentially offer insights into the underlying architectures of llms and contribute to their further development.