Q ual E val: Qualitative Evaluation for Model Improvement (2024)

Abstract: Quantitative evaluation metrics have been pivotal in gauging the advancements of AI systems like large language models (LLMs).However, due to the intricate nature of real-world tasks, a single scalar to quantify and compare performance trivializes the fine-grained nuances of model behavior. Additionally, metrics do not yield actionable diagnostics for model improvement, thus requiring extensive manual efforts of scientists, involving sifting through vast datasets and attempting hit-or-miss adjustments to training data or setups. In this work, we address the shortcomings of quantitative metrics by proposing QualEval, which uses automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights that when applied, accelerate model improvement. The insights are supported by a dashboard report with fine-grained visualizations and human-interpretable analyses. We corroborate the faithfulness of QualEval by demonstrating that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative on a challenging dialogue task (DialogSum) when compared to baselines. QualEval successfully increases the pace and quality of model development by eliminating the need of arduous manual analysis, thus serving as a data-scientist-in-a-box.

## introduction the recent success of large language models (llms) while can be attributed to data and compute scaling, has also been the result of evaluation metrics that allow benchmarking and comparison of models. this surge in the development of llms and associated tasks has reignited the need for innovative evaluation methods, aiming to provide more effective guidance throughout the model development process @xcite . traditional scalar quantitative metrics like perplexity, bleu, and rouge play an important role in objectively measuring improvements in model performance. however, these scalar metrics cannot capture the nuances of model behavior and therefore are unable to provide model developers actionable directions and diagnostics for model improvement @xcite . in practice, this deficiency necessitates model developers to collaborate with an army of data scientists and engineers to iterate over a diverse array of models and tasks, especially in rapidly evolving real-world settings. in this work, we use "quality over quantity" as a guiding principle to propose our model and task agnostic method qualeval, that uses qualitative evaluation to address the issues with quantitative metrics. given a model that is being developed for a task, quale-val serves as an automated data scientist by analyzing the dataset and the model's predictions to generate actionable directions that improve the model, supported by a comprehensive dashboard containing fine-grained analysis of the model's behavior (figure 1 ). the directions identified by qualeval to improve the model significantly expedite the model development lifecycle. rather than rejecting the use of metrics, qualeval uses them as just one of the parts of a more holistic and useful evaluation. qualeval's algorithm for facilitating model improvement can be broken down into three steps (figure 2 ): (1) attribute discovery: automatic discovery of domains and sub-tasks in the dataset, to help identify issues at a fine-grained level. (2) attribute assignment: utilize a novel flexible linear programming solver to assign attributes to instances and analyze the performance on different attributes to create a humanreadable dashboard. (3) insight generation: parse the generated dashboard to provide natural language insights that improve the model. qualeval's end-toend pipeline is completely automated and requires no human intervention. we demonstrate qualeval's potency on a wide range of tasks including code generation, dialogue summarization, and multiple-choice question answering. we harness these insights provided by quale-val to precisely and significantly improve the performance of the open-source llama 2 model on a dialog summarization task. in a demonstration of efficacy, qualeval's insights allow a model practitioner to make changes to the fine-tuning procedure by augmenting with the right instances, thus leading to an over- ## experimental setup datasets we evalaute qualeval on three datasets: mbpp @xcite (sanitized), dialog-sum @xcite , and mmlu @xcite (clinical knowledge split). mbpp and dialog-sum involve generative tasks and involve generating a python program from a prompt and summarizing a conversation respectively. mmlu contains a wide range of multiple-choice questions from different domains and we pick the clinical knowledge split to evaluate our model on knowledge-intensive tasks. we use the same evaluation splits as the original papers and use the test splits for mbpp and mmlu and the validation split for dialogsum. we use the pass@1, rouge-l, and accuracy as proficiency metrics for mbpp, dialogsum, and mmlu respectively. models we use both closed and open-sourced models: curie, davinci-2, and davinci-3 models from openai and llama 2 (7 billion chat models @xcite ). we use a temperature of 0.9 for all models and use two randomly sampled in-context samples for prompting models unless mentioned otherwise. we instantiate e with the gpt-3.5-turbo model @xcite . llama fine-tuning we use lora @xcite to efficiently fine-tune the llama 2 7 billion parameter model and train with 8 bit precision. we sweep over five learning rates (2e -5, 5e -5, 1e -4, 2e -4, 1e -3) and pick the checkpoint with the best validation performance. we train for up to 400 steps and we use a constant learning rate schedule. ## results we systematically present different aspects of our dashboard. firstly, we show that attribute discovery (domains and sub-tasks) of qualeval is well-grounded and faithful to the dataset. secondly, we show that qualeval's flexible lp solver correctly assigns attributes to instances of the dataset, allowing it to perform meta-reasoning over different domains and subtasks. finally, we validate that the concise natural language insight generated leads to improvement in the model's performance. ## related work model debugging/improvement prior work has attempted to address the problem of model debugging and improvement. @xcite propose to evaluate different pairs of models on separate evaluation splits to understand model behavior. they also generate feature-level importance scores from "symptom" instances provided by humans. @xcite introduce a model-agnostic method to find global features that "influence" the model evaluation score, allowing practitioners to exclude problematic features. @xcite develop a framework to generate explanations for model predictions to allow humans to give feedback and debug models. @xcite presents a framework to gener-ate test cases at scale to evaluate model robustness, but constrains the test cases to be generated from simple templates and lexical transformations. @xcite propose a framework to generate counterfactual explanations for model errors to enable a better understanding of model behavior. @xcite introduce self-debugging, a method to enable a large language model to debug the predicted computer program through few-shot demonstrations. some other works attempt to find error-prone slices of the data to improve the model @xcite @xcite . while these works provide limited insights into model behavior, they often require significant human intervention to understand model behavior and do not provide precise actionable insights for model improvement. finally, these works are constrained to simple classification and regression tasks or single domains like code generation and do not provide a task-agnostic, fully automated framework for model interpretation and improvement for real-world tasks. ## conclusion we propose qualeval, a qualitative evaluation framework that provides a comprehensive way of evaluating models with a keen eye on model improvement. rather than rely on scalar quantitative metrics that ignore the nuanced behavior of the model, qualeval augments quantitative metrics to test the model thoroughly and provides actionable insights through an interpretable dashboard to improve the model iteratively. we demonstrate that these insights are faithful and lead to up to 15% relative improvement. our work is the first step towards building a data-scientist in a box.