Towards Understanding the Robustness of LLM -based Evaluations under Perturbations (2024)

Abstract: Traditional evaluation metrics like BLEU and ROUGE fall short when capturing the nuanced qualities of generated text, particularly when there is no single ground truth. In this paper, we explore the potential of Large Language Models (LLMs), specifically Google Gemini 1, to serve as automatic evaluators for non-standardized metrics in summarization and dialog-based tasks. We conduct experiments across multiple prompting strategies to examine how LLMs fare as quality annotators when compared with human judgments on the SummEval and USR datasets, asking the model to generate both a score as well as a justification for the score. Furthermore, we explore the robustness of the LLM evaluator by using perturbed inputs. Our findings suggest that while LLMs show promise, their alignment with human evaluators is limited, they are not robust against perturbations and significant improvements are required for their standalone use as reliable evaluators for subjective metrics.

## introduction natural language generation (nlg) tasks such as abstractive summarization and dialog completion are essential for advancing human-computer interaction and automating content generation. however, the evaluation of such tasks poses unique challenges, especially when traditional metrics like bleu and rouge, which rely on token overlap with reference texts, fail to account for the inherent subjectivity and flexibility in the human language. this limitation has sparked substantial research efforts to explore more effective automated evaluation methods @xcite . in tasks where there is no definitive ground truth, nuanced metrics such as coherence and fluency are critical for evaluating text quality. although human evaluation has long been the gold standard in such contexts, it presents several limitations. human evaluators are susceptible to errors, and large-scale assessments demand significant time and resources. furthermore, relying exclusively on human evaluations can result in inconsistencies in evaluation quality, and evaluators may struggle to accurately assess content outside their area of expertise. in contrast, large language models (llms) like ope-nai's gpt-4 @xcite ), google's gemini @xcite , and meta's llama @xcite offer the potential to function as fast and inexpensive domain experts, utilizing their extensive background knowledge to provide more consistent and informed evaluations. as a result, it is important to investigate automated evaluation methods to reduce the reliance on human evaluators, thereby enhancing the efficiency and scalability of the evaluation process. in this paper, we aim to: evaluation metrics in nlg have traditionally focused on bleu @xcite , meteor @xcite , and rouge @xcite , which compare generated text with reference ground truths. while effective for tasks like machine translation, they struggle with abstractive summarization and dialog evaluation, where multiple valid outputs exist, and coherence is critical. recent research has introduced reference-free metrics like bertscore @xcite and qaeval @xcite , but these approaches require human intervention and lack flexibility @xcite across diverse generation tasks. more recent efforts explore using llms for evaluation @xcite , which significantly reduces costs but still faces challenges in robustness, especially in adversarial settings. studies like @xcite and @xcite show vulnerabilities in llm-based assessments, revealing their susceptibility to adversarial attacks that manipulate evaluation scores. these findings emphasize the need for more robust frameworks as llm-generated labels become more prevalent. ## results to assess the reliability of llm-based evaluations and their alignment with human raters, we used krippendorff's alpha @xcite as the primary metric. we chose krippendorff's alpha because it is well-suited for tasks requiring multiple raters and is capable of handling ordinal data, such as likert scale ratings, which are used in both the summeval and usr datasets. to calculate krippendorff's alpha, we extracted ratings for each metric from the summeval and usr datasets, treating them as ordinal data. we used the python krippendorff package, structuring the data as a matrix with rows as items and columns as ratings, handling missing values via the library's default settings. this enabled a direct comparison of llm and human reliability. ## conclusion and future work google's gemini-1 shows consistency across prompting strategies but is vulnerable to adversarial perturbations, emphasizing the need for stronger training to improve llm reliability in subjective evaluation tasks. our experiments revealed significant shifts in scores and justifications under adversarial conditions, underscoring the importance of robustness in llm evaluators. a limitation of this work is the lack of computational resources to test models like llama and gpt. including these models in future research could provide more diverse perspectives and improve evaluation robustness. future studies should explore small language models (slms) like google's gemma (et al, 2024a) and microsoft's phi series @xcite , as well as their ability to handle subjective metrics @xcite . expanding tasks to areas like translation quality and multilingual evaluation @xcite could offer deeper insights into how llms capture contextual nuances. finally, investigating why gemini rejected entries from the summeval dataset, possibly due to jailbreak attacks @xcite , could enhance llm reliability as evaluators.