G -Eval: NLG Evaluation using Gpt-4 with Better Human Alignment (2023)

Abstract: The quality of texts generated by natural language generation (NLG) systems is hard to measure automatically. Conventional reference-based metrics, such as BLEU and ROUGE, have been shown to have relatively low correlation with human judgments, especially for tasks that require creativity and diversity. Recent studies suggest using large language models (LLMs) as reference-free metrics for NLG evaluation, which have the benefit of being applicable to new tasks that lack human references. However, these LLM-based evaluators still have lower human correspondence than medium-size neural evaluators. In this work, we present G-Eval, a framework of using large language models with chain-of-thoughts (CoT) and a form-filling paradigm, to assess the quality of NLG outputs. We experiment with two generation tasks, text summarization and dialogue generation. We show that G-Eval with GPT-4 as the backbone model achieves a Spearman correlation of 0.514 with human on summarization task, outperforming all previous methods by a large margin. We also propose analysis on the behavior of LLM-based evaluators, and highlight the potential concern of LLM-based evaluators having a bias towards the LLM-generated texts.

## introduction evaluating the quality of natural language generation systems is a challenging problem even when large language models can generate high-quality and diverse texts that are often indistinguishable from human-written texts @xcite . traditional automatic metrics, such as bleu @xcite , rouge @xcite , and me-teor @xcite , are widely used for nlg evaluation, but they have been shown to have relatively low correlation with human judgments, especially for open-ended generation tasks. moreover, these metrics require associated reference output, which is costly to collect for new tasks. recent studies propose directly using llms as reference-free nlg evaluators @xcite . the idea is to use the llms to score the candidate output based on its generation probability without any reference target, under the assumption that the llms have learned to assign higher probabilities to high-quality and fluent texts. meanwhile, it is becoming popular to use more powerful llms like gpt-4 to evaluate smaller or student models, like in alpaca @xcite and vicuna @xcite . however, the validity and reliability of using llms as nlg evaluators have not been systematically investigated. in addition, meta-evaluations show that these llmbased evaluators still have lower human correspondence than medium-size neural evaluators @xcite . thus, there is a need for a more effective and reliable framework for using llms for nlg evaluation. in this paper, we propose g-eval, a framework of using llms with chain-of-thoughts (cot) @xcite to evaluate the quality of generated texts in a form-filling paradigm. by only feeding the task introduction and the evaluation criteria as a prompt, we ask llms to generate a cot of detailed evaluation steps. then we use the prompt along with the generated cot to evaluate the nlg outputs. the evaluator output is formatted as a form. moreover, the probabilities of the output rating tokens can be used to refine the final metric. we conduct extensive experiments on three meta-evaluation benchmarks of two nlg tasks: text summarization and dialogue generation. the results show that g-eval can outperform existing nlg evaluators by a large margin in terms of correlation with human evaluations. finally, we conduct analysis on the behavior of llm-based evaluators, and highlight the potential issue of llm-based evaluator having a bias towards the llm-generated texts. to summarize, our main contributions and findings in this paper are: ## experiments following @xcite , we meta-evaluate our evaluator on three benchmarks, summeval, topical-chat and qags, of two nlg tasks, summarization and dialogue response generation. ## analysis will g-eval prefer llm-based outputs? one concern about using llm as an evaluator is that it may prefer the outputs generated by the llm itself, rather than the high-quality human-written texts. to investigate this issue, we conduct an experiment on the summarization task, where we compare the evaluation scores of the llm-generated and the human-written summaries. we use the dataset collected in @xcite , where they first ask freelance writers to write high-quality summaries for news articles, and then ask annotators to compare human-written summaries and llmgenerated summaries (using gpt-3.5, text-davinci-003). the dataset can be divided in three categories: 1) human-written summaries that are rated higher than gpt-3.5 summaries by human judges, 2) human-written summaries that are rated lower than gpt-3.5 summaries by human judges, and 3) human-written summaries and gpt-3.5 summaries are rated equally good by human judges. we use g-eval-4 to evaluate the summaries in each category, and compare the averaged scores. 2 the results are shown in figure 2 . we can see that, g-eval-4 assigns higher scores to humanwritten summaries when human judges also prefer human-written summaries, and assigns lower scores when human judges prefer gpt-3.5 summaries. however, g-eval-4 always gives higher scores to gpt-3.5 summaries than human-written summaries, even when human judges prefer humanwritten summaries. we propose two potential reasons for this phenomenon: our work should be considered as a preliminary study on this issue, and more research is needed to fully understand the behavior of llm-based evaluators to reduce its inherent bias towards llmgenerated text. we highlight this concern in the context that llm-based evaluators may lead to self-reinforcement of llms if the evaluation score is used as a reward signal for further tuning. and this could result in the over-fitting of the llms to their own evaluation criteria, rather than the true evaluation criteria of the nlg tasks. the effect of chain-of-thoughts we compare the performance of g-eval with and without chain-of-thoughts (cot) on the summeval benchmark. ## related work ngram-based metrics ngram-based metrics refer to the scores for evaluating the nlg models by measuring the lexical overlap between a generated text and a reference text. bleu @xcite is the most widely used metric for machine translation evaluation, which calculates the geometric mean of modified n-gram precision and a brevity penalty. rouge @xcite is a recall-oriented metric for summarization evaluation, which measures the n-gram overlap between a generated summary and a set of reference summaries. it has been shown that more than 60% of recent papers on nlg only rely on rouge or bleu to evaluate their systems @xcite . however, these metrics fail to measure content quality @xcite or capture syntactic errors @xcite , and therefore do not reflect the reliability of nlg systems accurately. embedding-based metrics embedding-based metrics refer to the scores for evaluating the nlg models by measuring the semantic similarity between a generated text and a reference text based on the word or sentence embeddings. wmd @xcite is a metric that measures the distance between two texts based on the word embeddings. bertscore @xcite measures the similarity between two texts based on the contextualized embedding from bert @xcite . moverscore @xcite improves bertscore by adding soft alignments and new aggregation methods to obtain a more robust similarity measure. @xcite propose a metric that evaluates multi-sentence texts by computing the similarity between the generated text and the reference text based on the sentence embeddings. task-specific evaluators task-specific metrics refer to the scores for evaluating the nlg models by measuring the quality of the generated texts based on the specific task requirements. for example, summarization tasks need to assess the consistency of the generated summaries @xcite @xcite , and dialogue response generation tasks need to assess the coherence of the generated responses @xcite @xcite . however, these metrics are not generalizable to other nlg tasks, and they are not able to measure the overall quality of the generated texts. unified evaluators recently, some evaluators have been developed to assess text quality from multiple dimensions by varying the input and output contents @xcite or the model variants (mehri and eskenazi, 2020) they use. unieval @xcite is a unified evaluator that can evaluate different aspects of text generation as qa tasks. by changing the question format, it can handle different evaluation tasks. @xcite propose gptscore, a new framework that evaluated texts with generative pre-training models like gpt-3. it assumes that a generative pre-training model will assign a higher probability of high-quality generated text following a given instruction and context. @xcite conduct a preliminary survey of using chatgpt as a nlg evaluator. @xcite ; @xcite proposed to use gpt models for evaluating machine translation tasks. very recently, @xcite investigated the problem of unfairness when using large models in evaluating dialogue responses. ## conclusion in this paper, we propose g-eval, a framework of using llm with chain-of-thoughts (cot) to evaluate the quality of generated texts. we conduct extensive experiments on two nlg tasks, text summarization and dialogue generation, and show that g-eval can outperform state-of-the-art evaluators and achieve higher human correspondence. we also propose preliminary analysis on the behavior of llm-based evaluators, and highlight the potential issue of llm-based evaluator having a bias towards the llm-generated texts. we hope our work can inspire more research on using llms for nlg evaluation, and also raise awareness of the potential risks and challenges of using llms as evaluators.