Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025)

Abstract: Retrieval-Augmented Generation (RAG) has proven its effectiveness in alleviating hallucinations for Large Language Models (LLMs). However, existing automated evaluation metrics cannot fairly evaluate the outputs generated by RAG models during training and evaluation. LLM-based judgment models provide the potential to produce high-quality judgments, but they are highly sensitive to evaluation prompts, leading to inconsistencies when judging the output of RAG models. This paper introduces the Judge-Consistency (ConsJudge) method, which aims to enhance LLMs to generate more accurate evaluations for RAG models. Specifically, ConsJudge prompts LLMs to generate different judgments based on various combinations of judgment dimensions, utilizes the judge-consistency to evaluate these judgments, and selects the chosen and rejected judgments for DPO training. Our experiments show that ConsJudge can effectively provide more accurate judgments for optimizing RAG models across various RAG models and datasets. Further analysis reveals that judgments generated by ConsJudge have a high agreement with the superior LLM. All codes are available at https://github.com/OpenBMB/ConsJudge.

## introduction retrieval-augmented generation (rag) @xcite @xcite has proven effective in mitigating hallucinations @xcite @xcite in large language models (llms). rag retrieves relevant knowledge from the knowledge base and incorporates the external information as the input context @xcite , benefiting various knowledge-intensive tasks @xcite @xcite . existing studies typically employ automated evaluation metrics to assess the outputs of rag systems during both training @xcite and evaluation @xcite phases. these metrics primarily focus on string-level exact matching, which is less effective for determining whether the llm-generated responses align with the ground truth @xcite , posing a challenge for rag systems. to enable more accurate evaluation of rag models, some works incorporate llms as judgment models to assess the quality of generated responses @xcite @xcite , relying on their human-equivalent evaluation capabilities @xcite @xcite . these judgment models introduce specific evaluation dimensions, such as hallucination and comprehensiveness, to prompt llms to verify whether the generated responses align with the facts in the retrieved documents and whether all relevant information has been properly extracted and integrated @xcite @xcite . however, llm-based judgment models are highly sensitive to the design of evaluation prompts @xcite , which may lead to inconsistencies in judgments when different evaluation dimensions are employed. in this paper, we introduce judge-consistency (consjudge), a method that enhances llm-based judgment models to generate more accurate evaluations for rag models in a self-improvement framework. consjudge incorporates specialized evaluation dimensions and employs a multiple-choice selection strategy for evaluation modeling. additionally, consjudge leverages the direct preference optimization (dpo) method @xcite to enhance the judgment capabilities of llms, while also implementing a "judge as a judge" mechanism during choosing preference pairs of judgments. specifically, consjudge encourages llms to generate judgment results based on various combinations of judgment dimensions, evaluates the quality of these judgments using judge-consistency, and selects the chosen and rejected judgments for dpo training. this approach enhances the performance of the llm-based judgment model without necessitating distillation from more powerful llms @xcite . our experiments demonstrate the effectiveness of the consjudge method as a reward model for optimizing the rag model, resulting in significant improvements over vanilla llms. further analysis shows that consjudge helps llms select appropriate evaluation dimensions for assessing response quality by incorporating the agreements and consistency across different evaluation dimensions during optimization. additionally, compared to other baselines, consjudge exhibits higher judgment consistency with the superior llm, glm-4plus @xcite , across various rag evaluation datasets, further highlighting its effectiveness in optimizing llms to produce more accurate judgments on diverse rag tasks. ## related work retrieval-augmented generation (rag) has proven its effectiveness in various tasks, such as open-domain question answering @xcite , dialogue systems @xcite , and code generation @xcite . by integrating external knowledge into the input context @xcite , large language models (llms) can alleviate hallucination issues and produce more accurate responses @xcite . to evaluate the performance of llm responses, most existing rag models rely on automatic metrics like exact-match during both evaluation @xcite and training @xcite . however, these metrics may fail to offer a fair assessment when the llm-generated responses are lengthy or semantically similar to the ground truth but not character-matched @xcite . recently, llms, such as chatgpt (openai, 2023), have demonstrated human-equivalent performance @xcite @xcite and are now commonly used as judgment models to assess generated responses in various tasks @xcite @xcite . these studies typically prompt llms to evaluate generated responses based on specific evaluation dimensions @xcite . however, using apis of these closed-source models incurs significant costs and reduces reproducibility, as the models may evolve behind the api @xcite . to address this, some researchers are turning to open-source llms as alternatives, using judgments from closed-source llms to fine-tune open-source models and improve their evaluation capabilities @xcite . moreover, rag models also employ llms as judges to assess the quality of generation, focusing on the relevance and faithfulness of the responses @xcite @xcite . these models typically design prompts to instruct llms to determine whether the generated response aligns with the facts in the retrieved document and whether all relevant information has been fully extracted and integrated @xcite @xcite . however, llm-based evaluations are highly sensitive to prompt designs @xcite , making the judgments inconsistent when using different dimensions for evaluating rag responses. to mitigate this is- query: which actor played spider man in spider man 3? a: tobey maguire b: will smith c: tom cruise d: jon hamm different llm give a {hybrid evaluation}, which of the four options {abcd} is the best and which is the worst for the given {query} and {gold} . select dimensions, get k different hybrid evaluations. sue, consjudge introduces a judge-consistency approach to self-improve the judgment performance of llms in rag systems, avoiding distillation from larger-scale llms @xcite . ## methodology in this section, we introduce the judge-consistency (consjudge) method, which optimizes large language models (llms) as judgment models to evaluate the effectiveness of the retrieval-augmented generation (rag) system. first, we discuss the evaluation methods used in consjudge (sec.3.1). next, we present how the consjudge method is utilized to optimize the judgment model, improving its judgment accuracy (sec.3.2). finally, we apply the consjudge method as the reward model to optimize the rag system (sec. 3.3). ## experimental methodology this section describes the datasets, evaluation metrics, baselines, and implementation details used in our experiments. more implementation details are shown in appendix a.6. datasets. we describe the datasets used for training consjudge and rag training and evaluation. consjudge training. for training consjudge, we collect 11 knowledge-intensive tasks from previous works @xcite to collect 73,831 instances and 3,886 instances to construct both training and development sets. ## evaluation result in this section, we first show the performance of consjudge by regarding it as a reward model to optimize the rag model. then we conduct the ablation studies to show the effectiveness of different modules in consjudge. subsequently, we evaluate the judgment quality generated by consjudge and explore the consistency of judgments across different evaluation dimensions. finally, case stud- ## conclusion this paper proposes the consjudge method, which enhances the judgment ability of llms in a selfimprovement method. it prompts llms to generate different judgments based on various combinations of judgment dimensions and utilizes the consistency between judgments to enhance llms to generate more accurate judgments. our experimental results show that consjudge has the ability to choose appropriate evaluation dimensions to generate reliable and comprehensive judgments.