Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework (2025)

Abstract: Large Language Models (LLMs) are being used more and more extensively for automated evaluation in various scenarios. Previous studies have attempted to fine-tune open-source LLMs to replicate the evaluation explanations and judgments of powerful proprietary models, such as GPT-4. However, these methods are largely limited to text-based analyses under predefined general criteria, resulting in reduced adaptability for unseen instructions and demonstrating instability in evaluating adherence to quantitative and structural constraints. To address these limitations, we propose a novel evaluation framework, ARJudge, that adaptively formulates evaluation criteria and synthesizes both text-based and code-driven analyses to evaluate LLM responses. ARJudge consists of two components: a fine-tuned Analyzer that generates multi-faceted evaluation analyses and a tuning-free Refiner that combines and refines all analyses to make the final judgment. We construct a Composite Analysis Corpus that integrates tasks for evaluation criteria generation alongside text-based and code-driven analysis generation to train the Analyzer. Our results demonstrate that ARJudge outperforms existing fine-tuned evaluators in effectiveness and robustness. Furthermore, it demonstrates the importance of multi-faceted evaluation and code-driven analyses in enhancing evaluation capabilities.

## introduction the rapid advancement of large language models (llms) has highlighted the critical need for robust output evaluation methods @xcite . while proprietary models like gpt-4 have emerged as predominant evaluation approaches given their superior capabilities, transparent and controllable considerations have driven research toward fine-tuning open-source llms for evaluation tasks @xcite . recent work has established the viability of open-source alternatives by training llms to replicate the evaluation explanations and judgments of proprietary models @xcite @xcite . however, existing fine-tuned evaluators rely solely on text-based analysis with predefined evaluation criteria, leading to two key limitations @xcite @xcite . first, evaluation based on predefined criteria can not fully capture the nuanced task requirements. for example, general criteria for writing, such as conciseness or logical structure, may not be sufficient for evaluating creative writing tasks that require an engaging plot. moreover, it is challenging to effectively adapt predefined criteria to new and diverse instructions @xcite . second, llm-based evaluators demonstrate significant instability in evaluating adherence to complex instruction requirements, particularly objective criteria such as quantitative or structural constraints @xcite . for instance, they struggle to reliably assess basic textual attributes such as word counts, a common requirement in writing-related instructions @xcite . these limitations also extend to the evaluation of formatting constraints. in this work, we argue that developing robust fine-tuned evaluators requires the ability to adaptively generate evaluation criteria and conduct multi-faceted analyses @xcite . these abilities enhance the evaluators' comprehensive performance in both what to evaluate and how to evaluate. even for unseen instructions, the evaluators can define tailored criteria and assess instructions with nuanced precision. furthermore, evaluators should use automated tools to assess objective requirements @xcite . these tools provide reproducible feedback, offering reliable verification that helps overcome llms' inherent limitations in objective evaluation. to address these challenges, we propose ar-judge, a novel evaluation framework that combines adaptive criteria generation with text-based and code-driven analysis generation to comprehensively assess llm outputs. arjudge comprises two core components: (1) an analyzer that generates multi-faceted evaluation with text-based and code-driven analyses and (2) a refiner that synthesizes and refines these analyses to produce wellreasoned judgments. we train arjudge on a curated composite analysis corpus, which contains tasks for generating evaluation criteria and performing multi-faceted analyses in both text and code. this corpus enables the analyzer to learn context-sensitive evaluation logic, such as deriving criteria from instructions and assessing responses accordingly. extensive experiments across multiple benchmarks demonstrate arjudge's superiority and robustness over existing open-source evaluators. our further analysis validates the necessity and effectiveness of integrating code-driven analyses, which improve accuracy in evaluating instruction following by up to 11.1% compared to text-only methods. the main contributions of this work include: ## composite analysis corpus collecting comprehensive and detailed evaluation analysis data is essential for fine-tuning an llm to improve evaluation performance @xcite . previous studies @xcite @xcite focus exclusively on text-based analysis with predefined general evaluation criteria, showing limited generalization and robustness @xcite . to address these limitations, we develop a composite analysis corpus to improve llms' ability to determine what to evaluate and how to evaluate effectively. the process of constructing the corpus involves three steps: (1) establishing evaluation criteria specifically for each instruction ( §2.1), ( 2 ) conducting text-based analyses to assess responses using multiple criteria ( §2.2), and (3) designing code-driven analyses to assess whether responses satisfy the objective requirements of the instructions ( §2.3). first of all, we collect a large set of instructions from publicly available preference datasets based on @xcite . these datasets @xcite @xcite consist of preference pairs of llmgenerated responses to identical instructions. each pair is annotated with a preference label that identifies the better response. in line with @xcite , non-english instructions and multi-turn interactions are removed. then, we establish multiple evaluation criteria for each instruction. instruction 𝒙 propose evaluation criteria instruction 𝒙 ! with constraints questions type 2 questions type 1 instruction 𝒙 r1 r2 + questions type 1 detailed analysis & judgment compare & evaluate instruction 𝒙 ! generate evaluation script yes/no reverse validate (1) evaluation criteria (2) text-based analysis (3) code-driven analysis if yes + questions type 2 self-instruct responses sample responses sample + figure 2: the overview of the corpus construction. "r1" and "r2" denote two candidate responses with a preference annotation. "sample responses" are newly sampled responses that we use as references to generate evaluation questions and code scripts. step (1) produces two types of evaluation questions, respectively. step (2) and step (3) develop corresponding text-based and code-driven analyses. ## arjudge after constructing the corpus, we collect around 25k composite training samples. we fine-tune an llm based on them and develop arjudge, a novel evaluation framework that adaptively evaluates llm-generated responses and integrates both text-based and code-driven analyses. arjudge consists of two components: a fine-tuned analyzer and a tuning-free refiner. figure 1 presents the overall framework. the analyzer is trained on the composite analysis corpus to adaptively generate evaluation criteria for any instruction and produce multi-faceted evaluation, including both text-based and code-driven analyses. the refiner leverages the general llm's generalist evaluation capabilities to refine the analysis results produced by the analyzer and make the final judgment. this framework partially preserves the generalist evaluation pattern of the general model while enhancing the evaluation pattern in the fine-tuning dataset. ## experiments 4.1 implementation details to construct the composite analysis corpus, we prompt gpt-4o to generate evaluation questions for each instruction and collect text-based analysis. besides, we prompt claude-3.5-sonnet to generate python functions for code-driven objective analysis. we selected claude-3.5-sonnet due to its superior performance in code generation. we fine-tune qwen2.5-7b-instruct (qwen, 2025) on the corpus, creating a model we refer to as the analyzer for performing multi-faceted evaluations. we use the same model in a zero-shot setting as the refiner, with carefully crafted prompt templates. all generation in the main experiments is performed using greedy decoding by setting the temperature to 0. details are described in appendix a. ## case studies we show an example of a multi-faceted evaluation generated by arjudge in figure 5 . given an instruction and two responses, the analyzer first generates three evaluation questions and the corresponding multi-faceted analyses. the last question is analyzed by constructing a python function and assessing execution feedback to determine requirement completeness. then, the refiner reviews the preliminary analysis and refines it by reconsidering q1: does the response provide a clear, structured, and relevant first draft tailored to the "secretary of state"? q2: does the response maintain a professional tone and format suitable for a cover letter, and does it align with the instruction to discuss and edit based on the applicant's background? q3: does the response include a reference to each bullet point? ## conclusion this work proposes a novel evaluation framework, arjudge, which adaptively designs evaluation criteria and performs multi-faceted evaluation in both text and code. a new composite analysis corpus, designed for both criteria generation and multifaceted analysis, is developed to train arjudge. extensive experiments demonstrate the superiority and robustness of our framework across diverse evaluation benchmarks. notably, with code-driven analyses, arjudge gains strong evaluation capabilities for assessing instruction following. future studies can explore the effective use of more tools, such as a search engine, to improve evaluation honesty and mitigate hallucination.