A lign B ench: Benchmarking C hinese Alignment of Large Language Models (2024)

Abstract: Alignment has become a critical step for instruction-tuned Large Language Models (LLMs) to become helpful assistants. However, effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. To fill in this gap, we introduce AlignBench, a comprehensive multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese. We tailor a human-in-the-loop data curation pipeline, containing 8 main categories, 683 real-scenario rooted queries and corresponding human verified references.To ensure references’ correctness, each knowledge-intensive query is accompanied with evidences collected from reliable webpages (including the url and quotation) by our annotators.For automatic evaluation, our benchmark employs a rule-calibrated multi-dimensional LLM-as-Judge (CITATION) with Chain-of-Thought to generate explanations and final ratings as evaluations, ensuring high reliability and interpretability.All evaluation codes and data are publicly available at https://github.com/THUDM/AlignBench

## introduction large language models (llms) @xcite @xcite @xcite have experienced a surge in development thanks to popular products such as chatgpt @xcite . during the period, alignment @xcite , including supervised fine-tuning (sft), reinforcement learning from human feedback (rlhf), and related techniques, has been justified as a key strategy to endow pre-trained llms (which can hardly follow instructions) with strong grasping of human intentions and preferences. after training, aligned llms have not only mastered a wide array of established nlp tasks @xcite but also versatile language-grounded missions @xcite @xcite . as a result, llms have paced a firm step towards practical applications in the wild. meanwhile, reliably benchmarking the broad and strong competence of llms has also become a significant challenge. in english, there have been mmlu @xcite , @xcite , and helm @xcite ; in chinese, there are c-eval @xcite and cmmlu @xcite . however, prior arts hardly examine aligned llms' fulfillment of user intention and human preference in real-world conditions, and even fall short to tell the difference between aligned and base llms. consequently, dedicated benchmarks are crucial for development and meaningful comparisons of aligned llms. nevertheless, designing a comprehensive and reliable benchmark for llm alignment is nontrival. an alignment benchmark should meet several important requirements, which correspond to the unique strengths of llms and their applications for users: there have been recent attempts to introduce llm-as-judge @xcite for evaluating the general alignment of llms. al-pacaeval @xcite compares target llm's replies against text-davinci-003's, but has been shown unstable and uninterpretable due to its direct and pairwise scoring. mt-bench @xcite harnesses point-wise scoring with chain-of-thought (cot) @xcite explanations for better accuracy and transparency. however, it employs only 80 test samples and a scoring prompt that judges queries of different tasks and domains uniformly. lastly, both benchmarks are designed only in english and cannot well reflect the level of alignment of many emerging chinese llms. in light of all mentioned issues, in this work we present alignbench, a comprehensive multidimensional benchmark for evaluating llms' alignment in chinese. based on observations from an online llm service (anonymous due to review policy), we set up a semi-automatic data curation pipeline with human in the loop to create high-quality queries to construct alignbench. alignbench summarizes a taxonomy comprising 8 major categories of queries (cf. figure 1 ) to comprehensively cover and align with real-scenario usages. in order to make the judge model generate objective and fair evaluations, each sample is accompanied with a human-corrected llmgenerated reference. to ensure reference correctness for knowledge intensive queries (which accounts for 66.5%), we ask annotators to search on the web, provide evidences including urls and quotations, and finally synthesize references. to enhance the automation of the evaluation, similar to mt-bench @xcite , align-bench distinctly leverages gpt-4 (openai, 2023) as the major model evaluator in its development, which serves to discern the data samples and evaluate by referenced point-wise scoring with cot. differently, alignbench further highlights strategies of rules-calibration and task-specific multidimensional judgement in the scoring. our experiments demonstrate that these strategies contribute to alignbench's better consistency with human judgement and better explanation quality. based on alignbench, we evaluate 17 popular api-based or open-sourced llms that support chinese, providing detailed comparisons of these llms across fine-grained capabilities on chinese alignment. in summary, the contributions of our work are: ## methods to effectively evaluate the quality of responses, alignbench employs gpt-4 (openai, 2023) as a major evaluator to analyze and subsequently grade the responses following adopted practices @xcite @xcite . however, a significant designing space still exists regarding prompting, score calibration, critique explanability, and evaluation dimensions, which have been hardly explored. therefore, in alignbench we design a novel rule-calibrated multi-dimensional point-wise llmas-judge method. the detailed prompts are in appendix a.2 and an example is displayed in fig 3 . point-wise grading & chain-of-thought @xcite . when llm-as-judge is leveraged, two grading methods have been previously implemented: point-wise @xcite or pairwise grading @xcite . nevertheless, previous study has indicated that the point-wise grading possessed comparable agreement with humans than the pair-wise grading, which suffers from position bias @xcite . additionally, considering the evaluating efficiency, compared to pair-wise grading's quadratic number of comparisons, the point-wise grading has advantages in terms of expenses and time. therefore alignbench adopts point-wise grading either. during the evaluation, the inputs are the query, the model's response, and a human-curated reference answer, and the output is an multi-dimensional analytical explanation and a final rating, ranging from 1 to 10. as the task of grading involves complex reasoning, introducing chain-of-thought in the scoring process has also been proved useful to augment both the score reliability and interpretability @xcite . specifically, gpt-4 is instructed to generate explanations from multiple dimensions before providing a final grade on a scale of 1 to 10. rule-calibrated referencing. given that many of the questions in alignbench are of significant complexity and difficulty even for gpt-4, we provide a high-quality reference answer, which is primarily generated by gpt-4 and modified by human annotators to ensure its correctness and improve its quality. to guide the evaluator to compare the answer with the reference and generate more controllable scores, we provided detailed grading rules elaborating the relationship between score intervals and the answer's quality compared to the reference. additionally, we set the reference answer to score 8 as a reference scoring pivot. we plot the cumulative distribution of human judge, general judge and rule-calibrated judge in figure 2 to show that rule-calibration judge has a narrower gap to human evaluation's cumulative distribution. typically, rule-calibrated judge scores much fewer top scores (9 and 10) than general judge, aligned with human scoring habits and therefore enhance the discrimination of alignbench. multi-dimensional analysis. as tasks vary in their nature and characteristics, applying the same evaluation criteria to all tasks would be unjust. for instance, writing tasks should prioritize creativity, whereas logical reasoning tasks primarily require logical coherence. as a solution, we propose a multi-dimensional scoring approach to evaluate llms' responses, tailoring the evaluation to the specific task at hand, promising a more comprehensive and organized explanation. specifically, we set up different evaluation dimensions based on different types of questions, as shown in ## human evaluation on alignbench to justify the rule-calibrated multi-dimensional point-wise llm-as-judge method we design for alignbench, we conduct extensive human evaluation over alignbench's selected queries. we especially focus on two aspects: the method's agreement with human judging, and the method's critique quality for more human-interpretable results. ## alignbench: benchmarking results based on the validness of alignbench's llmas-judge scoring, we systematically benchmark a wide array of llms on chinese alignment with the help of alignbench. we use gpt-4-0613 (ope-nai, 2023) as the judge model to evaluate model performances. main results. results are shown in ## conclusion we introduce alignbench, a comprehensive multi-dimensional benchmark for evaluating llms' alignment in chinese. a sustainable humanin-the-loop data curation pipeline and a better llm-as-judge method has been tailored to enable alignbench's high-quality automatic evaluation of llms' chinese alignment. benchmarking results of 17 chinese-supported llms are reported.