Efficient LLM Comparative Assessment: A Product of Experts Framework for Pairwise Comparisons (2024)

Abstract: LLM-as-a-judge approaches are a practical and effective way of assessing a range of text tasks. However, when using pairwise comparisons to rank a set of candidates, the computational cost scales quadratically with the number of candidates, which has practical limitations. This paper introduces a Product of Expert (PoE) framework for efficient LLM Comparative Assessment. Here individual comparisons are considered experts that provide information on a pair’s score difference. The PoE framework combines the information from these experts to yield an expression that can be maximized with respect to the underlying set of candidates, and is highly flexible where any form of expert can be assumed. When Gaussian experts are used one can derive simple closed-form solutions for the optimal candidate ranking, as well as expressions for selecting which comparisons should be made to maximize the probability of this ranking. Our approach enables efficient comparative assessment, where by using only a small subset of the possible comparisons, one can generate score predictions that correlate well with human judgements. We evaluate the approach on multiple NLG tasks and demonstrate that our framework can yield considerable computational savings when performing pairwise comparative assessment. With many candidate texts, using as few as 2% of comparisons the PoE solution can achieve similar performance to when all comparisons are used.

## introduction the advent of instruction-following @xcite large language models (llms) @xcite has enabled systems to exhibit impressive zeroshot capabilities on a range of natural language processing (nlp) tasks. one such practical application is in natural language generation @xcite evaluation @xcite , where llms can be prompted to assess the quality of texts for particular attributes @xcite @xcite . a popular approach is llm comparative assessment, where pairwise comparisons are used to determine which of two texts is better @xcite @xcite . although using pairwise comparisons has been shown to better align with human preferences @xcite than llm scoring approaches @xcite , the set of all comparisons scales quadratically with the number of inputs, which may be impractical in real-world use cases. therefore, one may instead consider methods that only use a subset of comparisons to predict the scores, such that performance is maintained in computationally efficient settings. due to its applicability to sports, search and many other domains, the task of going from a subset of comparisons to a final ranking/scoring has been well-studied and extensively explored @xcite @xcite . however, in the majority of setups, the comparative decisions are binary (win/loss, although occasionally also win/loss/tie). llms, however, not only provide the outcome of the comparison but also additional information, such as the associated probability that a is better than b. despite this available information, current llm comparative works often leverage naive metrics such as win-ratio @xcite @xcite and average probability @xcite , with little analysis on how to maximally extract the information from the comparisons. this paper introduces a theoretical framework for viewing comparative assessment that enables practical scoring even in cases when the full set of comparisons is not used. we conceptualize the process as a product of experts (poe) @xcite , where each comparative decision is assumed to provide information on the quality difference between the two competing texts. the framework is highly flexible and can use any form of expert. by considering two forms of experts, namely 1) the gaussian distribution with linear assumptions and 2) an extension of the bradley-terry (bt) model for soft probabilities (motivated by looking at its limiting behaviour), we demonstrate that the poe framework for comparative assessment can achieve efficient and effective nlg assessment. with the gaussian expert, the framework yields a closed-form solution for the scores, which conveniently yields standard metrics when using the full set of comparisons. we demonstrate that our product of expert framework leads to significant performance boosts across models, datasets and assessment attributes, and even when using a fraction of the possible comparisons, can achieve high performance with minimal performance degradation from the full set. this paper makes several contributions. 1) we introduce the poe perspective of comparative assessment, a highly flexible theoretical framework which enables one to directly model the distribution of scores given a set of comparisons. 2) we propose two experts, a soft bradley-terry expert (by considering the limiting behaviour of bt) and a gaussian expert that has closed-form solutions and can be used to select the most informative comparisons. 3) we demonstrate practically that the poe solution yields significant computational savings and empirically show that convergence is reached significantly faster than when using other baseline approaches for several datasets. ## background and related work traditional/tailored nlg evaluation: initially, the outputs of nlg systems were evaluated against ground-truth human-annotated references, using n-gram overlap metrics @xcite @xcite or similarity metrics @xcite . for more fine-grained evaluation, later studies developed bespoke evaluators for particular task dimensions such as summary consistency @xcite @xcite or dialogue coherence @xcite . further extensions considered unified evaluators, which evaluate multiple independent attributes @xcite @xcite . a drawback with these traditional nlg evaluation approaches is that they typically are bespoke towards particular tasks and attributes and, therefore, cannot easily be extended to new domains. llm-based nlg evaluation: given the impressive instruction-following @xcite capabilities of llms such as gpt-4 @xcite and open-sourced variants @xcite , recent works have studied leveraging these llms for general zero-shot nlg evaluation. methods include gptscore @xcite , which computes the llm likelihood of generating the response, and llm-as-a-judge approaches @xcite that prompt models to provide scores @xcite @xcite or use pairwise comparisons to determine which of two responses is better @xcite . ## a product of experts perspective of comparative assessment let x 1:@xmath0:@xmath1. given a set of k pairwise comparisons, c 1:k , the objective is to determine a predicted set of scores, ŝ1:n , that are close to the true scores, s * 1:n . ## conclusions comparative assessment using llms has been shown to be effective for text assessment. this paper investigates framing the scoring process within a product of experts framework, where the comparison information (including model confidence) can be easily combined to determine a set of scores that effectively capture text quality. this enables comparative assessment to not suffer from slow convergence rates, as now only a subset of the possible comparisons is used to predict the scores, but maintain the performance from when using the full set of comparisons. further, using gaussian experts yields a closed-form solution and provides a basis for deriving a greedy-optimal set of comparisons. the paper demonstrated the effectiveness of the approach on multiple different standard nlg evaluation datasets, such as summeval and topicalchat, as well as for large datasets where @xmath2, which led to substantial savings in computation against standard methods.