Ranking Large Language Models without Ground Truth (2024)

Abstract: Evaluation and ranking of large language models (LLMs) has become an important problem with the proliferation of these models and their impact. Evaluation methods either require human responses which are expensive to acquire or use pairs of LLMs to evaluate each other which can be unreliable. In this paper, we provide a novel perspective where, given a dataset of prompts (viz. questions, instructions, etc.) and a set of LLMs, we rank them without access to any ground truth or reference responses. Inspired by real life where both an expert and a knowledgeable person can identify a novice our main idea is to consider triplets of models, where each one of them evaluates the other two, correctly identifying the worst model in the triplet with high probability. We also analyze our idea and provide sufficient conditions for it to succeed. Applying this idea repeatedly we propose two methods to rank LLMs. In experiments on different generative tasks (summarization, multiple-choice, and dialog), our methods reliably recover true rankings without reference data. This points to a viable low-resource mechanism for practical use.

## introduction recent advancement in llm capabilities have resulted in a significant challenge for assessing or measuring these capabilities. to benchmark llm performance, we need a set of input prompts, outputs from the llms for these prompts, and a metric that measures how good the llm performance is on its own or in comparison with other models. a common strategy in benchmarking is to use datasets that come with both input prompts and reference ("ground truth") responses in which case automated metrics can assess the model responses by comparing them with reference responses. these @xcite we see the intuition behind the triplet approach. the three models m1, m2 and m3 have accuracies of 80%, 60% and 40% respectively based on their responses to five prompts (green are correct responses and red incorrect) when compared with the ground truth which is unknown to us. our triplet approach ranks m3 as the worst model, since it is ranked as such by both m1 (only two answers match) and m2 (only one answer matches). this core idea (with slight variations) can be applied repeatedly to rank an arbitrary number of models as described by the algorithms in section 3. for these reasons, the idea of llm-as-a-judge is starting to be employed in several scenarios in order to serve as a more accurate proxy for human preferences, which cannot be captured easily by simple metrics. however, llm-as-a-judge may suffer biases @xcite and also assume the pre-selected choice of a trusted llm to serve as the judge. this paper aims to address the scenarios under which we cannot determine which llms can be trusted as judges. we consider the setup where we have n llms and a dataset of prompts, which could be questions like in q&a datasets or instructions such as in summarization or in other extractive/generative tasks. our goal then is to rank these n llms in the order of their performance on the chosen task. a priori, we do not assume anything about the quality of the models nor do we assume access to "ground truth" or reference responses that can act as the golden standard. the core idea of our method, which can be leveraged to design algorithms that rank models in practice, is illustrated in figure 1 . we consider three models at a time and let each one judge the rest. based on the cumulative judgements, we decide which is the worst model for that round. note that if we consider only two models, we will not be able to decide the worst model in the round with much trust, since there is no a priori assumption on model quality. the triplet of models idea stems from the real-life intuition that an expert in an area should be able distinguish between a knowledgeable person and a novice. the knowledgeable person should also be able to rank the novice lower than the expert. hence, both the expert and the knowledgeable person will rank the novice lower than the other and this consensus will help us identify the novice. this core idea is a part of both the greedy and full ranking methods we propose in this paper (algorithms 1 and 2). other strategies may also be possible that stem from this idea. being a judge in figure 1 does not necessarily imply only prompting the judge to choose between two responses, but one could also use nlp metrics (viz. rouge, bert score, nli models) to find closeness of the responses of other models to the judge's responses, and pick the closest response as the winner. in contrast to our approach, in recommender systems, items are recommended to users based on some notion of user preference, whereas here the llms (equivalently considered as items) rank each other without any other intervention or availability of partial ratings. as a part of this work, we analyze the triplet approach and provide conditions for it to succeed. we also discuss the time complexities of the proposed methods. we show that using summarization, multiple choice question-answering, and dialog as applications, our methods retrieve rankings reliably without any reference data. we exceedingly see the benefit of this approach in summarization and dialog, that is tasks where the responses are long texts, rather than single tokens such as in multiple choice, as other strategies such as ensembling of responses could be employed in the latter case. ## related work benchmarking benchmarking is heavily relied upon by the community to assess the performance of llms. it is viewed as one of the most important problems that need to be addressed with urgency, but it is also recognized that there are no "onesize-fits-all" solutions to this problem. in addition to performance benchmarking, there can be trust and safety benchmarking along different dimensions such as bias, toxicity, social stigma, stereotype, privacy, adversarial robustness, social norms, and machine ethics. there are also comprehensive benchmarks such as helm @xcite that include both performance and safety measures. the benchmarking results are conveniently hosted in locations such as the open llm leaderboard @xcite , helm leaderboard @xcite , trustllm leaderboard @xcite , and llm safety leaderboard @xcite . for tasks with binary labels such as classification, or multiple choice question answering (mcqa), usually the metric used is accuracy. @xcite discuss the necessity for incorporating uncertainty when measuring an llm's performance. for generative tasks, the metrics used can be based on comparison of the generated text to a reference text such as bleu @xcite , rouge @xcite , rouge-n @xcite , @xcite , and bert score @xcite . natural language inferencing (nli) benchmarks require human level understanding in order to assess given a premise statement whether it supports (entailment), contradicts or has no relationship (neutral) in relation to a hypothesis statement @xcite . libraries such as unitxt @xcite combine these metrics and many others in a customizable manner. computing accuracy or a related measure on human judgements or preferences is also a very common approach for benchmarking. human preference of model responses along the dimensions of helpfulness and harmlessness are collected in the hh-rlhf dataset @xcite . another significant dataset with quality labels and human preferences of model responses is openassistant conversations @xcite . human preference data for model responses to 80 multi-turn questions was collected in the mt-bench dataset @xcite . ## methods we now propose two methods based on the triplet idea to rank a given set of models when no groundtruth is available for the given dataset. the first one is a more efficient greedy approach, while the latter considers all triplets and performs an evolving weighted majority based ranking. ## experiments we test our methods on three cases: summarization task, multiple-choice, and dialog. as a basis of comparison we devise a new method based on the 'most common answer' (mca). the mca ranker parses outputs from all llms to determine the most common output which is then treated as the reference to rank the models. for contexts such as (single token) multiple choice q&a, this is possibly the preferred method as it is easy to aggregate responses and ensembling in such contexts can have state-of-the-art performance (mienye and @xcite . to measure effectiveness of the methods, we compare the estimated rankings with 'true' rankings derived using reference data. we use two metrics: rank-biased overlap (rbo) @xcite and mean average precision@k (map-k) to quantify how well the estimated rankings reflect the 'true' rankings. both measures span the unit interval, with 1.0 denoting perfect agreement. summarization: we test on two datasets: the cnn-daily mail (cnn/dm) summarization dataset @xcite and xsum @xcite . in both cases we gather responses from 40 llms for 3000 instances from helm @xcite . our evaluation functions, i.e. f in gtr and g in ftr, employ rouge scores which are widely used for summarization tasks and in helm. in particular, we consider the f-measure of bigrams. for the 'most-common answer', we use the top-k bigrams from all model responses and consider this to be the reference answer (@xmath0. for this case, we also compare with llm-as-ajudge approach using prometheus-2 @xcite , which is a state-of-the-art llm evaluator or judge model. true ranking, against which our estimated ranking is compared, is inferred by comparing model outputs to the reference response in the dataset. ## discussion given the proliferation of llms, and claims of superiority made by the developers, there needs to be a trustworthy mechanism to evaluate them. llm leaderboards are extremely beneficial for this purpose. however, it takes a lot of effort to collect reference data or judgements or preferences about the model responses, which only multiplies when we consider various domains and tasks which can also lead to benchmarks becoming obsolete. our proposed approaches can be seen as a first pass to substantially reduce this effort without imposing the need to have a trustworthy model for evaluation as demanded by llm-as-a-judge approaches. moreover, in addition to simply obtaining a ranking, the relative values of the reputation score for different llms in ftr could potentially be used to also gauge the performance gap between them. there are many interesting avenues to explore in the future. first, how we can incorporate and benefit from additional information such as a partial ordering between llms, which is reasonable to assume when some llms belong to a model family. the additional information could also be in the form of a few ground truth labels for the task at hand. in this case, there may be a wider set of possible methods such as those based on uncertainty or possibly even bayesian approaches. second, rather than triplets what if we consider larger sets to compare in each round. of course, for ftr the number of comparisons might scale exponentially with the set size so more efficient variants might have to be thought of. here it is possible that the variance in ranking the worse models might increase as a larger set of judges will produce their own ranking. moreover, asking each judge to rank a larger set of models might be more error prone. nonetheless, the bias-variance trade-off between different set sizes might be interesting to explore. third, more efficient variants than gtr may be worth exploring. for instance, in gtr the top two models at the end of the first round may be used to rank the rest resulting in a linear time variant. also methods from matrix completion could be explored where we obtain a few partial orderings running gtr or ftr for a few iterations. finally, our approach might find use in other contexts where, consolidation of responses into a single response is not easy such as: i) when evaluating non-expert human annotators on descriptive tasks. ii) in accelerated drug discovery type of applications where different algorithms might recommend different molecular structures that may be hard to consolidate or iii) even in multiagent systems where a protocol is needed to decide which agent to follow in a specific context.