On Unsupervised Comparisons of Large Language Models
Authors: Yinan Liu, Tengfei Ma, Weimin Lyu, Xinyu Dong, Xiaojie Fan, Rachel Wong, Richard N Rosenthal, Henry Lam, Fusheng Wang


Reviewer paper summaries: This long paper propose a method to characterize the difference in behaviour between two LLMs. This is presented as an alternative to evaluating LLMs on human-labelled benchmarks, which are costly to develop. It also provides a more targeted form of assessment, e.g. a user can compare *how* two models differ on a specific set of questions. The authors also claim this can be used to identify relevant data for fine-tuning models, and may eventually be used to predict approximate rankings based on distances between pairs of models. The main contribution is the framework to measure and identify behavioural differences between LLMs, by finding questions that produce the most different answers, which the authors term "distinguisher questions". This is inspired by distinguisher sets in the domain of cryptography. The method selects topics for the domain under investigation (given an existing topic tree), and samples questions for each topic. The selection of topics is optimized to maximize the difference between two models' outputs for the sampled questions, using Bayesian optimization. Their framework also includes a method to compute distances between models, based on earth mover's distance. The methods also include new metric embedding techniques to embed earth mover's distance in Euclidean space, which is required for standard Bayesian optimization. The framework assumes a topic-tree already exists for the domain under investigation -- the authors claim this is usually the case, but this is unsupported. It also assumes that there exists "a reasonable mechanism available for generating questions related to [a given topic] via some prompt engineering on LLMs", which is fair enough, but still requires developing a well performing prompt for question generation in the target domain. Experiments are conducted to see if their topic selection method is better than random (it is, but it is hard to know what to make of the differences in distance), and to see if model distances can be used to predict a ranking.
Summary of strengths: - It presents a novel method to characterize the behavioural differences between two LLMs, by automatically identifying a subset of topics for which sampled questions will produce very different answers from both models. This may be of some use for model assessment.
- The optimization problem and their solution to it may be of some interest.
Summary of weaknesses: - The method assumes there exists a tree topic for the domain under investigation. It also assumes a method to generate questions, which may require prompt engineering in the target domain. 
- The method is limited to pairwise comparisons between two models. Extending this to more than two models would require exponentially many pairwise comparisons, and require lots of manual analysis of the different answers. 
- The experiments did not provide convincing evidence that the method is actually finding a good subset of topics for which to generate questions that will then receive very different answers from two LLMs. They just show the value of the distance they define for different pairs of models, different numbers of topics, and different methods to select the topics, and it is hard to know what to make of the quantitative gains provided by their method, since this is a new distance, whose absolute values aren't really meaningful. They also show one example each of distinguisher and non-distinguiser questions found by their method, but the difference in answers for the distinguisher question only indicates that one model provides a more generic answer than the other. This doesn't seem like very useful insight into model behaviours. Overall, I come away with the impression that this method is not all that useful. Finally, the experiment on predicting a ranking based on pairwise distances only shows that the resulting ranking of models "appears consistent with general perception", without any further support, and despite the fact that LLaMA3 is much lower in the ranking than LLaMA2. So, in general, a lack of support for their conclusions is a weakness of this paper. 
- The paper is heavy on math and theory, light on experimental/applied findings. Experiments are limited to a single domain. Only one example of a "distinguisher question" identified by their method is shown, and compared to a non-distinguisher. These could have been cherry-picked. The paper would be more suitable for *ACL venues if some of the derivations were moved to the appendix, and more convincing evidence of the utility of the method was presented.
- The term "sigifificant difference" is used repeatedly, but no actual significance/hypothesis testing conducted. 
- Some parts of the paper lack clarity. I have listed several examples below. 
- There are many typos and style issues.
Comments, suggestions and typos: - Line 4: how does automatic evaluation take years to conduct? Do you mean developing the gold standard benchmarks?
- Line 6: how is LLM evaluation a supervised ranking problem? This does not make sense to me. 
- Line 27: "its" should be "their", or "algorithms" should be singular. 
- Line 41: benchmarks have been developed for these domains. It's not as if you have to create a new one each time you want to evaluate a new LLM in a given domain. 
- Line 45: again, it seems odd to call LLM evaluation a "supervised approach". This term is usually applied to learning tasks, not evaluation. Maybe replace with something like human annotation approach. 
- Line 51-53: this sentence seems misleading to me: you can only conclude that the models behave similarly, whereas a labelled benchmark can tell you about the quality of the model outputs. 
- Line 56-57: "previously suggested" by what, exactly? This seems vague.
- Line 61: "diff" should be "differ", presumably. 
- Line 103: what is the word "wire" doing here? Is this a typo? What could "wire correlations" mean?
- Line 110: "blackboxes" should be "black box".
- Line 118: "Simanese" should be "Siamese". 
- Lines 129 and following: "These evaluations" should be something like "The evaluated tasks/properties", "includes" should be "include", "multilingual" should be "multilinguality", and "factually" should be "factuality". 
- Line 135: it is not always costly to compile question and answer pairs. Some can be found in the wild. 
- Line 138: does your framework actually assess relevance?
- Line 139: "...aspects of the answers" in not grammatical.
- Line 140: This paragraph on reinforcement learning seems completely irrelevant. And I don't see how a set of answers can be considered an environment.
- Line 161: what do you mean by "impossible results"?
- Caption of Fig. 1: "intuitive" should be "intuitively". 
- Line 310: what does "GP" mean? Gaussian process?
- Line 433: add "the" before difference. 
- Line 445: add "the" before WHO.
- Caption of Table 2: add "the" before "objective".
- Line 475: "standerdize" should be "standardize".
- Line 481: avoid using the term "significantly" when no significance/hypothesis testing was conducted.
Overall assessment scores: 2