Benchmarking LLMs' Judgments with No Gold Standard

## Abstract

Abstract. We introduce the GEM (Generative Estimator for Mutual Information), an evaluation metric for assessing language generation by Large Language Models (LLMs), particularly in generating informative judgments, without the need for a gold standard reference. GEM broadens the scenarios where we can benchmark LLM generation performance-from traditional ones, like machine translation and summarization, where gold standard references are readily available, to subjective tasks without clear gold standards, such as academic peer review. GEM uses a generative model to estimate mutual information between candidate and reference responses, without requiring the reference to be a gold standard. In experiments on a human-annotated dataset, GEM demonstrates competitive correlations with human scores compared to the state-of-the-art GPT-4o Examiner, and outperforms all other baselines. Additionally, GEM is more robust against strategic manipulations, such as rephrasing or elongation, which can artificially inflate scores under a GPT-4o Examiner. We also present GRE-bench (Generating Review Evaluation Benchmark) which evaluates LLMs based on how well they can generate high-quality peer reviews for academic research papers. Because GRE-bench is based upon GEM, it inherits its robustness properties. Additionally, GRE-bench circumvents data contamination problems (or data leakage) by using the continuous influx of new open-access research papers and peer reviews each year. We show GRE-bench results of various popular LLMs on their peer review capabilities using the ICLR2023 dataset.

## 1. Introduction

High-quality and reliable Large Language Model (LLM) benchmarks can effectively guide research, encourage innovation, monitor their advancement, and inform users of which model to use for their purpose. The importance of the final goal is underscored by the over 900k models currently available on Hugging Face, an online platform for open-source LLMs(^1^11https://huggingface.co/docs/hub/en/index). Various benchmarks are proposed for evaluating LLMs’ ability in different aspects, including ARC (Chollet, ), HellaSwag (Zellers et al., ), Massive Multitask Language Understanding (MMLU) (Hendrycks et al., ), MT Bench (Zheng et al., ), GSM8K (Cobbe et al., ), TruthfulQA (Lin et al., ), Natural Questions (Kwiatkowski et al., ), etc.

Most benchmarks are based on multiple-choice questions or other questions with objective gold standard answers, since it is easy to verify the LLMs’ outputs. While they provide valuable evaluation for LLMs, open-ended tasks, e.g. providing judgment about a research paper, encompass a broader array of skills and require both objective and subjective reasoning. In addition, concern has been raised about data contamination (also called data leakage), where the training data contains information about the tasks in the benchmarks, as the LLMs are pre-trained on massive online data, which is also the source of some of the benchmark tasks. LLMs can show unreliably good performance due to data contamination (Sainz et al., ; Golchin and Surdeanu, ; Oren et al., ). In contrast, with open-ended questions, LLMs can be asked to provide judgments about newly created content, e.g. the latest academic papers, for which LLMs have yet to index human evaluations or responses.

However, it is not clear how to automate the evaluation of subjective response quality. An added challenge is that there is no gold standard quality response with which to compare.

We would like an evaluator to have two properties. First, it should be accurate and be sensitive to the semantic content response. As current LLMs have already shown strong language ability, we want to focus on evaluating the semantic informativeness of candidate responses instead of their syntax or style. Second, the evaluator should be manipulation-resistant-we should not be able to manipulate a response in a trivial fashion to consistently increase evaluation scores. This is important because otherwise one cannot determine whether a high evaluation score indicates that the LLM output a high-quality response or merely results from manipulation designed to achieve an artificially inflated score.

Given these gaps in previous research, our research question is: Can we develop accurate, manipulation-resistant, and automated evaluation metrics for textual responses with no gold standard reference to compare with?

A straightforward method may be to use another LLM as an oracle examiner directly to provide the evaluation, which has shown effectiveness in evaluating open-response QA (Bai et al., ), chatbot (Zheng et al., ), etc. However,
LLM examiners are susceptible to certain manipulations. In our experiment, elongating all responses by adding the same fixed sentences can significantly increase the GPT-4o LM examiner’s score. Interestingly, previous research has shown that even human evaluations can be manipulated by adding meaningless text (Goldberg et al., ).
Meanwhile, other automated natural language generation (NLG) evaluation metrics, including BLEU (Papineni et al., ), ROUGE (Lin, ), BERTScore (Zhang et al., ), BARTScore (Yuan et al., ) and GPTScore (Fu et al., ), rely on comparison with a gold standard reference.

### 1.1. Our Contributions

We propose the Generative Estimator for Mutual information (GEM) which uses the estimated Shannon mutual information (MI) between a set of candidate responses and a set of peer reference responses, which need not be of gold standard quality. The mutual information measures the amount of information revealed about the reference responses by obtaining the candidate responses. By artificially removing stylistic and syntactical information, our approach measures how much semantic information the candidate responses can reveal about the reference responses, which is related to the concept of semantic coverage (Yuan et al., ; Nenkova and Passonneau, ). We additionally propose a variant of our method,
Generative Estimator for Mutual Information with Synopsis (GEM-S), which estimates the mutual information conditional on a synopsis of the task, e.g. the abstract of the paper. This prevents a candidate response from receiving a high score based solely on superficial information. Consequently, the score emphasizes the *additional* semantic information gained from the candidate responses.

For implementation, we utilize a generative language model to estimate the conditional distribution between two text responses. Thus, the GEM can be categorized as an LLM-probability-based metric, with techniques similar to BARTScore (Yuan et al., ) and GPTScore (Fu et al., ), from which our metric inherits the effectiveness of evaluating objective tasks with a gold standard.

Our work creates a bridge between the information theoretical framework in the literature of information elicitation without verification (Kong and Schoenebeck, ; Lu et al., ) and the NLG evaluation problem. Though GEM and GEM-S resemble the GPPM and GSPPM mechanisms in Lu et al. (), with manipulation resistance aligned to their incentive compatibility, we make a necessary change to make the score more suitable for the NLG evaluation problem. Specifically, for incentive compatibility, the score can rely solely on accuracy of predicting the peer reference, while we use mutual information to capture the gain in that accuracy for evaluation purposes.

##### Results Overview

The results of experiments validate GEM’s accuracy and resistance to manipulation, and compare it with many different NLG evaluation metrics.

- 1.
Positive Correlation with Human Annotation (Section 4.1) On a human-annotated dataset, the GEM metrics, especially GEM-S, achieve a significant positive correlation with human-labeled quality scores and demonstrate competitive performance with GPT4-o Examiner while outperforming all other baseline metrics.
- 2.
Better Sensitivity to Degradations (Section 4.2) Compared to various baseline metrics, GEM and GEM-S are the only metrics that demonstrate significant sensitivity to all semantic degradations in our experiment, by effectively penalizing degraded responses.
- 3.
Better Robustness against Manipulations (Section 4.3) GEM and GEM-S are the only metrics that exhibit no significant score increases after meaningless elongation (Figure 4) and GPT-4o/Llama-3.1 rephrase, whereas LMExaminer show vulnerabilities by significantly increasing scores after all manipulations.

Building on GEM and GEM-S, we introduce the GRE-bench (Generating Review Evaluation Benchmark) to evaluate LLMs’ peer review capabilities with data from open-access research papers and peer reviews. With the continuous influx of new data each year, GRE-bench can mitigate data contamination issues. In addition, since GRE-bench is based on GEM or GEM-S, it inherits their nice properties demonstrated in the experiments above.

We run GRE-bench for various state-of-the-art LLMs on the ICLR 2023 dataset, and present the results in Section 5. We find a strong correlation between parameter size and GRE-bench score within LLM families, which further validates the effectiveness of GEM and GEM-S.

### 1.2. Related Work

##### LLMs in academic peer review.

Given the success of LLMs, they have the potential to assist with peer reviews when used appropriately (Robertson, ; Kuznetsov et al., ). Liang et al. () use a survey to evaluate GPT-4-generated reviews, and find that the GPT-generated reviews are thought helpful by more than 50% of participants. However, there have been concerns regarding issues such as hallucinations (Donker, ) and inconsistent performance (Liu and Shah, ). While our study uses the peer review scenario to evaluate LLM capabilities, it also provides a quantitative comparison for the effectiveness of various LLMs in generating informative reviews.

##### Information Elicitation.

Both information elicitation and machine learning employ a principal-agent model (Ali and Silvey, ). Specifically, the principal aims to elicit information from the agent, e.g. the probability of rain tomorrow. To incentivize the agent to provide truthful and informative reports, payment mechanisms are employed that reward truthful and informative reports more than untruthful or uninformative ones.

When the information is verifiable, e.g., we will eventually know whether it rains tomorrow, proper scoring rules (Good, ; Brier, ; Hendrickson and Buehler, ) can be applied, which are similar to the loss functions in supervised learning. When the information is subjective and unverifiable, Miller et al. () propose the peer prediction mechanism, suggesting rewarding data by how well it predicts a peer’s subjective report which is judged by a proper scoring rule. Prelec () propose the Bayesian Truth Serum mechanism. Kong and Schoenebeck () propose an information-theoretic framework, suggesting paying the agents according to the f-mutual information between their reports. The framework provides a unified theoretical view of previous mechanisms in Dasgupta and Ghosh (), and Prelec (). Lu et al. () first generalize the peer prediction to elicit informative textual responses with LLMs, providing the foundation for our GEM metrics.

Most information elicitation mechanisms theoretically ensure that truthful and informative reports yield higher expected scores. However, high ex-post correlation between scores and report qualities is critical for practical application, as highlighted by Burrell and Schoenebeck (); Xu et al. (). In our study, to validate GEM’s effectiveness in evaluating textual reports, we measure both the ex-post correlation with human-annotated quality scores and expected score change after several degradation and manipulation strategies.

##### Estimation of Mutual Information.

Kong and Schoenebeck () bridges information elicitation without verification and co-training, by formulating an optimization problem that maximizes the mutual information between two Bayesian predictors with two data sources. Cao et al. (); Xu et al. () apply this information theoretical approach in co-training and de-noising respectively. Concurrently and independently, Belghazi et al. () and Hjelm et al. () propose neural network-based methods for estimating mutual information, which can be used as an optimization goal in unsupervised and self-supervised learning settings.

## 2. Preliminaries

In this section, we first introduce our model for benchmarking LLMs’ judgments with no gold standard and then present Shannon mutual information (MI) and pointwise mutual information (PMI) as an unbiased estimator of MI.

##### Model

We consider a scenario with a candidate under evaluation and a peer reference working on the same tasks. We denote $\mathcal{W}$ as the space of all tasks, with an underlying distribution over it, denoted as $\Delta\mathcal{W}$. Given a sample task $W\sim\Delta\mathcal{W}$, both the candidate and the reference generate a response, represented by the random variable $X$ and $Y$ respectively. The response space for both $X$ and $Y$ is denoted by $\Sigma$. The tuple $(W,X,Y)$ follows an underlying distribution $\mathcal{D}:=\Delta(\mathcal{W}\times\Sigma^{2})$. Note that the reference’s responses do not need to be gold standard. We assume that $X$ and $Y$ are independent conditioning on the task $W$.

Figure: Figure 1. Our model
Refer to caption: extracted/5992515/figures/model.png

Empirically, in a dataset $D$, we have a list of sampled tasks $\mathbf{w}=\{w_{1},\cdots,w_{n}\}$, and a list of corresponding reference responses $\mathbf{y}=\{y_{1},\cdots,y_{n}\}$. Let the candidate generate a list of responses $\mathbf{x}=\{x_{1},\cdots,x_{n}\}$ to the corresponding tasks. We adopt a common assumption that all the samples $(w_{i},x_{i},y_{i})$ are generated i.i.d. following the underlying distribution $\mathcal{D}$.

##### Benchmarking the candidate

The evaluation metric $f:\mathcal{W}\times\Sigma^{2}\rightarrow\mathbb{R}$ maps a tuple $(w,x,y)$ to a score $S$. Note that some evaluation metrics only utilize one or two of the items in the tuple. If the evaluation metric utilizes a language model to compute the score, we call the model used as the evaluation-LM. Different evaluation-LMs may lead to different evaluation results. A nice property of our GRE-bench is the consistency over evaluation-LMs, i.e., using a smaller evaluation-LM will not significantly change the evaluation results, we will discuss it in Section 5. Given a dataset $D$ and an evaluation metric $f$, we can estimate $\mathbb{E}_{(W,X,Y)\sim\mathcal{D}}[f(W,X,Y)]$, by using the average of the evaluation metrics over all samples, which is named the benchmark.

##### Shannon Mutual Information

Ideally, we aim to score the candidate according to the informativeness of her responses, thus, Shannon Mutual Information (MI) (Shannon, ) is a suitable metric. The MI between two random variables, $X$ and $Y$, provides a quantitative measure of the information shared between them. Specifically, it assesses how much knowing one variable can inform us about the other. This is formally defined as:
$\operatorname{I}(X;Y)=\sum_{x,y}\Pr[X=x,Y=y]\log\frac{\Pr[X=x,Y=y]}{\Pr[X=x]
\Pr[Y=y]}.$ The conditional mutual information $I(X;Y\mid Z)$ measures the mutual information between $X$ and $Y$, conditioned on a random variable $Z$:

$$ $\operatorname{I}(X;Y|Z)=\sum_{z}\Pr[Z=z]\sum_{x,y}\Pr[X=x,Y=y|Z=z]\log\frac{ \Pr[X=x,Y=y|Z=z]}{\Pr[X=x|Z=z]\Pr[Y=y|Z=z]}.$ $$

In an academic peer review scenario, $Z$ can be some superficial information of a paper (e.g. the abstract), the conditional MI measures how much information is revealed beyond the abstract and
emphasizes the MI gained from the semantics judgment.

##### Point-wise Mutual Information as an Estimator

To estimate the MI with samples, we employ Point-wise Mutual Information (PMI) (Fano and Hawkins, ; Church and Hanks, ), an unbiased estimator of the MI. Formally, the PMI between realization $x$ and $y$ is defined as

$$ $\displaystyle\operatorname{PMI}(x;y):=$ $\displaystyle\log\frac{\Pr[X=x,Y=y]}{\Pr[X=x]\Pr[Y=y]}=\log\Pr[Y=y\mid X=x]- \log\Pr[Y=y].$ $$

Similarly, the PMI between $x$ and $y$ conditional on $z$ is defined as

$$ $\displaystyle\operatorname{PMI}(x;y\mid z)=$ $\displaystyle\log\Pr[Y=y\mid X=x,Z=z]-\log\Pr[Y=y\mid Z=z].$ $$

## 3. Generative Estimator for Mutual Information (GEM)

In this section, we propose the Generative Estimator for Mutual Information (GEM), prove its theoretical effectiveness even if the reference’s responses are not gold standard under certain assumptions, and discuss its empirical practice.

As discussed in Section 2, we use the mean of sample PMIs as an unbiased estimator of the MI. Given a sampled tuple with a task $w_{i}$, a candidate response $x_{i}$ and a reference response $y_{i}$, we aim to compute $\operatorname{PMI}(x_{i};y_{i})=\log\Pr[Y=y\mid X=x]-\log\Pr[Y=y]$.

Following Yuan et al. (); Fu et al. (); Lu et al. (), we use a generative language model to estimate the probability distribution of the reference response conditional on the candidate’s output, denoted by $\Pr_{\operatorname{LLM}}[Y=y\mid X=x]$, and the marginal distribution, denoted by $\Pr_{\operatorname{LLM}}[Y=y]$.

Specifically, we use the LLMs’ inherent token prediction function. We tokenize $y$ as $y=y^{(1)}y^{(2)}\cdots y^{(|y|)}$, and then integrate $x$ into the prompt. Given any length-$k$ prefix $y^{(1)}y^{(2)}\cdots y^{(k)}$, we use the LLM to predict $\Pr_{\operatorname{LLM}}[Y^{(k+1)}=y^{(k+1)}\mid Y^{(1)}\cdots Y^{(k)}=y^{(1)}
\cdots y^{(k)},X=x]$. With Bayesian rule, by multiplying over $k$, we have

$$ $\Pr_{\operatorname{LLM}}[Y=y\mid X=x]=\prod_{k=0}^{|y|}\Pr_{\operatorname{LLM} }[Y^{(k+1)}=y^{(k+1)}\mid Y^{(1)}\cdots Y^{(k)}=y^{(1)}\cdots y^{(k)},X=x].$ $$

Similarly, we can estimate the marginal distribution $\log\Pr_{\operatorname{LLM}}[Y=y]$. Thus, we have the estimated PMI, denoted as $\widehat{\operatorname{PMI}}(x_{i};y_{i})=\log\Pr_{\operatorname{LLM}}[Y=y\mid
X
=x]-\log\Pr_{\operatorname{LLM}}[Y=y]$, and consequently, the estimated MI over dataset $D$ is
$\widehat{I}(X;Y)=\frac{1}{n}\sum_{i=1}^{n}\widehat{\operatorname{PMI}}(x_{i};y
_{i}).$

### 3.1. Theoretical Guarantees: Benchmarking LLMs with No Gold Standard

We now show that even when the reference is not gold standard, the estimated MI can still be a benchmark where better candidates achieve higher scores theoretically. We adopt a widely-used model in decision-making theory (Blackwell, , ). The candidate’s response follows an information structure, a mapping $\sigma:\mathcal{W}\rightarrow\Delta\Sigma$, where $\Sigma$ is the response space. It represents the distribution of responses conditional on the task.

Within this model, Blackwell’s order (Blackwell, , ) provides a partial order of the informativeness of information structures. Consider two candidates, H and L, whose responses $X_{H}$ and $X_{L}$ follow information structures $\sigma_{H}$ and $\sigma_{L}$ respectively. Information structure $\sigma_{H}$ is more informative than $\sigma_{L}$ in the sense of Blackwell order if there exists a stochastic mapping $\Gamma$, such that $\sigma_{L}=\Gamma\sigma_{H}$. Intuitively, the information structure with a lower Blackwell order is more noisy, and thus, can be regarded as lower quality.

We now show that higher Blackwell order information leads to (approximately) higher GEM scores when the LLM can provide a “good” estimation. Formally, we have

###### Proposition 3.1.

When the KL-divergence(^2^22The KL-divergence between two distributions over the same probability space is $D_{\text{KL}}(P\|Q)=\sum_{x}P(x)\log\left(P(x)/Q(x)\right).$) between the LLM estimated distribution and the underlying distribution satisfies

$$ $D_{KL}\left[\Pr_{\operatorname{LLM}}[Y=\cdot\mid X=x_{H}]\Big{\|}\Pr[Y=\cdot \mid X_{H}=x_{H}]\right]<\epsilon,~{}\forall x_{H}.$ $$

For the two candidates $H$ and $L$ discussed above, the information structure of $H$ Blackwell dominates $L$’s, when the size of dataset $n$ goes to infinity, we have
$\widehat{I}(X_{H};Y)\geq\widehat{I}(X_{L};Y)-\epsilon$.

The proof is provided in Appendix B, following previous information elicitation literature (Lu et al., ). It mainly relies on the data processing inequality (Shannon, ; Thomas and Joy, ). Additionally, we provide a similar proposition for conditional MI in Appendix B.

We highlight that while this theoretical result requires a strong assumption, it gives intuitions of why and how our GEM scores could work. In the next several sections, we present our experiment validating the effectiveness of GEM with real data.

###### Proposition 3.1.

### 3.2. Filtering Out Shortcuts

Textual data is inherently high-dimensional, encompassing various aspects. For example, peer reviews may have a hierarchical information structure, including language style, the topic of the paper, and critical judgments, as shown in Figure 2. While each of these dimensions influences LLM predictions and consequently contributes to the mutual information, they are not equally important when assessing the quality of LLM-generated responses, particularly in the context of peer reviews.

Empirically, the correlation between superficial information can act as “shortcuts” (Geirhos et al., ) confounding LLM predictions, cause unintended bias in the estimated mutual information, and consequently distort the evaluation metric towards superficial correlations rather than thorough judgments. Since we focus on the semantic quality of these responses, we aim to filter out dimensions like language styles that could skew the evaluation metric by offering “shortcuts” in LLM predictions.

Figure: Figure 2. Example of Hierarchical Information Structure in Peer Reviews
Refer to caption: extracted/5992515/figures/hierarchy.png

##### Preprocessing

A straightforward yet effective method proposed by Lu et al. (), is preprocessing responses using a specific LLM to rephrase and summarize the content. This preprocessing step standardizes language style and eliminates superficial information, thereby reducing the influence of irrelevant dimensions. We denote the preprocessed versions of responses $x_{i}$ and $y_{i}$ as $\hat{x_{i}}$ and $\hat{y_{i}}$ respectively.

Figure: Figure 3. An Overview of Our Generative Estimator for Mutual Information
Refer to caption: extracted/5992515/figures/GEMI.png

We have now outlined all steps of our Generative Estimator for Mutual Information (GEM) framework, as illustrated in Figure 3. The GEM metric between a candidate response $x_{i}$ and a reference response $y_{i}$ is defined as follows:

$$ $\operatorname{GEM}(x_{i},y_{i}):=\widehat{\operatorname{PMI}}(\hat{x_{i}},\hat {y_{i}}).$ $$

By taking the arithmetic average of all $\operatorname{GEM}(x_{i},y_{i})$ over the dataset $D$, we have an estimated mutual information between random variables $X$ and $Y$, as shown in Figure 3.

For comparison, we also introduce the GEM-raw metric calculated without preprocessing as follows: $\operatorname{GEM-raw}(x_{i},y_{i}):=\widehat{\operatorname{PMI}}(x_{i},y_{i})$, which serves as a baseline.

##### GEM-S: conditioning out the synopsis

Kong and Schoenebeck () suggest surface-level information can inflate mutual information through simple correlations, reducing the mutual information’s effectiveness in reflecting semantic informativeness, especially when the responses have a hierarchical information structure as shown in Figure 2. As some superficial information can be contained in a synopsis of the task, e.g. the abstract of the paper to be reviewed, conditional mutual information can be used to distill the semantic informativeness (Lu et al., ).

The GEM-S metric is designed to mitigate this issue by conditioning out the synopsis of the task, denoted as $\theta(w_{i})$. The GEM-S metric is formally defined as:

$$ $\operatorname{GEM-S}(\theta(w_{i}),x_{i},y_{i})=\widehat{\operatorname{PMI}}( \hat{x_{i}},\hat{y_{i}}\mid\theta(w_{i})).$ $$

With the conditional formulation above, only the additional, non-trivial mutual information is measured, thereby offering a more accurate assessment of the candidate’s performance in providing information beyond superficial content.

Moreover, we can adjust the granularity of the synopsis to make the GEM-S metric focus on evaluating different aspects of the candidate LLMs’ ability. In the example of Figure 2, the standard GEM measures the general ability to produce relevant reviews. By conditioning on the abstract, GEM-S (abstract) evaluates the LLM’s ability to generate judgments, including retrieving author-stated strengths and weaknesses (ASSW) and providing critical feedback. By further conditioning on a summary of these author-stated strengths and weaknesses, GEM-S (ASSW) narrows the focus to the LLM’s capacity for critical thinking and delivering constructive feedback.

## 4. Experiment: Validating GEM’s Effectiveness

In this section, we first provide an overview of our experiment setup to validate the effectiveness of the GEM metric in benchmarking LLMs without gold-standard references, and show the results. We provide our code repository at [https://github.com/yx-lu/Benchmarking-LLMs--Judgments-with-No-Gold-Standard](https://github.com/yx-lu/Benchmarking-LLMs--Judgments-with-No-Gold-Standard).

##### GEM setup

We present the empirical setup of the GEM metric and its variants. In the validation experiment in this section, Llama-3.1 8B is used as the evaluation-LM to estimate $\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x]$ and $\log\Pr_{\operatorname{LLM}}[Y=y]$. After confirming the effectiveness of GEM metrics using the small model, for even better performance, we scaled up to a larger, 70B-parameter version of Llama-3.1 for computing the GRE-bench on the ICLR 2023 dataset in Section 5, where we also show the correlation between results of the smaller (8B) and larger (70B) models, to ensure consistency of our approach.

For text preprocessing, we employ GPT-4o. For GEM-S, without further indication, we use the abstract of the paper as the synopsis in this section. The prompts for probability estimation and preprocessing can be found in Appendix C.

##### Baselines

We compare the performance of GEM and its variants with the following established evaluation metrics commonly used in NLG evaluation, including two metrics from the pre-LLM era, BLEU (Papineni et al., ) and ROUGE-L (Lin, ), and an embedding-based metric, BERTScore (Zhang et al., ), a probability-based metric BARTScore (Yuan et al., ), and a GPT-based metric LMExaminer (Language Model as Examiner) (Bai et al., ). Here, we mainly discuss the LLM-based metrics, the detailed implementation of all baselines is provided in Appendix D.3.

BERTScore (Zhang et al., ) evaluates generated text by measuring the cosine similarity between token embeddings from a pre-trained language model. Instead of BERT, we use a recent model stella_en_400M_v5 (Zhang, ) as the evaluation-LM for better token embedding performance.

BARTScore (Yuan et al., ) is a probability-based metric. It has three variants, precision, recall, and F1-score. Instead of BART, we use the state-of-the-art Llama3.1-8b as the evaluation-LM for estimating the probability, which is the same as our GEM metrics.

LMExaminer (Bai et al., ) is one of the recently popular methods where a language model (LM) is used to evaluate the quality of generated text based on specific evaluation criteria, serving as an examiner. With empirical evidence suggesting that GPT-4 outperforms open-source models and even finetuned models of pointwise grading on AlignBench (Ke et al., ), we use GPT-4o as a baseline in our experiment. For peer review tasks, our prompt adopts criteria based on Review Quality Indicators (RQIs) (Goldberg et al., ; Van Rooyen et al., ; Superchi et al., ) including four aspects, understanding, coverage, substantiation, constructiveness.

### 4.1. Result 1: Positive Correlation with Human Annotation

In this section, we show the experiment results utilizing a human-annotated dataset to validate how well the GEM metrics correlate with human-labeled quality scores compared to various baselines. A higher correlation indicates that the evaluation metric better aligns with human preference.

Note that, a perfect correlation can be impossible to achieve for several reasons: Human annotations are inherently noisy (Goldberg et al., ), evaluation of individual responses can also be noisy due to subjectivity, and there is no gold-standard reference. Nonetheless, our goal is to demonstrate systematic-level positive correlation.

##### Score

For task $w_{i}$ with corresponding responses $\mathbf{x}_{i}=\{x_{ij}\}$ where $x_{ij}$ is the $j$-th response. The score of response $x_{ij}$ is computed by $\frac{1}{|\mathbf{x}_{i}|-1}\sum_{k\neq j}f(w_{i},x_{ij},x_{ik})$ where $f$ is the evaluation metric.

##### Human-Annotated Peer Grading Dataset

This dataset contains 30 student project proposals in a graduate-level class on machine learning and about 180 peer evaluations of the proposals. We use GPT-4o to generate a short abstract for each proposal. Each proposal has at least 4 reviews, and each review contains an overall score and textual feedback, including “Strengths of the project”, “Weaknesses of the project/likely roadblocks”, and “Ideas for improvement or specific directions”. According to the quality of students’ textual feedback, an instructor manually grades the reviews with grade A, B, or C.

**Table 1. Spearman’s correlation coefficient ($\rho$) and p-values between various evaluation metrics and instructor-annotated grades. Significant positive correlations ($p<0.05$) are bolded.**
| Evaluation Metric | Spearman’s $\rho$ | p-value |
| --- | --- | --- |
| BLEU | 0.023 | 0.772 |
| ROUGE-L | -0.244 | 0.002 |
| BERTScore | -0.061 | 0.439 |
| BARTScore-F1 | -0.237 | 0.002 |
| BARTScore-prec. | -0.511 | 2.3e-12 |

##### Statistics Metric and Results

Table 1 shows Spearman’s correlation coefficient $\rho$ between the scores of various evaluation metrics and the instructors’ grades. All variants of our GEM metrics show a significant positive correlation. The GEM-S has the highest correlation over all three GEM variants, implying the effectiveness of filtering out shortcuts. It also shows competitive performance compared to GPT-4o LMExaminer which has the highest correlation. All other baselines perform worse than GEM, and only BARTScore-Recall has a significant positive correlation. In later sections, we provide further comparisons between GEM and LMExaminer. (^3^33An intriguing future direction is to re-calculate the GEM score with rephrased system prompts or different preprocessing methods, and take the average to reduce the noise of GEM metric.)

### 4.2. Result 2: Better Sensitivity to Degradation

In this section, we aim to compare how sensitive the evaluation metrics are to degradations that obviously reduce the semantic informativeness of the responses. A statistically significant score decrease indicates the sensitivity of the metric. We first introduce our validation workflow in Algorithm 1, the ICLR peer review dataset, and then introduce several degradation strategies. The workflow and dataset will also be used in the robustness check again manipulation strategies in Section 4.3.

Figure: ALGORITHM 1 Validation Workflow

Note that the evaluation metrics that we compare are in different scales, thus, instead of comparing the mean difference, we standardize the scores with pooled standard deviation $\sigma_{\text{pooled}}:=\sqrt{(\sigma^{2}+\sigma^{\prime 2})/2}$, and compare the standardized mean difference (SMD) $d:=\frac{\mu^{\prime}-\mu}{\sigma_{\text{pooled}}}$ (Andrade, ). This form of SMD is also known as Cohen’s d (Cohen, ).

##### ICLR Dataset

We use the ICLR 2023(^4^44At the time of our experiment, ICLR 2024 review data was not yet available through the OpenReview API.) peer review data publicly available on OpenReview. We randomly select 300 papers as our benchmark dataset, and for each paper, we randomly select 3 original human reviews: one review to serve as a human candidate, and the other two as reference responses.

##### Degradation Strategies

We introduce three degradation strategies as follows. Note that the degraded responses after Sentence Deletion or Deletion & Completion can be regarded as Blackwell-dominated by the original one. The detailed setup is provided in Appendix D.1.

- (1)
Sentence Deletion. We delete every other sentence of $x_{i}$ to get a degraded response $x_{i}^{\prime}$.
- (2)
Deletion & Completion. After deletion, we use GPT-4o to complete the deleted sentence with consistent language style and semantics to the original response, which provides an informatively degraded response $x_{i}^{\prime}$ but with a similar length to the original one.
- (3)
Abstract-only Review. We use Claude-3-sonnet(^5^55We intentionally use a different LLM from Deletion & Completion to make sure the effect of score decrease is not caused only by the LM.) to create a fictitious review $x_{i}^{\prime}$ with only the abstract of the paper, and use $x_{i}^{\prime}$ to replace $x_{i}$.

##### Statistics and Results

We compare the standardized mean differences (SMD) of evaluation metrics in Tabel 2: GEM and GEM-S are only two metrics that consistently have significant negative SMD on all three degradations. The LMExaminer shows the highest effectiveness on Sentence Deletion and Deletion & Completion, but failed to penalize reviews only based on the abstract.

**Table 2. Standardized Mean Differences (SMD) $d:=(\mu^{\prime}-\mu)/{\sigma_{\text{pooled}}}$ of scores after degradations and manipulations of various evaluation metrics. Significant score decreases ($p<0.05$) after degradations are highlighted in bold green, implying the evaluation metric can effectively penalize the degradation. Significant score increases ($p<0.05$) after manipulations are highlighted in bold red, implying the evaluation metric is not robust against the manipulation. We provide 95% confidence interval of SMD values in Appendix A.1.**
| Evaluation | Sentence | Deletion & | Abstract-only | GPT-4o | Llama3.1 | Meaningless |
| --- | --- | --- | --- | --- | --- | --- |
| Metric | Deletion | Completion | Review | Rephrase | Rephrase | Elongation |
| BLEU | -0.282 | 0.016 | -0.692 | -0.975 | -0.920 | -0.165 |
| ROUGE-L | -0.073 | 0.091 | 0.022 | 0.028 | 0.120 | -0.196 |
| BERTScore | -0.100 | -0.188 | 0.840 | 0.134 | 0.063 | 0.064 |
| BARTS.-F1 | 0.401 | 0.201 | 0.394 | 0.130 | 0.332 | -0.304 |
| BARTS.-prec. | 0.627 | 0.317 | 0.617 | 0.218 | 0.541 | -0.439 |
| BARTS.-recall | -0.017 | -0.016 | 0.004 | -0.027 | -0.028 | 0.004 |
| LMExaminer | -1.290 | -0.417 | 0.715 | 0.187 | 0.104 | 0.105 |
| GEM-raw | -0.123 | -0.126 | 0.020 | -0.123 | -0.126 | 0.020 |
| GEM | -0.401 | -0.308 | -0.191 | -0.058 | -0.107 | -0.063 |
| GEM-S | -0.409 | -0.206 | -0.566 | -0.046 | -0.114 | -0.070 |

### 4.3. Result 3: Better Robustness against Manipulation

Evidence has been found that the LLM evaluators, and even humans, are not robust against specific manipulation strategies, they may rely on language style or length as shortcuts instead of providing an evaluation based on semantic quality (Panickssery et al., ; Bai et al., ; Goldberg et al., ), thus, it is important to further check our metric’s robustness against several manipulations.

Specifically, we employ two manipulation strategies, which do not significantly change the semantics, but may change the language style or add meaningless starting sentences. After manipulation, if the score significantly increases, the evaluation metric fails to pass the robustness check. We use the same workflow and dataset introduced in Section 4.2.

##### Manipulation Strategies

We now introduce the manipulation strategies. The detailed setup, including the prompts and examples, is provided in Appendix D.2.

- (1)
GPT-4o/Llama-3.1 Rephrase. Following Bai et al. (), we use GPT-4o and Llama-3.1 to rephrase the response $x_{i}$ while trying to keep the semantics unchanged. We test whether the evaluation metrics have preference towards GPT-4o or Llama-3.1 generated language.
- (2)
Meaningless Elongation. We adopt a meaningless elongation method similar to Goldberg et al. ()’s human subject experiment where a set of fixed meaningless sentences is added to each section of all the reviews as shown in Figure 4.

Figure: Figure 4. Meaningless Elongation
Refer to caption: extracted/5992515/figures/elongation.png

##### Statistics and Results

The SMDs shown in Tabel 2 demonstrate that LMexaminer has significant score increases after all three manipulations, indicating its lack of manipulation-resistance. In contrast, GEM and GEM-S exhibit no significant score increase across these manipulations. Furthermore, comparing SMDs after degradations and manipulations in Table 2, we observe that the scores of GEM and GEM-S decrease less after manipulations compared to degradations, indicating that manipulations result in less information loss than degradations.

## 5. GRE-bench: Benchmarking LLMs with GEM

After validating our evaluation metrics GEM and GEM-S in Section 4.1, 4.2, and 4.3, we now show the results of GRE-bench (Generating Review Evaluation Benchmark) based on GEM and GEM-S. We use the same ICLR 2023 peer review dataset introduced in Section 4.2.

We prompt LLMs to generate detailed reviews for each paper in the dataset, with the prompt inspired by Liang et al. (), which covers various aspects, including paper summary, strengths, weaknesses, questions, etc. Details of the prompts are provided in Appendix C.3.

We then use GEM and GEM-S to score the LLM-generated reviews. Specifically, as discussed in Section 3.2, we use GEM-S (abstract) and GEM-S (ASSW), where the synopsis consists of an abstract for GEM-S (abstract), and an abstract supplemented with a summary of author-stated strengths and weaknesses (ASSW) for GEM-S (ASSW). We prompt GPT-4o to get the ASSW summary of each paper, the prompt is shown in Appendix C.4. We visualize the results of the GRE-bench based on GEM, GEM-S (abstract), and GEM-S (ASSW) respectively in Figure 5. More detailed numerical results are provided in Table 6 of Appendix A.2.

Figure: Figure 5. Results of GRE-bench based on three evaluation metrics with 90% confidence intervals vs. model parameter size. The grey line represents the average human baseline, with the 90% confidence interval shaded in grey.
Refer to caption: extracted/5992515/figures/GRE-bench-70b.png

Figure 5 shows that, within each LLM family, such as the Llama-3 or Claude series, models with larger parameter sizes tend to perform better. Moreover, newer versions tend to outperform their predecessors with similar model sizes, as seen in comparisons between Llama-3.1 and Llama-3. These observations further validate the effectiveness of our GEM and GEM-S metrics in accurately evaluating LLMs.

Notably, several models with large parameter sizes surpass the human baseline (gray line) when using GEM and GEM-S (abstract) metrics. We hypothesize that the reason is that strong LLMs like Claude-3 opus can effectively retrieve contributions and limitations stated by the authors in the paper, and will include that in their reviews. In contrast, human reviewers tend to omit some judgments that are less important or already stated by the author due to writing and expression costs, resulting in a loss of mutual information. This hypothesis can be supported by the observation that, when we condition out the author-stated strengths and weaknesses, the human reviewer has the highest GRE-bench score based on GEM-S (ASSW).

This observation further validates the hierarchical information structure as we discussed in Section 3.2. By conditioning out different levels of information, the different benchmarks capture subtly different abilities. GEM-S (ASSW) focuses more on quality of critical judgment, while GEM and GEM-S (abstract) are more effective in evaluating LLMs’ abilities in more basic aspects, including retrieving useful information from the paper, which is especially valuable for evaluating LLMs with smaller parameter size.

##### GRE-bench vs. Other benchmarks

The ability to generate informative reviews relies on several key factors, including logical reasoning, critical thinking, and fact-checking. Various widely used benchmarks measure different aspects of LLM performance. To better understand which abilities are most critical for generating high-quality reviews, in the sense of GRE-bench scores, we compared our results with several other benchmarks.

We present the Spearman’s correlation coefficients between the GRE-bench scores of all the models and their corresponding scores on other benchmarks in Table 3, providing insights into which factors most strongly influence review generation quality. Notably, GRE-bench highly correlates with HellaSWAG and ARC-C which measures reasoning ability, and less correlates with benchmarks for coding ability (HumanEval) or math ability (MATH, GSM8K).

**Table 3. Spearman’s correlation coefficients between GRE-bench and other popular benchmarks.**
| Base Metric | MMLU | ARC-C | HellaSwag | GSM8K | MATH | HumanEval | GPQA |
| --- | --- | --- | --- | --- | --- | --- | --- |
| GEM | 0.55 | 0.68 | 0.74 | 0.58 | 0.37 | 0.36 | 0.60 |
| GEM-S (abstract) | 0.66 | 0.70 | 0.82 | 0.73 | 0.43 | 0.43 | 0.67 |
| GEM-S (ASSW) | 0.68 | 0.78 | 0.84 | 0.73 | 0.43 | 0.48 | 0.71 |

##### Consistency over Various Model Sizes

The Spearman’s correlation coefficients between the GRE-bench scores based on GEM-S (ASSW) metric computed using Llama-3.1 8B and that same score computed with Llama-3.1 70B is 0.93. This indicates that the score is largely robust with respect to the specific evaluation LM used. The analogous coefficient for the GRE-bench scores based on GEM-S (abstract) and GEM are 0.92 and 0.90 respectively. We further provide the comprehensive results of GRE-bench with Llama-3.1 8B in Table 7 of Appendix A.3. This consistency shows that with our metrics, a small model with 8B parameters that can be deployed on a single NVIDIA L4 GPU, plus proper preprocessing, is still effective in evaluating the quality of judgments generated by much larger models, though larger evaluation LM may have better evaluation performance.

## 6. Conclusion and Discussion

In conclusion, we introduce GEM and GEM-S, two metrics for evaluating LLMs on tasks without gold-standard references. They have proved to be accurate and manipulation-resistant, outperforming other metrics in the experiment. Based on GEM and GEM-S, we present GRE-bench for assessing LLMs’ peer review capabilities while minimizing the risk of data contamination.

Our approach has certain limitations. First, although our experiments demonstrate its effectiveness, the theoretical results depend on the accurate estimation of the underlying distribution by the LLM, which may not always be reliable. With ongoing advancements in LLM technology, we hope that these estimations will become more accurate over time. Second, while our method does not require gold-standard references, it requires reference that includes original human-generated information. If all reviewers submit solely LLM-generated reviews—such as those only from GPT-4o—the mutual information-based approach may unfairly assign the highest score to GPT-4o. This echoes recent research demonstrating that the absence of original human-generated data in AI evaluation can result in model collapse (Shumailov et al., ).

We list several potential directions for future work. We can use new open-access peer review data to update GRE-bench every year and observe how the evaluation evolves over time, to monitor any improvements in LLMs’ peer review capabilities. This will also ensure the benchmark stays current and minimizes data contamination.

Moreover, for specific tasks with a rich dataset, e.g., peer review in ICLR, we can finetune the evaluation-LM to better estimate the joint distribution. However, the widely-used supervised finetuning may not be suitable for subjective tasks, as the responses in the dataset may have different judgments on the same task. We provide initial results in Appendix E, which show better efficacy than baselines, but are not significantly better than the un-finetuned model, thus, a more suitable finetuning method may be further explored.

## Acknowledgment

We would like to thank Sanzeed Anwar from University of Michigan for data processing. We would also like to thank Zebang Xu and Tianzi Liu from Cornell University for consultation on the statistical methods applied in our experiments.

## Appendix A Additional Results

### A.1. Result 2 and 3 with Confidence Intervals

We present the SMDs with 95% confidence intervals of various evaluation metrics under degradations and manipulations in Table 4 and 5.

**Table 4. Standardized Mean Differences (SMD) $d:=(\mu^{\prime}-\mu)/{\sigma_{\text{pooled}}}$ of scores after degradations of various evaluation metrics with 95% confidence intervals in parentheses. Significant score decreases ($p<0.05$) after degradations are highlighted in bold green, implying the evaluation metric can effectively penalize the degradation.**
| Evaluation Metric | Sentence Deletion | Deletion & Completion | Abstract-only Review |
| --- | --- | --- | --- |
| BLEU | -0.282 | 0.016 | -0.692 |
|  | (-0.346,-0.218) | (-0.013,0.046) | (-0.778,-0.607) |
| ROUGE-L | -0.073 | 0.091 | 0.022 |
|  | (-0.122,-0.025) | (0.062,0.121) | (-0.054,0.098) |
| BERTScore | -0.100 | -0.188 | 0.840 |
|  | (-0.131,-0.069) | (-0.222,-0.155) | (0.769,0.910) |
| BARTScore-F1 | 0.401 | 0.201 | 0.394 |
|  | (0.380,0.422) | (0.184,0.218) | (0.344,0.445) |
| BARTScore-precision | 0.627 | 0.317 | 0.617 |
|  | (0.595,0.659) | (0.292,0.343) | (0.536,0.698) |
| BARTScore-recall | -0.017 | -0.016 | 0.004 |
|  | (-0.019,-0.016) | (-0.017,-0.014) | (0.001,0.007) |
| LMExaminer | -1.290 | -0.417 | 0.715 |
|  | (-1.343,-1.238) | (-0.472,-0.363) | (0.630,0.799) |
| GEM-raw | -0.123 | -0.126 | 0.020 |
|  | (-0.126,-0.120) | (-0.132,-0.121) | (0.017,0.023) |
| GEM | -0.401 | -0.308 | -0.191 |
|  | (-0.448,-0.354) | (-0.358,-0.258) | (-0.261,-0.122) |
| GEM-S | -0.409 | -0.206 | -0.566 |
|  | (-0.455,-0.362) | (-0.254,-0.158) | (-0.639,-0.492) |

**Table 5. Standardized Mean Differences (SMD) of scores $d:=(\mu^{\prime}-\mu)/{\sigma_{\text{pooled}}}$ after manipulations of various evaluation metrics with 95% confidence intervals in parentheses. Significant score increase ($p<0.05$) after manipulations are highlighted in bold red, implying the evaluation metric are not robust against the manipulation.**
| Evaluation Metric | GPT-4o Rephrase | Llama3.1 Rephrase | Meaningless Elongation |
| --- | --- | --- | --- |
| BLEU | -0.975 | -0.920 | -0.165 |
|  | (-1.020,-0.930) | (-0.971,-0.870) | (-0.230,-0.101) |
| ROUGE-L | 0.028 | 0.120 | -0.196 |
|  | (0.009,0.047) | (0.080,0.160) | (-0.241,-0.151) |
| BERTScore | 0.134 | 0.063 | 0.064 |
|  | (0.113,0.155) | (0.032,0.093) | (0.042,0.086) |
| BARTScore-F1 | 0.130 | 0.332 | -0.304 |
|  | (0.120,0.140) | (0.299,0.365) | (-0.308,-0.299) |
| BARTScore-precision | 0.218 | 0.541 | -0.439 |
|  | (0.204,0.233) | (0.490,0.593) | (-0.446,-0.433) |
| BARTScore-recall | -0.027 | -0.028 | 0.004 |
|  | (-0.028,-0.027) | (-0.029,-0.027) | (0.004,0.005) |
| LMexaminer | 0.187 | 0.104 | 0.105 |
|  | (0.153,0.221) | (0.060,0.147) | (0.069,0.140) |
| GEM-raw | -0.123 | -0.126 | 0.020 |
|  | (-0.126,-0.120) | (-0.132,-0.121) | (0.017,0.023) |
| GEM | -0.058 | -0.107 | -0.063 |
|  | (-0.090,-0.026) | (-0.143,-0.070) | (-0.097,-0.030) |
| GEM-S | -0.046 | -0.114 | -0.070 |
|  | (-0.079,-0.013) | (-0.149,-0.078) | (-0.104,-0.036) |

### A.2. Detailed Results of GRE-bench with Llama3.1 70B

We present numerical results of various language models on the GRE-bench based on GEM, GEM-S (abstract), and GEM-S (ASSW) with Llama3.1 70B, alongside their estimated parameter sizes in Table 6. The corresponding results of GRE-bench with Llama3.1 8B is shown in Table 7.

**Table 6. Results of GRE-bench based on GEM, GEM-S (abstract), and GEM-S (ASSW) with Llama3.1 70B. In the column of parameter size, the symbol $\sim$ indicates that this size is estimated.**
| Model Name | Para. Size | GRE-bench |  |  |
| --- | --- | --- | --- | --- |
|  | (in Billion) | GEM | GEM-S(abs.) | GEM-S(ASSW) |
| openai/gpt-4o-mini (v 20240718) | $\sim$ 8 | 16.31 | 5.25 | 0.52 |
| openai/gpt-4o (v 20240513) | $\sim$ 200 | 20.08 | 7.62 | 2.82 |
| openai/gpt-3.5-turbo (v 20240125) | $\sim$ 20 | 19.28 | 6.31 | 1.40 |
| anthropic/claude-2 | $\sim$ 137 | 20.04 | 7.17 | 2.27 |
| anthropic/claude-3-haiku (v 20240307) | $\sim$ 20 | 20.32 | 7.63 | 2.50 |
| anthropic/claude-3-sonnet (v 20240229) | $\sim$ 70 | 24.11 | 9.53 | 4.25 |
| anthropic/claude-3-opus (v 20240229) | $\sim$ 2000 | 25.77 | 11.52 | 5.89 |
| google/gemini-pro (v 002) | $\sim$ 20 | 14.73 | 4.09 | -0.26 |
| google/gemini-pro-1.5 (v 002) | $\sim$ 175 | 22.76 | 9.61 | 4.47 |
| meta-llama/llama-3-8b-instruct | 8 | 16.68 | 5.81 | 1.07 |
| meta-llama/llama-3-70b-instruct | 70 | 18.35 | 7.23 | 3.37 |
| mistralai/mixtral-8x7b-instruct (v 0.1) | 56 | 16.86 | 5.22 | 1.23 |
| mistralai/mixtral-8x22b-instruct (v 0.1) | 176 | 18.94 | 5.82 | 1.64 |
| meta-llama/llama-3.1-8b-instruct | 8 | 18.74 | 7.60 | 2.40 |
| meta-llama/llama-3.1-70b-instruct | 70 | 19.95 | 8.00 | 3.47 |
| mistralai/mistral-large (v 2407) | 123 | 20.40 | 7.44 | 2.72 |
| mistralai/mistral-small (v 2409) | 22 | 18.08 | 6.70 | 1.28 |
| mistralai/mistral-tiny (v mistral-7b-0.3) | 7 | 17.21 | 4.11 | 0.15 |

### A.3. Detailed Results of GRE-bench with Llama3.1 8B

**Table 7. Results of GRE-bench based on GEM, GEM-S (abstract), and GEM-S (ASSW) with Llama3.1 8B. In the column of parameter size, the symbol $\sim$ indicates that this size is estimated.**
| Model Name | Para. Size | GRE-bench |  |  |
| --- | --- | --- | --- | --- |
|  | (in Billion) | GEM | GEM-S(abs.) | GEM-S(ASSW) |
| openai/gpt-4o-mini (v 20240718) | $\sim$ 8 | 42.55 | 14.44 | 10.25 |
| openai/gpt-4o (v 20240513) | $\sim$ 200 | 48.01 | 18.73 | 12.78 |
| openai/gpt-3.5-turbo (v 20240125) | $\sim$ 20 | 45.62 | 15.86 | 10.72 |
| anthropic/claude-2 | $\sim$ 137 | 55.94 | 24.29 | 17.22 |
| anthropic/claude-3-haiku (v 20240307) | $\sim$ 20 | 49.77 | 19.58 | 13.34 |
| anthropic/claude-3-sonnet (v 20240229) | $\sim$ 70 | 56.49 | 24.16 | 16.52 |
| anthropic/claude-3-opus (v 20240229) | $\sim$ 2000 | 60.94 | 27.94 | 20.21 |
| google/gemini-pro (v 002) | $\sim$ 20 | 40.65 | 12.86 | 9.49 |
| google/gemini-pro-1.5 (v 002) | $\sim$ 175 | 52.94 | 22.43 | 16.06 |
| meta-llama/llama-3-8b-instruct | 8 | 43.50 | 15.49 | 11.24 |
| meta-llama/llama-3-70b-instruct | 70 | 46.39 | 18.60 | 13.68 |
| mistralai/mixtral-8x7b-instruct (v 0.1) | 56 | 45.18 | 15.61 | 11.25 |
| mistralai/mixtral-8x22b-instruct (v 0.1) | 176 | 47.29 | 17.03 | 11.85 |
| meta-llama/llama-3.1-8b-instruct | 8 | 46.51 | 18.33 | 12.69 |
| meta-llama/llama-3.1-70b-instruct | 70 | 48.29 | 19.59 | 13.91 |
| mistralai/mistral-large (v 2407) | 123 | 47.84 | 17.84 | 12.52 |
| mistralai/mistral-small (v 2409) | 22 | 45.01 | 15.90 | 11.10 |
| mistralai/mistral-tiny (v mistral-7b-0.3) | 7 | 43.90 | 13.34 | 9.94 |

## Appendix B Omitted Proofs

See 3.1

###### Proof of Proposition 3.1 .

By the definition of estimated PMI, $\widehat{\operatorname{PMI}}(x;y)=\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x]-
\log\Pr_{\operatorname{LLM}}[Y=y]$, we have

$$ $\displaystyle\widehat{I}(X;Y)$ $\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\widehat{\operatorname{PMI}}(x_{i};y_{i})$ $\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\log\Pr_{\operatorname{LLM}}[Y=y_{i} \mid X=x_{i}]-\log\Pr_{\operatorname{LLM}}[Y=y_{i}]$ $\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\log\Pr_{\operatorname{LLM}}[Y=y_{i} \mid X=x_{i}]-\frac{1}{n}\sum_{i=1}^{n}\log\Pr_{\operatorname{LLM}}[Y=y_{i}].$ $$

Note that $\frac{1}{n}\sum_{i=1}^{n}\log\Pr_{\operatorname{LLM}}[Y=y_{i}]$ does not change whatever the candidate response follows $X_{H}$ or $X_{L}$. In addition, recall that we assume that all task-response tuples $(w_{i},x_{i},y_{i})$ are i.i.d. sampled from the underlying distribution $\mathcal{D}$. Therefore, with the law of large numbers, we have $\widehat{I}(X_{H};Y)\geq\widehat{I}(X_{L};Y)-\epsilon$ further equivalent to

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H}] \right]\geq\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L}]\right ]-\epsilon.$ $$

On the left hand side, we have

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H}]\right]$ $\displaystyle=$ $\displaystyle\sum_{x_{H}}\Pr[X_{H}=x_{H}]\sum_{y}\Pr[Y=y\mid X_{H}=x_{H}]\log \Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H}]$ (This step follows the condition of bounded KL-divergence in Proposition 3.1 .) $\displaystyle\geq$ $\displaystyle\sum_{x_{H}}\Pr[X_{H}=x_{H}]\left(\sum_{y}\Pr[Y=y\mid X_{H}=x_{H} ]\log\Pr[Y=y\mid X_{H}=x_{H}]-\epsilon\right)$ $\displaystyle=$ $\displaystyle\sum_{x_{H},y}\Pr[X_{H}=x_{H},Y=y]\log\Pr[Y=y\mid X_{H}=x_{H}]-\epsilon$ $\displaystyle=$ $\displaystyle-H(Y\mid X_{H})-\epsilon$ $\displaystyle=$ $\displaystyle I(Y;X_{H})-H(Y)-\epsilon.$ $$

On the right hand side, we have

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L}]\right]$ $\displaystyle=$ $\displaystyle\sum_{x_{L}}\Pr[X_{L}=x_{L}]\sum_{y}\Pr[Y=y\mid X_{L}=x_{L}]\log \Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L}]$ (This step follows the fact that logarithm is a proper scoring rule [Hendrickson and Buehler, 1971 ] .) $\displaystyle\leq$ $\displaystyle\sum_{x_{L}}\Pr[X_{L}=x_{L}]\sum_{y}\Pr[Y=y\mid X_{L}=x_{L}]\log \Pr[Y=y\mid X_{L}=x_{L}]$ $\displaystyle=$ $\displaystyle-H(Y\mid X_{L})$ $\displaystyle=$ $\displaystyle I(Y;X_{L})-H(Y).$ $$

Since $X_{H}$ follows an information structure $\sigma_{H}$ Blackwell dominates $\sigma_{L}$ that $X_{L}$ follows, by the information process inequality, we have $I(Y;X_{H})>I(Y;X_{L})$.

Therefore, combining all these together, we have

$$ $\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H}]\right]\geq \mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L}]\right]-\epsilon$ $$

and consequently, we have $\widehat{I}(X_{H};Y)\geq\widehat{I}(X_{L};Y)-\epsilon$.

∎

###### Proposition B.1.

For a given random variable $Z$, when the KL-divergence between the LLM estimated distribution and the underlying distribution satisfies

$$ $D_{KL}\left[\Pr_{LLM}[Y=\cdot\mid X=x_{H},Z=z]\Big{\|}\Pr[Y=\cdot\mid X_{H}=x_ {H},Z=z]\right]<\epsilon,~{}\forall x_{H}$ $$

For the two candidates $H$ and $L$ discussed above, and the information structure of $H$ Blackwell dominates $L$’s, when the size of dataset $n$ goes to infinity, we have

$$ $\widehat{I}(X_{H};Y\mid Z)\geq\widehat{I}(X_{L};Y\mid Z)-\epsilon$ $$

###### Proof.

By following similar steps of the proof of Proposition 3.1, we reduce the problem to showing

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H},Z=z] \right]\geq\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L},Z=z] \right]-\epsilon.$ $$

Again, following similar steps, we have that the left hand side follows

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{H},Z=z]\right]$ $\displaystyle\geq$ $\displaystyle\sum_{z}\Pr[Z=z]\left(I(Y;X_{H}\mid Z=z)-H(Y\mid Z=z)-\epsilon\right)$ $\displaystyle=$ $\displaystyle I(Y;X_{H}\mid Z)-H(Y\mid Z)-\epsilon$ $$

Similarly, we have the right hand side

$$ $\displaystyle\mathbb{E}\left[\log\Pr_{\operatorname{LLM}}[Y=y\mid X=x_{L},Z=z]\right]$ $\displaystyle\leq$ $\displaystyle\sum_{z}\Pr[Z=z]\left(I(Y;X_{L}\mid Z=z)-H(Y\mid Z=z)-\epsilon\right)$ $\displaystyle=$ $\displaystyle I(Y;X_{L}\mid Z)-H(Y\mid Z)$ $$

Therefore, combining all these together, we have

$$ $\widehat{I}(X_{H};Y\mid Z)\geq\widehat{I}(X_{L};Y\mid Z)-\epsilon$ $$

∎

###### Proof of Proposition 3.1 .

###### Proposition B.1.

###### Proof.

## Appendix C Implementation Details

In this section, we provide a detailed interpretation of how we implement our proposed metrics to facilitate the replication of our results. This includes comprehensive descriptions of all prompts utilized for generating peer-review judgments, preprocessing the judgments, and predicting the judgments.

### C.1. Judgments Preprocessing

To filter out the shortcuts in the review judgments, we use a preprocessing procedure. We employ an LLM to rewrite the original review judgments in a certain compressed format, eliminating the impact of various aspects such as semantics, syntax, and language styles. Here, we employ the LLM GPT-4o (specifically, gpt-4o-2024-05-13) to preprocess the judgments generated by all candidate LLMs (and human reviewers). Here is the prompt we use:

### C.2. Judgments Prediction

We compute our metric, the Generative Estimator for Mutual Information (GEM), by utilizing both the original and fine-tuned versions of Llama-3.1-8B-Instruct(^6^66We employ 4-bit quantization.) to predict the conditional probability $\Pr_{\text{LLM}}[Y=y\mid X=x]$ and the marginal probability $\Pr_{\text{LLM}}[Y=y]$. In this context, $x$ and $y$ represent preprocessed review judgments. The same methodology is employed for calculating the GEM-S score. Below is the prompt used for prediction.

### C.3. Peer-review Judgments Generation

In GRE-bench, we employ academic peer review as the task for candidate LLMs to generate informative judgments. Below is the prompt utilized to request each candidate LLM to produce its review judgments.

### C.4. Summary of Author-stated Strengths and Weaknesses

In GEM-S (ASSW), we need to embed the strength and weakness text stated by the authors into the input of LLM. To do this, we employ gpt-4o to summarize it using the following prompt.

## Appendix D Experimental Details

In this section, we provide detailed information regarding the experiments we conducted. This includes the specific processes used to degrade and manipulate reviews, as well as the implementation of the baseline metrics.

### D.1. Degradations

#### D.1.1. Review Degradation: Sentence Deletion

The ICLR 2023 review comments must follow a specific formatting with four sections including “Summary Of The Paper”, “Strengths And Weaknesses”, “Clarity, Quality, Novelty And Reproducibility” and “Summary Of The Review”. When performing sentence deletion, we maintain the original format unchanged. Other sentences are identified based on periods, question marks, exclamation points, and line breaks, which mark the end of a sentence. In each section, we retain all sentences with odd-numbered positions and delete those in even-numbered positions. Below is a toy example for illustration.

#### D.1.2. Review Degradation: Deletion & Completion

When applying the deletion & completion method to degrade reviews, we initially delete half of the sentences, similar to the sentence deletion process, and replace the omitted content with “[There is one missing sentence]”. Subsequently, we utilize GPT-4o to complete these sentences. Below is a toy example.

#### D.1.3. Review Degradation: Abstract-only Review

In this degradation process, we utilize claude-3-sonnet to generate a review based solely on the paper’s abstract, which serves as a substitute for the original review. Below is the prompt we use to complete this task.

### D.2. Manipulations

#### D.2.1. Review Manipulation: LLM Rephrasing

To rephrase the review without altering the semantics, we employ GPT-4o and Llama-3.1-instruct to rewrite the original review judgments, modifying the phrasing while maintaining the original meaning. The same prompt is used for both models to accomplish this task.

#### D.2.2. Review Manipulation: Meaningless Elongation

We employ a verbose and uninformative template content alongside summaries of the existing review to achieve a meaningless elongation of the original review judgments. Specifically, we adhere to the fixed review format established by ICLR 2023, which consists of four sections. At the beginning of each content section within these sections, we incorporate an identical introductory statement. Additionally, we utilize GPT-4o to generate a one-sentence summary for the “Strengths and Weaknesses” section of the review, embedding this summary within the added text. Below is an example, including the meaningless content we add to the original review.

The prompts that request GPT-4o to generate the Strength Summary and Weakness Summary are presented as follows.

### D.3. Baseline Metrics

In the paper, we use BLEU, ROUGE-L, BERTScore and LMExaminer as the baseline metrics. BLEU and ROUGE-L are both detail free. We employ NLTK as the tokenizer and utilized existing library functions to compute the BLEU and ROUGE-L scores. When calculating BERTScore, it is also necessary to split the text into several segments and embed each segment before computing the cosine similarity. We choose to split the text by sentence and then embedding each sentence individually. The model used for sentence embedding is stella_en_v5.

LMExaminer utilizes a specific LLM to score answers to questions. The original LMExaminer requires the input of the question, reference answer, and the answer to be scored into the LLM. However, since we are dealing with the subjective nature of peer review, we input the full text of the paper along with the review to be judged into the LLM and prompt it to score based on multiple dimensions. We employ GPT-4o as the examiner, and below are our prompts.

## Appendix E GEM / GEM-S with Fine-tuned Evaluation-LM

We not only use the original Llama-3.1-8B-Instruct as the Evaluation-LM, but also fine-tune it to to better estimate the conditional and marginal probabilities. In this section, we provide the details of how we fine-tune the model, as well as the experimental results.

### E.1. Implementation

We fine-tune the original Llama-3.1-8B-Instruct model using 1,000 papers and their corresponding reviews from ICLR 2023 (which are separate from the papers used for experiments). Let $\texttt{gen}(\text{metric},x,y)$ represent the function that generates the full prompt for predicting judgments (both input and output), where $x=\texttt{Null}$ indicates that we are estimating the marginal probability. For each paper, we randomly selected three reviews, denoted as $r_{1}$, $r_{2}$, and $r_{3}$, along with their preprocessed versions $\hat{r}_{1}$, $\hat{r}_{2}$, and $\hat{r}_{3}$. From each paper, we generated nine data points for GEM and nine data points for GEM-S as follows: $\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{1},\hat{r}_{2})$,
$\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{1},\hat{r}_{3})$,
$\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{2},\hat{r}_{1})$,
$\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{2},\hat{r}_{3})$,
$\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{3},\hat{r}_{1})$,
$\texttt{gen}(\texttt{GEM(-S)},\hat{r}_{3},\hat{r}_{2})$,
$\texttt{gen}(\texttt{GEM(-S)},\texttt{Null},\hat{r}_{1})$,
$\texttt{gen}(\texttt{GEM(-S)},\texttt{Null},\hat{r}_{2})$,
$\texttt{gen}(\texttt{GEM(-S)},\texttt{Null},\hat{r}_{3})$.

In total, this process yield 18,000 input-output pairs. We used the Unsloth framework to fine-tune the model, running on a physical server equipped with a single NVIDIA A100 GPU, provided by Google Colab. Here is the settings we use for fine-tuning:

### E.2. Experimental Results

**Table 8. Standardized Mean Differences (SMD) $d:=(\mu^{\prime}-\mu)/{\sigma_{\text{pooled}}}$ of scores after degradations of various evaluation metrics with 95% confidence intervals in parentheses. Significant score decreases ($p<0.05$) after degradations are highlighted in bold green, implying the evaluation metric can effectively penalize the degradation.**
| Evaluation Metric | Sentence Deletion | Deletion & Completion | Abstract-only Review |
| --- | --- | --- | --- |
| GEM-raw | -0.123 | -0.126 | 0.020 |
|  | (-0.126,-0.120) | (-0.132,-0.121) | (0.017,0.023) |
| GEM | -0.401 | -0.308 | -0.191 |
|  | (-0.448,-0.354) | (-0.358,-0.258) | (-0.261,-0.122) |
| GEM-S | -0.409 | -0.206 | -0.566 |
|  | (-0.455,-0.362) | (-0.254,-0.158) | (-0.639,-0.492) |
| GEM-finetune | -0.155 | -0.135 | -0.043 |
|  | (-0.199,-0.110) | (-0.180,-0.090) | (-0.106,0.020) |
| GEM-S-finetune | -0.192 | -0.099 | -0.355 |
|  | (-0.244,-0.140) | (-0.148,-0.049) | (-0.422,-0.288) |

**Table 9. Standardized Mean Differences (SMD) of scores $d:=(\mu^{\prime}-\mu)/{\sigma_{\text{pooled}}}$ after manipulations of various evaluation metrics with 95% confidence intervals in parentheses. Significant score increase ($p<0.05$) after manipulations are highlighted in bold red, implying the evaluation metric are not robust against the manipulation.**
| Evaluation Metric | GPT-4o Rephrase | Llama3.1 Rephrase | Meaningless Elongation |
| --- | --- | --- | --- |
| GEM-raw | -0.123 | -0.126 | 0.020 |
|  | (-0.126,-0.120) | (-0.132,-0.121) | (0.017,0.023) |
| GEM | -0.058 | -0.107 | -0.063 |
|  | (-0.090,-0.026) | (-0.143,-0.070) | (-0.097,-0.030) |
| GEM-S | -0.046 | -0.114 | -0.070 |
|  | (-0.079,-0.013) | (-0.149,-0.078) | (-0.104,-0.036) |
| GEM-finetune | -0.014 | -0.010 | -0.056 |
|  | (-0.044,0.015) | (-0.044,0.024) | (-0.088,-0.025) |
| GEM-S-finetune | -0.026 | -0.045 | -0.074 |
|  | (-0.062,0.010) | (-0.085,-0.004) | (-0.112,-0.037) |

### E.3. GRE-bench based on finetuned GEM and GEM-S

Figure: Figure 6. Results of GRE-bench based on finetuned GEM and GEM-S with 90% confidence intervals vs. model parameter size. The grey line represents the average human baseline, with the 90% confidence interval shaded in grey.
Refer to caption: extracted/5992515/figures/GRE-bench-ft.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Using LLM to estimate the conditional probility seems problematic:  While P(x,y)=P(x)*P(y|x)=P(y)*P(x|y), P_LLM(x)*P_LLM(x|y) != P_LLM(y)*P_LLM(x|y).  I wonder if this will cause problems in the estimation.
soundness: 3
strengths: Using mutual information for evaluation subjective generative response is novel as my understanding.

The paper give theoretic guarantees of the proposed method.

Experiments show the method works well on the task of evaluating reviews for academic papers or student proposals.
summary: The paper introduces GEM, a novel metric for evaluating the performance of large language models in generating informative judgments using given references but not necessary gold standards, and presents GRE-bench, a benchmark for assessing the quality of peer reviews generated by LLMs.

GEM is designed to assess the informativeness of judgments in scenarios where gold standard references are unavailable, expanding the scope of benchmarking to subjective tasks like peer review.
weaknesses: Only one type of evaluation tasks has been assessed: paper review evaluation.  It would be more convicing if the paper can conduct experiments of more subjective evaluation tasks, such as summarization, dialog, etc.  

The paper uses Llama 3.1 8B as the evaluation LM.  I would like to see how different LMs effect the performance of the evaluation.

Figure 4 is not mentioned in the paper.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: This is a kind of optional question out of my curiosity. 

It seems that GEM is a relative scoring/evaluation framework. If so, what would be the pros and cons of GEM compared to other relative evaluation frameworks, such as the ELO rating system commonly used in generation task leaderboards, like those in Chatbot arenas?
soundness: 3
strengths: The paper is clearly written and rigorously formulated. The research topic is very timely and important, given the increasing use of LLMs and the challenges of evaluating their performance in subjective tasks.
summary: This paper presents GEM (Generative Estimator for Mutual Information), a new evaluation metric for assessing the quality of text responses generated by LLMs in tasks that lack a clear gold standard reference.  GEM aims to provide an accurate and manipulation-resistant metric by estimating the mutual information between candidate responses and a set of reference responses, which are not required to be perfect. The main idea is to measure the semantic informativeness of candidate responses by filtering out aspects like language style and syntax, focusing on how much information the candidate's response reveals about the references.
weaknesses: I have some questions and concerns:

1. Impact of Suboptimal Human References: I am concerned about GEM's reliability when human references are of lower quality than LLM outputs. Could the LLM used for estimating probability distributions learn to favor responses that resemble flawed human references, even if they are less informative, potentially leading to inaccurate scores? 

2. Performance stability according to the choice of evaluation-LM and preprocessing LLM: How sensitive is GEM to the choice of LLMs used for evaluation (Llama-3.1) and preprocessing (GPT-4)? Would using different LLMs, especially open-source ones, impact the results? LLMs are released and updated frequently, and I wonder how we can deal with such frequent updates to make the GEM score consistent and reliable over time.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - Missing citations: 
    - [Finding Blind Spots in Evaluator LLMs with Interpretable Checklists](https://arxiv.org/abs/2406.13439)
    - [Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?](https://aclanthology.org/2024.findings-eacl.71/)
- In Appendix D3, I see that the whole paper is given in text format, but most of the scientific literature also contains images that are quite important and referenced multiple times. What happens in that case? Similarly, how are tables being managed? Also, are these papers converted from PDF to text format, if so, are they being verified for OCR errors? The papers presented are usually 10 pages in ICLR and 8 pages in *CL, so are you sure the paper is not beyond the token limit of the LLM? Lastly, which LLM was used for the prompt in D3?
- For ICLR dataset, it is mentioned that one reviewer is the candidate and the remaining 2 are reference. Aren't only 2 references too less? I feel in this case the sample bias will be quite high. I urge the authors to clear this for me. 
- For the rephrase manipulation, were any sanity checks conducted to make sure that the semantics did not change? Since we are operating at very long-context lengths (document-level), this could be an issue for smaller models like llama-3.1-8b. GPT-4o is quite large and powerful as compared to llama-3-8b and has much better instruction following capabilities, so was this disparity maintained purposefully? If not, I feel llama-3.1-405B or any of the other larger models should be used in place of it which are good at instruction-following.
- For the GRE-Bench, where the LLM is generating reviews, were all 3 human reviews taken as references?
- Line 472 is very redundant and but obvious. Smaller models do not have good instruction following abilities. 
- The word Table has been misspelled as Tabel at a couple of places. Please fix it. 
- In Figure-4 (and Table-6), I am quite dubious about the parameter counts being mentioned there, as most of closed-source models do not reveal anything, especially for GPT models. Also, the color-scheme in Fig-4 is not comfortable to the eye to make clear demarcations across the bars. Please fix that as well. 
- In Prompt D2, the model is provided with one review, which resembles a 1-shot example in a way. In case of generative tasks, there have been cases when the model tries to follow the verbatim or structure of the provided few shots, and the final result lacks creativity. Was this checked in any way? 
- What were the generation parameters in each prompting case?
- How were the outputs checked for hallucinations as the generations are quite long?
- In F1, was the same prompt format as in D2 used while finetuning the model?
- There should be some comparison of llama-3.1-8b-Instruct and its finetuned version (as mentioned in F1) to see the gains due to finetuning. Also, finetuning an already finetuned model is slightly tricky as the distribution shift may cause the model to collapse. The general trend is to use a base model directly for this case.
soundness: 2
strengths: - The paper is well-written, and the problem statements is well-motivated. The authors provide cogent and coherent arguments throughout the paper. I personally think this is a very valid issue for LLM-based evaluators as not always we have a single human-verified gold-standard reference, but a set of good-enough responses to aggregate and use as a ground-truth. The paper also has a good mathematical rigor to prove its hypothesis. 
- The benchmark created in the dataset is a plus point as it can help future works. 
- I also appreciate the finding that GEM is robust to perturbations as lately there has been a good discussion around LLM having blind spots in evaluation and missing out on subtle disturbances to the input.
summary: The paper mainly discusses benchmarking LLM's judgments in the lack of gold standard reference answer, but with a pool of vanilla reference answers by estimating the mutual information between a candidate and reference. The authors also show that this technique is also robust against perturbations and manipulations. Lastly, they also present a benchmark/test set to evaluate how good LLMs can generate high-quality reviews for academic paper.
weaknesses: - I do not see any blaring errors in the paper, however, I am a suspicious about the experimental setup as such. Bottom line, the problem statement discusses a **very challenging** problem: _LLMs generating "judicious" reviews for very long and complex scientific papers_. I am not sure if most of the existing LLMs are good enough for such a humongous task and if they do provide good results which can be analyses with some guarantee, as the LLMs are required to have a decent knowledge of the domain as well as a good understanding of the hypothesis in the paper, which even humans struggle to understand in some cases. In a way, this feels like just applying an LLM to a task to see how it fares. But the authors to talk about previous works that have discussed using LLMs as reviewers, and I see that most of them have shown negative results in such cases.
- Related to the above point, I have several questions for the authors, which I hope will be answered to give me some more clarity on the nitty-gritty details of the work.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. I'm not sure I understand the setup of the first result, am I right in understanding that what you do here is generate k candidates per review, and then get the average GEM score for each candidate by averaging over all other generated candidates when used as references? And then you make the classification by choosing the candidate that resulted in the highest GEM score? How is the classification made? If I'm correct in understanding this, then I don't see how the following sentence from the conclusion applies: "...while our method does not require gold-standard references, it requires reference that includes original human-generated information." Since the way I understand it now, it doesn't really need any human-generated references to generate these scores.

2. Why do you think LMExaminer performs so much better in Sentence Deletion and Deletion & Completion? I feel like that's quite a significant result that hasn't been discussed.

3. "Figure 5 shows that, within each LLM family, such as the Llama-3 or Claude series, models with larger parameter sizes tend to perform better. Moreover, newer versions tend to outperform their predecessors with similar model sizes, as seen in comparisons between Llama-3.1 and Llama-3. These observations further validate the effectiveness of our GEM and GEM-S metrics in accurately evaluating LLMs."
Can GRE-Bench only work with the GEM metrics? Do you think you would observe something similar if you used different metrics?
soundness: 2
strengths: The paper presents a valuable step towards doing away with the reliance on gold-standard references in evaluation. It's original and creative in the way it makes use of mutual information in the context of generative models.

The method is described rigorously and thoroughly. The flow and structure of the paper makes sense and conveys their ideas well.

The authors also seem to be highly aware and open about the limitations of the method. The paper actively critiques itself and presents answers and discussions on these critiques.
summary: This work is motivated by the importance of finding a benchmarking method that is not reliant on gold-standard references. The proposed method(s) also attempt to circumvent known issues about using generative models as evaluators, where they are often sensitive to degradation strategies and score-increasing manipulations. The method is empirically compared against a wide range of widely known evaluation metrics to showcase its effectiveness.

The authors also introduce a new benchmark for review generation which is built upon their method that inherits all its benefits. The benchmark is also claimed to combat data contamination by being updated with new review data as it comes out.
weaknesses: While the paper is generally highly verbose discussion-wise, I find the experiments and results parts of the paper to be a bit lacking in discussion and analysis. My concerns on this matter are made more specific in the "Questions" part of this review.

Some minor grammar/spelling mistakes: lines 96-97 "..the score can relies solely...", line 455 "Table" is misspelled.

Meta-review: The paper introduces GEM, a metric for evaluating large language models (LLMs) based on the informativeness of their generated responses without needing gold-standard references. GEM estimates mutual information between candidate and reference responses, making it ideal for subjective tasks like peer reviews. The accompanying benchmark, GRE-bench, assesses peer review quality generated by LLMs using GEM, offering a robust, manipulation-resistant evaluation. GRE-bench mitigates generative model evaluation issues and updates its data to prevent contamination. GEM's effectiveness is validated against existing metrics.

On a positive note, the reviewers commend the paper for its innovative use of mutual information to evaluate subjective generative responses. The approach provides theoretical guarantees and effectively demonstrates its capability in tasks such as academic paper review evaluation. The paper is clearly written and rigorously structured, effectively addressing the pressing challenge of assessing LLM performance in subjective tasks where gold-standard references are absent. Its mathematical rigor, coupled with the introduction of a new benchmark dataset, significantly contributes to ongoing and future research efforts. Additionally, GEM stands out for its robustness to input perturbations. The authors are also praised for their transparency in acknowledging the method's limitations, offering insightful self-critical analysis and discussions throughout the paper.

The reviewers express several critiques and concerns regarding the paper's scope and methodology. They recommend broadening the evaluation to encompass additional tasks such as summarization and dialogue, which would enhance the study's credibility. The reliance on a single language model, Llama 3.1 8B, raises questions about the potential impact of using diverse models on the results. Moreover, there are concerns about GEM's reliability when faced with suboptimal human references and its sensitivity to the choice of both evaluation and preprocessing LLMs. The capability of current LLMs to produce insightful reviews for complex scientific papers is questioned, suggesting that the study may lean more towards an exploratory approach. Although the paper offers detailed discussions, the sections on experiments and results are perceived as lacking in-depth analysis, prompting the reviewer to seek further clarification in future revisions.

The authors have made significant updates to the paper, effectively addressing the previously raised issues. 

The paper makes a significant contribution by introducing a method to estimate mutual information between candidate and reference responses without necessitating the reference to be a gold standard. I think this approach holds the potential for substantial impact. I am leaning to recommend acceptance of the paper.

Decision: Accept (Poster)