SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yushi Bai $^{1*}$ , Jiahao Ying $^{2*}$ , Yixin Cao $^{2}$ , Xin Lv $^{1}$ , Yuze He $^{1}$ , Xiaozhi Wang $^{1}$ , Jifan Yu $^{1}$ , Kaisheng Zeng $^{1}$ , Yijia Xiao $^{3}$ , Haozhe Lyu $^{4}$ , Jiayin Zhang $^{1}$ , Juanzi Li $^{1}$ , Lei Hou $^{1\boxtimes}$
$^{1}$ Tsinghua University, Beijing, China  $^{2}$ Singapore Management University, Singapore
<sup>3</sup>University of California, Los Angeles, CA, USA
<sup>4</sup>Beijing University of Posts and Telecommunications, Beijing, China
Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets, however, we see two main issues within previous benchmarking pipelines, namely testing leakage and evaluation automation. In this paper, we propose a novel benchmarking framework, Language-Model-as-an-Examiner, where the LM serves as a knowledgeable examiner that formulates questions based on its knowledge and evaluates responses in a reference-free manner. Our framework allows for effortless extensibility as various LMs can be adopted as the examiner, and the questions can be constantly updated given more diverse trigger topics. For a more comprehensive and equitable evaluation, we devise three strategies: (1) We instruct the LM examiner to generate questions across a multitude of domains to probe for a broad acquisition, and raise follow-up questions to engage in a more in-depth assessment. (2) Upon evaluation, the examiner combines both scoring and ranking measurements, providing a reliable result as it aligns closely with human annotations. (3) We additionally propose a decentralized Peer-examination method to address the biases in a single examiner. Our data and benchmarking results are available at: http://lmexam.xlore.cn.
Recently, many large foundation models [1], such as ChatGPT [2], LLaMA [3], and PaLM [4], have emerged with impressive general intelligence and assisted billions of users worldwide. For various users' questions, they can generate a human-like response. However, the answers are not always trustworthy, e.g., hallucination [5]. To understand the strengths and weaknesses of foundation models, various benchmarks have been established [6, 7, 8, 9, 10].
Nevertheless, we see two main hurdles in existing benchmarking methods, as summarized below. (1) Testing leakage. Along with increasing tasks and corpus involved in pre-training, the answer to the testing sample may have been seen and the performance is thus over-estimated. (2) Evaluation automation. Evaluating machine-generated texts is a long-standing challenge. Thus, researchers often convert the tasks into multi-choice problems to ease the quantitative analysis. This is clearly against real scenarios — as user-machine communications are mostly open-ended Question Answering (QA) or freeform QA [11]. On the other hand, due to the existence of a vast number of valid "good" answers, it is impossible to define one or several groundtruth, making similarity-based matching
Figure 1: Overview of our benchmarking method. The left part shows the use of language model as an examiner. The examiner generates questions from various domains, allowing it to probe for comprehensive understanding (knowledge breadth) as well as deep specialization (knowledge depth) through follow-up questions (FQs). It then scores and ranks other models' responses according to its understanding of the subject, providing a reliable evaluation. The right part presents peer-examination, a novel decentralized method that provides fairer evaluation results, which potentially demands higher workload of running multiple LM examiners, compared to running a single LM examiner.
measurements (e.g., Exact Match, ROUGE-L [12], and BERTScore [13]) ineffective [11, 14, 15]. Therefore, recent works target a well-trained evaluator language model (LM) to assess the answer quality in a reference-free manner [16, 17, 18]. However, using LM as an evaluator also presents a problem: What if the evaluator hallucinates and makes wrong judgments during assessment?
As an attempt, our pilot study utilizes GPT-4 [19] to evaluate the correctness of LLaMA [3] on Natural Questions [20], where non-negligible 18 out of 100 judgments are incorrect (cases in Appendix A). We attribute the main reason to the inadequate knowledge of the evaluator itself regarding the questions. A straightforward solution is to use the LM not just as an evaluator to assess the responses, but as a knowledgeable examiner to also formulate questions, which is guaranteed a thorough understanding of the judgments. And, it naturally addresses the testing leakage issue by generating new questions periodically. Yet, relying on a centralized examiner can hardly be considered fair, especially when evaluating the examiner itself — A man who is his own lawyer has a fool for his client.
In this paper, we propose a novel benchmarking framework, Language-Model-as-an-Examiner, to assess current foundation models, mitigating the aforementioned issues. Herein, the language model acts as a knowledgeable examiner that poses questions based on its inherent knowledge and evaluates others on their responses. We devise three strategies to alleviate potential bias:
- Increasing Knowledge Breadth and Depth. In terms of breadth, according to a predefined taxonomy, we select as many diverse domains as possible to generate questions. In terms of depth, to probe models deeply within a specific subfield, we propose a multi-round setting where the evaluator mimics an interviewer, posing more sophisticated follow-up questions based on the interviewee model's preceding responses. We release our dataset, namely LMEexamQA, which is constructed using GPT-4 [19] as an examiner.
- Reliable Evaluation Measurement. We explore two evaluation metrics, namely Likert scale scoring and Ranking, offering a more comprehensive evaluation result. The results from both metrics correlate closely with human annotations, significantly outperforming all previous metrics.
- Peer-examination Mechanism. To avoid the potential bias arising from a single model as examiner, we propose a decentralized evaluation setting where all participating models are invited to be the examiner and assess each other.
In experiments, our benchmarking pipeline yields fruitful results on 8 popular foundation models. We also demonstrate that peer-examination can generate a more diverse set of questions for knowledge probing and balance the biases from individual evaluator models, ultimately leading to a more equitable evaluation outcome.
Benchmarks for Foundation Models. Various benchmarks have been proposed to assess foundation models on open-ended question answering, since it is the most natural setting for user-machine interaction in real scenarios. Some prominent such benchmarks include MS MARCO [21], SQuAD [22, 23], Natural Questions [20], WebQuestions [24] and OpenBookQA [25]. On the other hand, there exist a limited number of datasets that feature long-form QA. One of the widely-recognized examples is ELI5 [26], which comprises questions that necessitate lengthy descriptive and explanatory answers. One notable limitation of these benchmarks is their reliance on human curation and annotation, which inherently constrains their scalability. Our approach, by comparison, utilizes LMs to construct datasets, offering the advantage of effortless extensibility.
Automating NLG Evaluation. To evaluate machine-generated responses to the questions, several automatic metrics have been adopted, including the F1 score, Exact Match (EM), BLEU [27], ROUGE [12], and METEOR [28]. However, each metric has its own shortcomings, resulting in large discrepancies between the tested and actual performance [14, 29, 30].
To address these issues, well-trained LMs are utilized in NLG evaluation [31, 32, 33, 34]. One mainstream of previous methods is reference-based, where they derive the similarity between the candidate and the reference using an LM. Some prominent metrics in this class include MoverScore [35], BERTScore [13]. These metrics measure the distributional similarity rather than lexical overlap [36], making them appropriate for contexts that require more flexible generation. Recent studies [16, 17, 18, 37, 38, 39, 40, 41] have demonstrated that large language models (LLMs), such as ChatGPT [2], can conduct NLG evaluations in a reference-free manner. They can rate a candidate text (or perform a comparative assessment of two candidates) based on a specified evaluation aspect, displaying a high correlation with human assessments in tasks such as summarization and story generation [42, 43]. In these studies, the evaluations primarily focus on lexical quality aspects, such as coherence and fluency, of a generated text. However, their capability to evaluate crucial aspects in a QA response, including factual correctness and information comprehensiveness, remains uncertain. Moreover, a single evaluator inevitably brings bias to the assessment [17]. Our work aims to resolve these issues by leveraging LM not just as an evaluator but also as an examiner, assessing the performance of other models through self-generated questions, and deploying multiple LM examiners to ensure balanced evaluation.
In this section, we discuss the methodology in language-model-as-an-examiner, including the LMEx-amQA dataset construction, the evaluation metric design, and the peer-examination pipeline.
3.1 Dataset Construction
Question Generation towards Knowledge Breadth. We employ a language model (LM) as an examiner that generates diversifying and high-quality questions across various domains. To ensure wide coverage of knowledge, we choose the Google Trends Categories  ${}^{2}$  as the domain taxonomy, and randomly select  $n$  domains from it. For each domain,we prompt the LM to generate  $m$  distinct questions. Our designed prompt (shown in Appendix B) is formulated to ensure that the generated questions possess three essential characteristics: diversified question forms, varied cognitive levels, and most importantly, assurance that the LM has a comprehensive understanding of the knowledge surrounding the question it poses. Figure 2 shows the distribution of question forms based on their interrogative words, and the distribution of question domains. According to Bloom's taxonomy [44], we divide the questions into 3 categories based on their required cognitive levels, from low to high-level,namely knowledge memorization,knowledge comprehension,and knowledge analysis:
- Knowledge memorization. Questions of such level demand recognition or recollection of certain entities and attributes, such as a person, location, or time.
- Knowledge comprehension. These questions involve demonstrating an understanding of particular instances or concepts, such as "What is ...", "Why ...", and "How ..."
$^{2}$ https://github.com/mat310/google-trends-api/wiki/Google-Trends-Categories.
(a) Question word distribution.
(b) Question domain distribution.
Figure 2: Statistics of generated questions in LMExamQA.
MS [21]	SQuAD2.0 [23]	NQ [20]	ELI5 [26]	Ours	Example questions in our dataset
Analysis	1%	4%	3%	0%	35%	What are the potential short and long-term impacts of divorce on children?
Comprehension	4%	13%	19%	100%	43%	How does towing capacity affect a truck's performance and what factors influence its maximum towing limit?
memorization	95%	83%	78%	0%	22%	Which international organization publishes the World Economic Outlook report?
Table 1: Proportions of each level of questions. MS and NQ are short for MS MARCO and Natural Questions. We also list an example question in LMEexamQA for each category.
- Knowledge analysis. Questions of this type require more advanced cognitive skills and they typically question the impact, comparison, or advantages and disadvantages of a given topic.
By adopting GPT-4 to categorize the questions in LMExamQA and previous open-ended QA datasets into three levels  $^{3}$ , we obtain the distribution with respect to the 3 cognitive levels as listed in Table 1, and show an example for each type of question. Compared with previous datasets, LMExamQA achieves a more balanced distribution across these 3 levels, thus providing a means of quantifying foundational models' proficiency at each cognitive level. Furthermore, LMExamQA includes a larger proportion of questions classified within higher cognitive levels, particularly at the analysis level, indicating a greater level of challenge.
To justify the reliability of the LM examiner as an evaluator on these questions, we employ it to produce a groundtruth answer with the prompt, "Answer the questions accurately and completely, without providing additional details." Upon evaluation by human experts on a random selection of 100 questions, the answers offered by the LM exhibit a  $100\%$  accuracy rate, thereby demonstrating mastery over the questions it generates.
Multi-round Follow-up Question Generation towards Knowledge Depth. To further probe the model's comprehension of a topic in depth, we develop an evaluation procedure involving multiple rounds of follow-up inquiries, drawing inspiration from the interview process. We utilize the LM examiner to construct a series of follow-up inquiries (prompt is shown in the Appendix B). These follow-up questions are specifically tailored to delve deeper into the concepts presented within the model-generated answers from the previous round. As the follow-up questions are dependent on the model's generated answers, we only ask follow-up questions for the correctly answered queries (determined by the LM examiner) and calculate the proportion of correct responses in the subsequent round. We limit the total number of rounds to  $k$  in order to minimize topic deviation that might occur during longer sessions. Note that we only provide the interviewee model with the follow-up question as input, rather than engaging the "exam history"  $^4$ , since most models are not capable of multi-round conversations. We show an example of a follow-up question to Flan-T5 [48]:
3We manually label 100 of the questions in LMExamQA and find a high agreement  $(85\%)$  between human and GPT-4 annotations.
<sup>4</sup>It is important to note that our approach is essentially different with conversational QA [45, 46, 47], which places greater emphasis on evaluating the model's comprehension of the conversational context.
Question: Which material is primarily used to manufacture semiconductor devices?
Flan-T5: Silicon
Follow-up Question: What are the advantages of using silicon as the primary material for semiconductor devices?
Flan-T5: Silicon is a nonrenewable resource, and it is the most abundant element on Earth. X
3.2 Evaluation Metrics
Several methodologies are commonly employed to facilitate human-like evaluation in LMs, prominent among these are the Likert scale scoring [16, 17, 41] and pairwise comparison [38, 41]. For the purposes of our benchmark, we incorporate both Likert scale scoring and a variant of pairwise comparison, namely ranking.
Likert scale scoring functions as an absolute evaluative measure, where the evaluator assigns scores to a given response along predefined dimensions. We establish four distinct dimensions on our dataset: (1) Accuracy. This assesses the extent to which the provided response accurately answers the question. (2) Coherence. This evaluates the logical structure and organization of the response and the degree to which it can be comprehended by non-specialists. (3) Factuality. This examines whether the response contains factual inaccuracies. (4) Comprehensiveness. This gauges whether the response encompasses multiple facets of the question, thus providing a thorough answer. Each of these dimensions is scored on a scale of 1 to 3, ranging from worst to best. We also ask the evaluator to provide an overall score ranging from 1 to 5, based on the scores assigned to the previous 4 dimensions. This score serves as an indicator of the overall quality of the answer.
On the other hand, pairwise comparison operates as a relative evaluation method and is often more discerning compared to scoring. In this process, evaluators are given two responses and are tasked with determining which is superior, taking into account their accuracy, coherence, factuality, and comprehensiveness. Given that there are  $n$  contestant models, we implement a merge sort algorithm to rank the  $n$  responses, involving  $\mathcal{O}(n\log n)$  pairwise comparisons.
3.3 Decentralized Evaluation: Peer-Examination
We introduce a novel decentralized method that incorporates multiple models to serve as examiners, namely Peer-examination (illustrated in the right part of Figure 1), since relying only on one centralized model as the examiner introduces the following potential drawbacks to the benchmarking process. (1) Coverage of generated questions: The examiner may not have a holistic understanding of certain domain knowledge. As a result, the examiner may struggle to propose questions that examine in detail on these areas, which in turn renders the scope of generated questions insufficient. (2) Potential bias during evaluation: The model itself may have a bias during evaluation. The bias can manifest as a preference for certain types of responses or a predisposition towards perspectives irrelevant to the quality of the responses, such as response length or linguistic style. For example, [17] shows that GPT-4 [19] prefers ChatGPT [2] summaries compared to human-written summaries. Such biases may result in unfair ranking assessment outcomes.
To mitigate these issues, during peer-examination, each model is assigned the role of an examiner separately. As examiners, they are responsible for posing questions and evaluating the answers provided by the other models. We then combine the evaluation results from each of these models by voting, and obtain a final result. This approach leverages the collective expertise and diverse perspectives of all models to improve the coverage of questions as well as ensure fairer assessments.
To demonstrate the effectiveness of our Language-model-as-an-examiner framework, we first employ GPT-4 [19] as the examiner for a centralized evaluation, since it exhibits a broad understanding of knowledge [9, 49, 50] and a precise judgmental ability [16, 17]. In peer-examination, we also employ Claude (Claude-instant) [51], ChatGPT [2], Bard [52], and Vicuna-13B [38] as LM examiners.
ROUGE-1	ROUGE-2	ROUGE-L	BLEU	BERTScore	GPT-4
Overall Score
Spearman (ρ)	0.197	0.211	0.158	0.318	0.219	0.633
Kendall (τ)	0.147	0.159	0.118	0.241	0.164	0.554
Pairwise comparison
Accuracy	0.473	0.533	0.530	0.437	0.487	0.853
Table 2: LM examiner's correlation with human annotations, compared with previous metrics.
Single (0-shot / 5-shot)	Multi
All	memorization	Comprehension	Analysis	All
Models without SFT
BLOOMZ [53]	20.7 / 25.5	63.5 / 58.5	13.7 / 23.2	3.8 / 8.6	17.5
Flan-T5 [54]	17.0 / 26.0	49.8 / 62.7	11.2 / 21.4	4.6 / 9.7	15.1
Flan-UL2 [54]	15.5 / 24.8	51.3 / 64.4	9.2 / 20.1	2.2 / 7.0	17.0
GLM-130B [55]	14.9 / 33.3	45.9 / 60.7	9.1 / 31.9	3.7 / 18.7	18.6
LLaMA-13B [3]	29.5 / 54.3	72.3 / 83.9	23.7 / 53.2	11.2 / 38.1	43.1
LLaMA-65B [3]	38.5 / 62.9	78.2 / 88.2	34.0 / 62.7	20.4 / 48.1	53.2
Fine-tuned Models
Vicuna-13B [38]	96.5	89.9	98.27	98.3	92.4
ChatGPT [2]	99.3	97.7	99.71	99.8	100
Table 3: Percentage (\%) of full-mark answers on LMEexamQA. We show the 0-shot and 5-shot performance for models without SFT, with both results being separated by  ${}^{\prime \prime }/{}^{\prime \prime }$  .
Figure 3: Scores on each aspect. We take the 5-shot performance for models without SFT.
4.1 Metric Evaluation
To verify the reliability of our method for scoring and comparison based assessment, we perform metric evaluation. We conduct human evaluations on machine-generated responses. These evaluations are quantified using a 1-5 Likert scale for the overall score, and we let annotators to rank different responses for each question based on their holistic quality. We collect 300 annotations across 100 questions from LMExamQA. For each question, we randomly select 3 of the model responses, and obtain 3 scoring annotations and 3 pairwise comparison results. For Likert scoring, we calculate Spearman's  $\rho$  and Kendall's  $\tau$  between the overall scores given by the automatic metrics and human experts; for ranking, we compute the accuracy of pairwise comparisons offered by the automatic metrics, according to the human-labeled comparison results. Then we compare the LM examiner, GPT-4 [19], with previous automatic metrics, including ROUGE-1, ROUGE-2, ROUGE-L [12], BLEU [27], BERTScore [13] (F1), and report their correlation with human judgments in Table 2. We observe that employing GPT-4 [19] as an examiner results in a much higher correlation with human annotations compared to prior metrics. More profoundly, GPT-4's pairwise comparison achieves an agreement of over  $85\%$  with human's.
4.2 Centralized Benchmarking Results
Experiment Setup. We conduct a centralized benchmarking with GPT-4 [19] as the examiner. Following the method in Section 3, we construct the LMExamQA dataset with GPT-4, where we set  $n = 1,000$  domains and  $m = 10$  questions for each domain, resulting in a total of 10,000 questions. We evaluate 8 popular and open-access foundation models on our LMExamQA dataset, including BLOOMZ (the 176B model) [53], Flan-T5 (the XXL model, 11B) [54], Flan-UL2 (20B) [54], GLM-130B [55], LLaMA (the 13B model and the 65B model) [3], Vicuna-13B [38], and ChatGPT [2]. These models are categorized based on their training procedure: whether they have undergone Supervised Fine-Tuning (SFT) or not. The first 6 models are trained without SFT<sup>5</sup>, whereas the last 2 models are fine-tuned. For models without SFT, we assess their 0-shot and 5-shot performance. During generation, for the examiner and the subject models, we set the temperature to 0 for reproducibility. More details for reproducing our results are shown in Appendix C.2.
Single-round QA for Knowledge Breath. Table 3 presents the percentage of full-mark answers for each model on LMExamQA. Full-mark answers are defined as responses that receive a rating of 5 on the overall score, and the proportion of such responses is reported for each category of questions. Additionally, Figure 3 provides a radar plot depicting the average scores of models on 5 dimensions;
5Flan-T5 and Flan-UL2 are instruction fine-tuned, but they lack fine-tuning on more real scenario data.
We also conduct ranking evaluation over the 8 models (we only show the few-shot performance for models without SFT). In Figure 4, we visualize the ranking results via a win-rate heatmap (the  $(i,j)$ -th entry denotes the fraction of model  $i$  wins when compared against model  $j$ ) along with each model's average win-rate against all other models. We summarize our key findings.
1. The scaling law on L'MeXamQA. LLaMA-65B significantly outperforms LLaMA-13B across all question categories, adhering to the scaling law of LMs [4, 56].
2. Few-shot leads to more substantial improvement on higher cognitive-level questions. For models without SFT, we observe that 5-shot examples yield an average relative improvement of  $17\%$ ,  $123\%$ , and  $206\%$  on memorization, comprehension, and analysis type questions, respectively. This implies that the model may possess adequate knowledge to answer higher-level questions (e.g., distinguishing between two concepts). However, it may lack the ability to retrieve knowledge from its memory and structure appropriate language to form an answer. Few-shot examples serve to provide demonstrations on how to answer such questions.
3. What does SFT offer? We notice a huge performance gap between LLaMA-13B and Vicuna-13B (Vicuna-13B is fine-tuned on LLaMA-13B with 70k user-shared ChatGPT conversations), mainly on the latter two types of questions. This result suggests that SFT primarily plays a crucial role in aligning LM's responses for task adaptation, rather than enriching the model's knowledge — especially in the context of higher-level questions that demand more sophisticated answers.