SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Example: Question: "What's the primary gas in Earth's atmosphere?" Golden Answer: "Nitrogen" Generated Answer: "Oxygen, which makes up about  $78\%$  of the atmosphere."
Explanation: GPT-3.5 might focus on the correct percentage and overlook incorrect mention of "Oxygen" as a primary gas.
- Unknowable Reasons: The evaluator makes an incorrect judgment for an unknowable reason. Even humans cannot figure out why the LLM thinks the generated answer is correct since it has no correlation with the golden answer.
Example: Question: "Who was the first chief minister of West Bengal?" Golden Answer: "Prafulla Chandra Ghosh" Generated Answer: "The first Chief Minister of West Bengal was Dr. Bidhan Chandra Roy."
Explanation: GPT-3.5 takes the generated answer as correct, but Dr. Bidhan Chandra Roy is apparently not Prafulla Chandra Ghosh.
Figure 3: Correlation between the evaluation accuracy of GPT-3.5 and the answer length in tokens across all models.
E.4.3 Length Analysis on QA-Eval
Figure 3 depicts the relationship between GPT-3.5's evaluation accuracy and the number of tokens present in the answers produced by all models. The token count is segmented into six distinct categories: 0-35, 36-70, 71-105, 106-140, 141-175, and 175 and above. The corresponding accuracy for these ranges are 90, 71, 60, 58, 54, and 40 respectively. Additionally, the average token counts for the answers by each model are as follows: FiD (4.8 tokens), GPT-3.5 (31.4 tokens), ChatGPT (41.9 tokens), GPT-4 (39.9 tokens), and BingChat (49.7 tokens).
We can draw several observations: 1. GPT-3.5's evaluation accuracy exhibits an inverse correlation with the length of the answer. As the number of tokens in the answer escalates, the evaluation accuracy diminishes. This could indicate that GPT-3.5 may struggle to accurately evaluate more extended responses, potentially due to challenges in retaining context or comprehending intricate or unfamiliar constructs in longer text spans. 2. Considering the average token counts, FiD, the model that generates the shortest responses on average (4.8 tokens), would predominantly fall into the 0-35 token range where GPT-3.5 has its peak accuracy (90). This observation could imply that GPT-3.5 would exhibit optimal evaluation performance with responses generated by the FiD model. 3. Conversely, models like Bing Chat, which on average yield longer responses (49.7 tokens), would generally fall into the token ranges where GPT-3.5's evaluation accuracy is lower. This can partially explain why GPT-3.5 performs worse than Lexical Matching in NQ-BingChat and TQ-BingChat.
E.5 Enhancing QA-Eval through Prompt Engineering
Table 7 presents the performance of GPT-3.5 evaluator with different prompts on the EVOUNA-NQ dataset. Here are the insights: Directing GPT-3.5 to ignore the background information degrades performance on four datasets with long answers (NQ-GPT35/ChatGPT35/ChatGPT4/BingChat).
Table 12: Specific prompts used in each method for GPT-3.5 on QA-Eval.
Methods	Prompts
Original	Here is a question, a set of golden answers (split with /), an AI-generated answer. Can you judge whether the AI-generated answer is correct according to the question and golden answers, simply answer Yes or No
Ignoring Background	Here is a question, a set of golden answers (split with /), an AI-generated answer. Can you judge whether the AI-generated answer is correct according to the question and golden answers, please only consider the answer itself, ignore the background information. Simply answer Yes or No.
Giving Reasons	Here is a question, a set of golden answers (split with /), an AI-generated answer. Can you judge whether the AI-generated answer is correct according to the question and golden answers. Please make a judgment and give the reason. Your answer must be <Yes or No> | <Reason>
Chain-of-Thoughts	Here is a question, a set of golden answers (split with /), an AI-generated answer. Can you judge whether the AI-generated answer is correct according to the question and golden answers. Please think step by step and make a judgment in the end. You must give your chain of thoughts. Your answer must be <your chain of thoughts>| <Yes or No>. (chain of thoughts and final judgment must be split with 'I')
In-Context-Learning	Here is a question, a set of golden answers (split with /), an AI-generated answer. Can you judge whether the AI-generated answer is correct according to the question and golden answers, simply answer Yes or No. Here are some examples: Example 1: AAA; Example 2: BBB; Example 3: CCC; Example 4: DDD.
Requiring the model to reason its judgments negatively impacts performance across all datasets. The effects of Chain-of-Thoughts and In-Context-Learning vary. For instance, both methods significantly improve performance on four datasets with long answers, but Chain-of-Thoughts shows a substantial decline on the NQ-FiD. This variability suggests that the influence of these techniques depends on the data distribution.
E.6 Does retrieval Help in LLM?
In our quest to determine the impact of retrieval on Large Language Models (LLMs) in an Open-QA setting, we investigate two distinct scenarios. Firstly, we assess the performance of Bing Chat when retrieval is disabled. Secondly, we augment GPT-3.5 with a retrieval mechanism and gauge its effectiveness.
Performance of Bing Chat Without Retrieval In this experiment, we modify the standard prompt fed to Bing Chat by preceding the question  $q$  with the instruction "Please do not search, answer the following question directly:". We choose a sample of 500 questions from the NQ test dataset, filtering out those unsuitable for this setting. The results of this experiment are depicted in the left section of Figure 4.
The data suggests a significant decline in Bing Chat's performance when retrieval is disabled, dropping approximately 15 percentage points from 80.5 to 65.6. This is comparable to the performance of GPT-3.5 (65.0), which lacks a retrieval mechanism. This substantial decline implies that the retrieval
Figure 4: The performance of Bing Chat and GPT-3.5 on NQ set with or without retrieval.
Table 13: Performance of BERT-Score and BLEURT on the EVOUNA. In each cell, the left is the accuracy while the right is the Macro-F1.
NQ-FiD	NQ-GPT35	NQ-ChatGPT35	NQ-ChatGPT4	NQ-BingChat
Lexical Matching	89.7/92.0	84.8/86.9	80.3/84.9	82.5/87.6	82.3/87.8
BERT-Score	75.1/83.5	69.5/77.6	72.8/81.2	76.0/84.3	67.5/77.6
BELURT	84.4/89.4	74.1/83.1	78.0/86.3	82.2/89.6	82.8/90.0
GPT-3.5	93.6/95.3	84.1/87.2	82.2/86.9	80.9/86.9	69.5/77.2
GPT-4	94.5/96.0	91.0/93.2	90.6/93.7	92.0/95.1	91.4/94.7
Another Human	96.3/97.4	96.8/97.8	95.6/96.5	96.6/97.9	95.5/97.2
on EVOUNA-NaturalQuestions
TQ-FiD	TQ-GPT35	TQ-ChatGPT35	TQ-ChatGPT4	TQ-BingChat
Lexical Matching	91.8/94.7	92.3/94.8	92.3/95.2	91.1/94.8	89.8/94.1
BERT-Score	65.5/75.1	75.7/84.1	80.8/88.4	93.5/90.5	80.4/88.3
BELURT	88.1/92.9	82.1/89.4	85.2/91.5	88.8/93.8	90.7/95.0
GPT-3.5	95.7/97.3	91.2/94.2	92.5/95.5	92.4/95.7	80.9/88.2
GPT-4	97.3/98.3	97.5/98.4	97.5/98.5	97.8/98.8	96.5/98.1
Another Human	100/100	99.4/99.6	98.8/99.2	99.8/99.2	99.8/99.9
on EVOUNA-TriviaQA
component significantly boosts the performance of the LLM underpinning Bing Chat in an Open-QA context.
Augmenting GPT-3.5 with a Retrieval Mechanism For the second scenario, we employ the same Dense Retriever used in the DPR+FiD model (referenced in Section3.2) to fetch relevant passages from the database for a given question. We then integrate these passages into the prompt supplied to GPT-3.5. The prompt reads: "We have a question here: QUESTION. Now, we have the following relevant passages: PASSAGE 1; PASSAGE 2; PASSAGE 3; PASSAGE 4; PASSAGE 5. Please answer the question referring to the above passages."
The results of this experiment, shown in the right section of Figure 4, reveal a slight decrease in performance with the addition of retrieval, falling from 69.6 to 69.1. This suggests that simply injecting retrieved passages into the prompts, without any form of thoughtful adaptation, does not contribute positively to the LLM's performance in an Open-QA setting.
E.7 BLEURT Evaluator
Table 14: Open-QA and QA-Eval results of Atlas and Chat-Llama2 on 500 samples of NQ. In each cell, the left is the accuracy while the right is the Macro-F1.
NQ-Atlas	NQ-ChatLlama2
Lexical Matching	92.6/91.7	93.4/87.9
BERT-Score	67.1/72.5	73.4/61.9
GPT-3.5	93.2/92.7	77.6/43.0
GPT-4	93.2/92.6	91.3/85.1
Human Score on NQ-Atlas: 47.9; Human Score on NQ-ChatLlama2: 29.7
Table 15: Error results of Eval-Models on the EVOUNA. In each cell, the left is the error rates while the right is the times compared with another human results.
NQ-FiD	NQ-GPT35	NQ-ChatGPT35	NQ-ChatGPT4	NQ-BingChat
Lexical Matching	10.3/2.8x	15.2/4.8x	19.7/4.5x	17.5/5.1x	17.7/3.9x
BERT-Score	24.9/6.7x	30.5/9.5x	27.2/6.2x	24.0/7.1x	32.5/7.2x
GPT-3.5	6.4/1.7x	15.9/5.0x	17.8/4.0x	19.1/ 5.6x	30.5/ 6.8x
GPT-4	5.5/1.5x	9.0/2.8x	9.4/2.1x	8.0/2.4x	8.6/1.9x
Another Human	3.7/1.0x	3.2/1.0x	4.4/1.0x	3.4/1.0x	4.5/1.0x
on EVOUNA-NaturalQuestions
TQ-FiD	TQ-GPT35	TQ-ChatGPT35	TQ-ChatGPT4	TQ-BingChat
Lexical Matching	8.2/∞	7.7/6.4xx	7.7/6.4x	8.9/44.5x	10.2/51.0x
BERT-Score	34.5/∞	24.3/20.3x	19.2/16.0x	6.5/32.5x	19.6/98.0x
GPT-3.5	4.3/∞	8.8/7.3x	7.5/6.3x	7.6/38.0x	19.1/95.5x
GPT-4	2.7/∞	2.5/4.2x	2.5/2.1x	2.2/11x	3.5/17.5x
Another Human	0/∞	0.6/1.0x	1.2/1.0x	0.2/1.0x	0.2/1.0x
on EVOUNA-TriviaQA
We also conducted a QA-Eval analysis on a more recent Neural-Evaluation model, BLEURT [Sellam et al., 2020]. Similar to BERT-Score, we applied a threshold to BLEURT to make it suitable for QA-Eval. In this work, we set the threshold at 0.2 based on observed distributions. The results are shown in the Table 13. Although BLEURT outperforms BERT-Score on most datasets, it still lags significantly behind the performance of Lexical Matching, GPT-3.5 and human, especially in terms of Macro-F1.
E.8 Additional Open-QA Models
We have conducted experiments on more transparent Open-QA models, including Atlas [Izacard et al., 2022], Llama-2 [Touvron et al., 2023], Chat-Llama-2 [Touvron et al., 2023] on 500 samples on NQ test subset. During our experiments, we notice that the base version of LLaMa-2 occasionally deviated from our instructions. As a result, we chose to proceed with Chat-Llama-2 for a more consistent evaluation. The results are shown in Table 14.
It's evident from the results that the performance of ATLAS and Chat-Llama2 is somewhat below the models discussed in our paper. Moreover, the evaluators' performance on NQ-Atlas and NQChatLlama2 is consistent with the trends observed for the models we initially discussed.
Hashimoto et al. [2019] have also studied the correlations between human evaluation and automated metrics in NLP. However, there are key differences that set our research apart. First, We only discuss the Open-QA task, underscoring the nuances and challenges specific to this domain, while their research casts a wider net, aiming to bridge the gap between human and automated evaluation methods across various natural language generation tasks. Second, there are different emphases on Human Evaluation, We introduce the EVOUNA dataset, which is enriched with human-annotated results, providing a fresh perspective on evaluation in the Open-QA domain, while They advocate for a unified framework that correlates human judgments with statistical metrics, offering a holistic
approach to evaluation in NLP. Last, we present the QA-Eval task and the EVOUNA dataset, tailored specifically for evaluating Open-QA systems, while heir research offers a comprehensive framework designed for a broader spectrum of natural language generation tasks.