SlowGuess/NAACL_2022_OCR huggingface dataset:
Answer: Humidity is the percent of water in the air. The reason that  $100\%$  humidity would not mean water everywhere is due to the fact that the percentage you are referring to only suggests the amount of water the air can hold. [EXAMPLE]
Rate the below candidate examples (1=worst, 5=best) in terms of their fit with the given question and answer (imagine each choice in the place of the [EXAMPLE] marker)
For example, when it's hot outside air expands meaning their is more room between air particles which means more water can fill into the gaps.
Your rating for Example 1----(worst) O1 O2 O3 O4 O5 (best)
For example, it's a relative humidity, at best.
Your rating for Example 2----(worst)O1 O2 O3 O4 O5best
For example, if the current relative humidity is 65 percent, the air is holding 65 percent of the total amount of moisture that it is capable of holding at that temperature and pressure.
Your rating for Example 3----(worst) O1 O2 O3 O4 O5 (best)
B Variations in Recall@k
Table 4 shows that ColBERT outperforms DPR in various recall@k measures when  $k$  is relatively small (less than 100) and yet underperforms DPR in the average rank. Similar phenomenon occurs with EGRET-Books3-only and EGRET-ELI5-only too. We further computed the recall@k for all these models when  $k$  is very large (close to 10,000). Fig. 3 and Fig. 4 show that despite their lower recall@k when  $k$  is relatively small, DPR and EGRET-ELI5-only yield higher recall@k when  $k$  is relatively large, compared to ColBERT and EGRET-Books3-only respectively. With better performance at the long tail (when  $k$  is relatively large), DPR and EGRET-ELI5-only also produce lower average ranks in Table 4.
Figure 3: ColBERT gives higher recall@k compared to DPR when  $k$  is relatively small but lower recall@k when  $k$  is larger
Figure 4: EGRET-Books3-only gives higher recall@k compared to EGRET-ELI5-only when  $k$  is relatively small but lower recall@k when  $k$  is larger
Table A1: Different types of annotated examples in the three datasets. The anchor and example are highlighted.