# A Survey on Machine Reading Comprehension--Tasks, Evaluation Metrics and Benchmark Datasets
## 4 Benchmark Dataset
### The Size of Datasets
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Year** & **Datasets** & **Question Size** & **\#Training Questions** & **\#Dev Questions** & **\#Test Questions** & **Percentage of Training Set** \\ \hline
2016 & WikiReading [56] & 18.87M & 16.03M & 1.89M & 0.95M & 84.95\% \\
2016 & BookTest [57] & 14.160,825 & 14.140,825 & 10.000 & 10,000 & 99.86\% \\
2016 & Google MC-AFF [58] & 1,742,618 & 1,727,423 & 7602 & 7593 & 99.13\% \\
2015 & Daily Mail [99] & 997,467 & 897,450 & 64.835 & 53,182 & 88.17\% \\
2016 & Facebook CBT [28] & 687K & 669,343 & 8000 & 10,000 & 97.38\% \\
2018 & ReviewQA [25] & 587,492 & 528,665 & N/A & 58.827 & 89.99\% \\
2015 & CNN1[19] & 387,420 & 380,298 & 3924 & 3198 & 98.16\% \\
2019 & Natural Questions [80] & 332,045 & 307,373 & 7830 & 7842 & 95.15\% \\
2016 & Who-did-What [48] & 147/286 & 127,786 & 10,000 & 10,000 & 86.47\% \\
2018 & SQuAD 2.0 [66] & 151,054 & 130,319 & 11.873 & 8862 & 86.27\% \\
2017 & SearchQA [62] & 140,461 & 99,820 & 13,393 & 272.48 & 71.07\% \\
2018 & CoQA [47] & 127K & 110K & 7K & 10K & 86.61\% \\
2018 & ReCoRD [70] & 202,730 & 100,730 & 10,000 & 10,000 & 83.43\% \\
2016 & NewsQA [54] & 119K & 107K & 6K & 6K & 89.92\% \\
2018 & HotpotQA [67] & 105,374 & 90,564 & 7405 & 7405 & 85.95\% \\
2018 & CicK [71] & 104,919 & 91,344 & 6391 & 7184 & 87.06\% \\
2018 & DuoRC [76] & 100,316 & 70K & 15K & 15K & 70.00\% \\
2016 & SQuAD 1.1 [19] & 107,702 & 87,599 & 10,570 & 9533 & 81.33\% \\
2016 & WikiMoMo [53] & 116K & 96K & 10K & 10K & 82.76\% \\
2018 & CLUTH [69] & 99,433 & 76,850 & 11.067 & 11,516 & 77.29\% \\
2018 & QuAC [65] & 98,275 & 83,568 & 7354 & 7353 & 85.03\% \\
2017 & RACZ [60] & 97,687 & 87,866 & 4887 & 4934 & 89.95\% \\
2019 & DROP [59] & 96,567 & 77,409 & 9536 & 9622 & 80.16\% \\
2017 & TrivisQA-Web [59] & 95,956 & 76,496 & 9951 & 9509 & 79.72\% \\
2018 & PaperQA-T [64] & 84,803 & 77,298 & 3752 & 3753 & 91.15\% \\
2018 & DuoRC-S [64] & 84K & 60K & 12K & 12K & 12K & 70.00\% \\
2018 & PaperQA-L [76] & 80,118 & 71,804 & 4179 & 4135 & 89.62\% \\
2017 & TrivisQA-Wiki [59] & 77,582 & 61,888 & 7993 & 7701 & 79.77\% \\
2017 & Qangaroo-W [49] & 51,318 & 43,738 & 5129 & 2451 & 85.23\% \\
2017 & NarrativeQA [63] & 46,765 & 32,477 & 3461 & 10,557 & 70.02\% \\
2017 & Qusar-T [61] & 43,013 & 37,012 & 3000 & 3000 & 86.05\% \\
2017 & Qusar-S [61] & 37,362 & 31,049 & 3174 & 3139 & 83.10\% \\
2018 & ReicQA [34] & 36K & 29,657 & 3562 & 3567 & 80.62\% \\
2017 & TQA [31] & 26,260 & 15,154 & 5309 & 5797 & 57.71\% \\
2016 & MovieQA [32] & 21,406 & 14,166 & 2844 & 4396 & 66.18\% \\
2018 & McScript [78] & 13,939 & 9731 & 1411 & 2797 & 69.81\% \\
2017 & SqG [64] & 13,679 & 11,679 & 1000 & 1000 & 85.38\% \\
2019 & CommonSenseQA [82] & 12,102 & 9741 & 1221 & 1140 & 80.49\% \\
2019 & DREAM [81] & 10,197 & 6116 & 2040 & 2041 & 59.98\% \\
2018 & OpenBookQA [73] & 9597 & 4957 & 500 & 500 & 83.21\% \\
2018 & ARC-Easy Set [72] & 5197 & 2251 & 570 & 2376 & 43.31\% \\
2015 & WikQA [52] & 3047 & 2118 & 296 & 633 & 69.51\% \\
2018 & ARC-Challenge Set [72] & 2590 & 1119 & 299 & 1172 & 43.20\% \\
2017 & Qangaroo-M [49] & 2508 & 1620 & 342 & 546 & 64.59\% \\
2013 & MCTest-m=c00 [15] & 2000 & 1200 & 200 & 600 & 60.00\% \\
2018 & Estall [54] & 1834 & 1542 & 121 & 171 & 84.08\% \\
2019 & ShARC [38] & 948 & 628 & 69 & 251 & 66.24\% \\
2013 & MCTest-m=c160 [15] & 640 & 280 & 120 & 240 & 43.75\% \\
2018 & ProPar [79] & 488 & 391 & 54 & 43 & 80.12\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The question set size of machine reading comprehension datasets. Figure 22: The size of machine Reading Comprehension datasets: (**a**) Total question size of each dataset. (**b**) Percentages of training sets, development sets and test sets. Because different MRC datasets contain different corpora, we also give details of the corpus used in each MRC dataset, including the size of corpus and the unit of corpus, as well as the size of training set, development set, and testing set. As seen in Table 5, The units of corpus in MRC datasets are various, such as paragraphs, documents, and so forth.