# A Survey on Machine Reading Comprehension--Tasks, Evaluation Metrics and Benchmark Datasets
## 4 Benchmark Dataset
### The Availability of Datasets, Leaderboards and Baselines  
The release of MRC baseline projects and leaderboards can help the researchers evaluate the performance of their models. In this section, we try to find all the MRC dataset download links, leaderboards, and baseline projects. As shown in Table 7, all the download links of MRC datasets are available except PaperQA [76]. Most of the datasets provide leaderboards and baseline projects except only 19.3% of the datasets. We have published all the download links, leaderboards, and the baseline projects on our website [[https://mrc-datasets.github.io/](https://mrc-datasets.github.io/)]([https://mrc-datasets.github.io/](https://mrc-datasets.github.io/)).  
### The Availability of Datasets, Leaderboards and Baselines  
The release of MRC baseline projects and leaderboards can help the researchers evaluate the performance of their models. In this section, we try to find all the MRC dataset download links, leaderboards, and baseline projects. As shown in Table 7, all the download links of MRC datasets are available except PaperQA [76]. Most of the datasets provide leaderboards and baseline projects except only 19.3% of the datasets. We have published all the download links, leaderboards, and the baseline projects on our website [[https://mrc-datasets.github.io/](https://mrc-datasets.github.io/)]([https://mrc-datasets.github.io/](https://mrc-datasets.github.io/)).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Year** & **Datasets** & **Dataset Availability** & **Leaderboard Availability** & **Baseline Availability** \\ \hline
2019 & CommonSenseQA [82] & \(\surd\) & \(\surd\) & \(\surd\) \\
2018 & MCScript [78] & \(\surd\) & \(\surd\) & \(\times\) \\
2018 & OpenBookQA [73] & \(\surd\) & \(\surd\) & \(\times\) \\
2018 & RecoRD [70] & \(\surd\) & \(\surd\) & \(\times\) \\
2018 & ARC-Challenge Set [72] & \(\surd\) & \(\surd\) & \(\surd\) \\
2018 & ARC-Easy Set [72] & \(\surd\) & \(\surd\) & \(\surd\) \\
2018 & CLOTH [69] & \(\surd\) & \(\surd\) & \(\surd\) \\
2016 & Facebook CBT [28] & \(\surd\) & \(\times\) & \(\surd\) \\
2016 & NewsQA [54] & \(\surd\) & \(\times\) & \(\times\) \\
2018 & ProPara [79] & \(\surd\) & \(\surd\) & \(\times\) \\
2017 & RACE [60] & \(\surd\) & \(\surd\) & \(\surd\) \\
2016 & SQuAD 1.1 [19] & \(\surd\) & \(\surd\) & \(\surd\) \\
2017 & TriviaQA-Wiki [59] & \(\surd\) & \(\surd\) & \(\surd\) \\
2017 & TriviaQA-Web [59] & \(\surd\) & \(\surd\) & \(\surd\) \\
2019 & DROP [50] & \(\surd\) & \(\surd\) & \(\surd\) \\
2017 & NarrativeQA [63] & \(\surd\) & \(\times\) & \(\surd\) \\
2019 & ShARC [38] & \(\surd\) & \(\surd\) & \(\times\) \\
2018 & CoQA [47] & \(\surd\) & \(\surd\) & \(\surd\) \\
2019 & DREAM [81] & \(\surd\) & \(\surd\) & \(\surd\) \\
2018 & QuAC [65] & \(\surd\) & \(\surd\) & \(\surd\) \\
2013 & MCTest-mc160 [15] & \(\surd\) & \(\surd\) & \(\surd\) \\
2013 & MCTest-mc500 [15] & \(\surd\) & \(\surd\) & \(\surd\) \\
2015 & WikiQA [52] & \(\surd\) & \(\times\) & \(\times\) \\
2018 & CliCR [71] & \(\surd\) & \(\times\) & \(\surd\) \\
2018 & PaperQA([PERSON] et al.)[77] & \(\surd\) & \(\times\) & \(\times\) \\
2018 & PaperQA-L [76] & \(\times\) & \(\times\) & \(\times\) \\
2018 & PaperQA-T [76] & \(\times\) & \(\times\) & \(\times\) \\
2018 & ReviewQA [25] & \(\surd\) & \(\times\) & \(\times\) \\
2017 & SciQ [64] & \(\surd\) & \(\times\) & \(\times\) \\
2016 & WikiMovies [53] & \(\surd\) & \(\times\) & \(\surd\) \\
2016 & BookTest [57] & \(\surd\) & \(\times\) & \(\times\) \\
2015 & CNN [19] & \(\surd\) & \(\times\) & \(\surd\) \\ \hline \hline \end{tabular}
\end{table}
Table 7: The availability of datasets, leaderboards and baselines.