# A Survey on Machine Reading Comprehension--Tasks, Evaluation Metrics and Benchmark Datasets
## 4 Benchmark Dataset
### Descriptions of Each Mrc Dataset
#### 4.10.8 MCScript  
MCScript [78] is a large-scale MRC dataset with narrative texts and questions that require reasoning using commonsense knowledge. The dataset focuses on narrative texts about everyday activities, and the commonsense knowledge are required to answer multiple-choice questions based on these texts. The feature of the MCScript dataset is to evaluate the contribution of script knowledge to machine understanding. A script is a series of events (also called scenarios) that describe human behavior. The MCScript dataset also forms the basis of a shared task on commonsense and script knowledge organized at SemEval 2018 [78]. The official web page and CodaLab competition page of the SemEval 2018 Shared Task 11 are available on [[https://competitions.codalab.org/competitions/17184](https://competitions.codalab.org/competitions/17184)]([https://competitions.codalab.org/competitions/17184](https://competitions.codalab.org/competitions/17184)). 4.10.9. OpenBookQA  
OpenBookQA [73] consists of about 6000 elementary level science questions in the form of multi-choice (4957 training sets, 500 validation sets, and 500 test sets). Answering the questions in OpenBookQA requires broad common knowledge. OpenBookQA also requires a deeper understanding of both the topic (in the context of common knowledge) and the language it is expressed in Reference [73]. The baseline model provided by the authors has reached about 50% in this dataset, but many state-of-the-art pre-trained QA methods perform surprisingly even worse [73]. Dataset and leaderboard of OpenBookQA are available on [[https://leaderboard.allenai.org/open_book_qa/](https://leaderboard.allenai.org/open_book_qa/)]([https://leaderboard.allenai.org/open_book_qa/](https://leaderboard.allenai.org/open_book_qa/)). 4.10.10. ReCoRD  
ReCoRD [70] is a large-scale MRC dataset that requires deep commonsense reasoning. Experiments on the ReCoRD show that the performance of the state-of-the-art MRC model lags far behind human performance. The ReCoRD represents the challenge of future research to bridge the gap between human and machine commonsense reading comprehension. The ReCoRD dataset contains more than 120,000 queries from over 70,000 news articles. Each query has been verified by crowd workers [70]. The feature of the data set is that all queries and paragraphs in the records are automatically mined from news articles, which minimizes the artificially induced bias. So most records need deep commonsense reasoning. Since July 2019, the ReCoRD has been added to SuperGLUE as an evaluation suite. The ReCoRD dataset and leaderboard are available on [[https://sheng-z.github.io/ReCoRD-explorer/](https://sheng-z.github.io/ReCoRD-explorer/)]([https://sheng-z.github.io/ReCoRD-explorer/](https://sheng-z.github.io/ReCoRD-explorer/)). 4.10.11. CommonSenseQA  
CommonSenseQA [82] is a MRC dataset that requires different types of commonsense knowledge to predict the correct answer. It contains 12,247 questions. The CommonSenseQA dataset is split into a training set, validation set and, test set. The authors performed two types of splits: \"Random split\" which is the main evaluation split, and \"Question token split\" where each of the three sets has disjoint question concepts [82]. To capture common sense beyond association, the authors of CommonSenseQA extracted multiple target concepts from Conceptnet 5.5 [95] that have the same semantic relationship to a single source concept. Crowd workers were asked to propose multiple-choice questions, mention source concepts, and then distinguished each goal concept. This encouraged crowd workers to ask questions with complex semantics that often require prior knowledge [82]. The dataset and leaderboard of CommonSenseQA are available on [[https://www.tau-nlp.org/commonsenseqa](https://www.tau-nlp.org/commonsenseqa)]([https://www.tau-nlp.org/commonsenseqa](https://www.tau-nlp.org/commonsenseqa)). 4.10.12. WikiReading  
WikiReading [56] is a large-scale machine reading comprehension dataset that contains 18 million instances.