Machine Reading Comprehension (MRC) is a challenging NLP research field with wide real world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed the human performance on many datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need of improving existing datasets, evaluation metrics  and models to move the MRC models toward 'real' understanding. To address this lack of comprehensive survey of existing MRC tasks, evaluation metrics  and datasets, herein, (1) we analyzed 57 MRC tasks and datasets ; proposed a more precise classification method of MRC tasks with 4 different attributes  (2) we summarized 9 evaluation metrics of MRC tasks and (3) 7 attributes and 10 characteristics of MRC datasets; ( 4 ) We also discussed some open issues in MRC research and highlight some future research directions. In addition, to help the community, we have URLanized, and published our data on a companion URL where MRC researchers could directly access each MRC dataset, papers, baseline projects and browse the leaderboard.
Machine Reading Comprehension (MRC) is a challenging Natural Language Processing(NLP) research field with wide real-world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need for improving existing datasets, evaluation metrics , and models to move current MRC models toward "real" understanding. To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics , and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method of MRC tasks with 4 different attributes ; (2) we summarized 9 evaluation metrics of MRC tasks , 7 attributes and 10 characteristics of MRC datasets; ( 3 ) We also discuss key open issues in MRC research and highlighted future research directions. In addition, we have URLanized, and published our data on the companion URL where MRC researchers could directly access each MRC dataset, papers, baseline projects , and the leaderboard.
R
NLP
Natural Language Processing(NLP)
R
real world
real-world
D
the
R
many
various benchmark
R
of
for
A
,
R
the
current
R
'real'
"real"
R
this
the current
A
,
R
analyzed
analyze
R
; proposed
and propose
A
;
R
and (3)
,
R
4
3
R
discussed some
discuss key
R
highlight some
highlighted
D
to help the community,
R
a
the
R
and browse
, and