Although over 95 million people in the world speak the Vietnamese language, there are not any large and qualified datasets for automatic reading comprehension . In addition , machine reading comprehension  for the health domain offers great potential for practical applications; however, there is still very little machine reading comprehension research in this domain. In this study, we present ViNewsQA as a new corpus for the low-resource Vietnamese language to evaluate models of machine reading comprehension . The corpus comprises 10,138 human-generated question-answer pairs. Crowdworkers created the questions and  answers based on a set of over 2,030 online Vietnamese news articles from the VnExpress news website , where the answers comprised spans extracted from the corresponding articles. In particular, we developed a process of creating a corpus for the Vietnamese language . Comprehensive evaluations demonstrated that our corpus requires abilities beyond simple reasoning  such as word matching , as well as demanding difficult reasoning similar to inferences based on single-or-multiple-sentence information. We conducted experiments using state-of-the-art methods for machine reading comprehension to obtain the first baseline performance measures, which will be compared with further models' performances. We measured human performance based on the corpus and compared it with several strong neural models. Our experiments showed that the best model was BERT, which achieved an exact match score of 57.57\% and F1-score of 76.90 \% on our corpus. The significant difference between humans and the best model (F1-score of 15.93\% ) on the test set of our corpus indicates that improvements in ViNewsQA can be explored in future research . Our corpus is freely available on our website in order to encourage the research community to make these improvements.
Large-scale and high-quality corpora are necessary for evaluating machine reading comprehension models on a low-resource language like Vietnamese. Besides , machine reading comprehension (MRC) for the health domain offers great potential for practical applications; however, there is still very little MRC research in this domain. This paper presents ViNewsQA as a new corpus for the Vietnamese language to evaluate healthcare reading comprehension models . The corpus comprises 22,057 human-generated question-answer pairs. Crowd-workers create the questions and their answers based on a collection of over 4,416 online Vietnamese healthcare news articles , where the answers comprise spans extracted from the corresponding articles. In particular, we develop a process of creating a corpus for the Vietnamese machine reading comprehension . Comprehensive evaluations demonstrate that our corpus requires abilities beyond simple reasoning , such as word matching and demanding difficult reasoning based on single-or-multiple-sentence information. We conduct experiments using different types of machine reading comprehension methods to achieve the first baseline performances, compared with further models' performances. We also measure human performance on the corpus and compared it with several powerful neural network-based and transfer learning-based models. Our experiments show that the best machine model is ALBERT, which achieves an exact match score of 65.26\% and an F1-score of 84.89 \% on our corpus. The significant differences between humans and the best-performance model (14.53\% of EM and 10.90\% of F1-score ) on the test set of our corpus indicate that improvements in ViNewsQA could be explored in the future study . Our corpus is publicly available on our website for the research purpose to encourage the research community to make these improvements.
R
Although over 95 million people in the world speak the Vietnamese language, there are not any large and qualified datasets for automatic reading comprehension . In addition
Large-scale and high-quality corpora are necessary for evaluating machine reading comprehension models on a low-resource language like Vietnamese. Besides
A
(MRC)
R
machine reading comprehension
MRC
R
In this study, we present
This paper presents
D
low-resource
R
models of machine reading comprehension
healthcare reading comprehension models
R
10,138
22,057
R
Crowdworkers created
Crowd-workers create
A
their
R
set of over 2,030 online Vietnamese news articles from the VnExpress news website
collection of over 4,416 online Vietnamese healthcare news articles
R
comprised
comprise
R
developed
develop
R
language
machine reading comprehension
R
demonstrated
demonstrate
A
,
R
, as well as
and
D
similar to inferences
R
conducted experiments using state-of-the-art methods for
conduct experiments using different types of
R
to obtain
methods to achieve
R
performance measures, which will be
performances,
R
measured human performance based
also measure human performance
R
strong neural
powerful neural network-based and transfer learning-based
R
showed
show
R
model was BERT, which achieved
machine model is ALBERT, which achieves
R
57.57\% and
65.26\% and an
R
76.90
84.89
R
difference
differences
R
best model (F1-score of 15.93\%
best-performance model (14.53\% of EM and 10.90\% of F1-score
R
indicates
indicate
R
can
could
R
future research
the future study
R
freely
publicly
R
in order
for the research purpose