# Question Difficulty Estimation Based on Attention Model for Question Answering
## 5 Conclusions  
In this paper, we have proposed an attention model for question difficulty estimation. The proposed attention model first represents bi-directional relationships between a questionary sentence and information components, and then accumulates the inter-information relationship over the concatenated bi-directional relationships. As a result, the proposed method can model complicated relationships among the questionary sentence and information components.  
The contributions of this paper are three folds. The first is that the proposed model achieves the state-of-the-art performance in this task. It outperforms the existing model and pre-trained language models. The second is that the proposed model predicts the difficulty of high-level questions accurately. It is required to reason over multiple kinds of information components to predict the difficulty of high-level questions. Since the proposed model is designed to consider the complicated relationships among information components, the reasoning is taken place properly in the proposed model. The last is that the proposed method works efficiently and can be applied to any text-based QA tasks. The proposed method is based on the simple attention model and does not require any other pre-training models except the BERT. Furthermore, it is free from the number of information components.  
Through intensive experiments with three well-known QA data sets, it has been shown empirically that the proposed model achieves higher performances than all the previous study and pre-trained language models. Moreover, it is also shown that the proposed attention is essential for accurate prediction of the difficulty level for more difficult questions. Through these experiments, we have proven that the proposed model is plausible for predicting the question difficulty and helps to improve the performances of the question answering.  
Conceptualization, [PERSON]-J.S. and S.-H.Y.; methodology, [PERSON] and [PERSON]; visualization, [PERSON]-[PERSON]; validation, [PERSON]-[PERSON], [PERSON] and [PERSON]; funding acquisition, [PERSON]; writing--original draft preparation, [PERSON] and [PERSON]; writing--review and editing, [PERSON] and [PERSON]. All authors have read and agreed to the published version of the manuscript.  
This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2017-0-01772, Development of QA systems for Video Story Understanding to pass the Video Turing Test) and National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A4A1018607).  
Not applicable because this study is involved with neither humans nor animals.  
Not applicable.  
**Data Availability Statement:** Publicly available datasets were analyzed in this study. The data can be found here: RACE: [[https://www.cs.cmu.edu/~glail/data/race/](https://www.cs.cmu.edu/~glail/data/race/)]([https://www.cs.cmu.edu/~glail/data/race/](https://www.cs.cmu.edu/~glail/data/race/)) (accessed on 1 November  
Figure 6: Performance change model according to the ratio of data sets. The X-axis denotes the ratio of data used to train the proposed model and the Y-axis is F1-score. (**a**) RACE. (**b**) QuAC. (**c**) DramaQA.  
2021), QuAC: [[https://quac.ai](https://quac.ai)]([https://quac.ai](https://quac.ai)) (accessed on 1 November 2021), DramaQA: [[https://dramaqa.snu.ac.kr](https://dramaqa.snu.ac.kr)]([https://dramaqa.snu.ac.kr](https://dramaqa.snu.ac.kr)) (accessed on 1 November 2021).