# An Idiom Reading Comprehension Model Based on Multi-Granularity Reasoning and Paraphrase Expansion
## 2 Related Work  
Machine reading comprehension is not a new natural language processing topic; as early as 1977, [PERSON] [5] et al. built the question and answer program QUALM to comprehend stories, bringing contextual comprehension under study for the first time. In 1999, [PERSON] [6] et al. created a reading comprehension system containing a development set and a test set of 60 discourse items, each using reading material from grades 3 to 6. At that time, Deep Read's baseline system obtained 30-40% accuracy in eleven subtasks, while most machine reading comprehension systems of the same period were rule-based [7] and statistically based models [8]. [PERSON] [9] et al. designed a rule-based machine reading comprehension system, Quarc, containing multiple heuristic rules and morphological analysis as a means to provide answers. In 2010, [PERSON] [10] et al. started to use machine learning methods for machine reading, combining techniques such as bootstrap sampling, Markov logic, and self-supervised learning. However, the lack of high-quality, large-scale reading comprehension datasets in this area and the high reliance on manual rule sets or features constructed by many human beings led to a long period during which research in the field of machine reading comprehension did not receive sufficient attention. The above situation was resolved in 2015 with the emergence of neural machine reading comprehension and new large-scale benchmark datasets. On the one hand, deep-learning-based machine reading comprehension began to show its unique advantage in capturing contextual information, and on the other hand, the availability of datasets such as the CNN/Daily Mail dataset [11], Stanford Question-Answering Dataset (SQuAD) [12], and MS MARCO [13] dataset provided data support for deep neural network architectures and evaluation testbeds. Deep learning does not rely on linguistic feature tools, does not require manual feature construction, and has more robust generalization than traditional methods. [PERSON] [8] et al. proposed Attentive Reader, a supervised attentional bidirectional long- and short-term memory (LSTM) model based on the CNN/Daily Mail dataset in the form of a completion filler, with an accuracy of 63.8%, obtaining a performance improvement of more than 10% compared to the traditional model. The following year, [PERSON] [14] et al. introduced a bilinear layer to replace the tanh layer, thus further improving the accuracy to over 70%. Although neural models have been successful in NLP tasks, their performance improvement in MRC accuracy is still insufficient, a significant reason being that the current datasets are negligible for most supervised NLP tasks (except machine translation), and deep neural networks usually have a large number of parameters. Smaller training datasets lead to the appearance of overfitting. As a result, early neural models for NLP tasks were relatively shallow, often containing only one to three layers. A large body of research in recent years has shown that pre-trained models (PTMs) based on large corpora can learn generic linguistic representations and benefit downstream NLP tasks while avoiding training models from scratch. With the development of computational power, the emergence of deep models (such as Transformer [15]) and the enhancement of training techniques have allowed PTMs to evolve from shallow to deep. In 2018, ELMo [16] used bidirectional LSTM as a feature extractor to learn deep contextual word representations. A new paradigm of pre-training + fine-tuning gradually formed and opened a new era of PLMs. The current deep PLM has shown a powerful ability to learn universal language representations. The Transformer [17] structure was widely used in a series of large PLMs such as BERT [18] and OpenAI [15] in subsequent studies due to its more powerful feature extraction ability. At the same time, the emergence of the attention mechanism allowed the model to learn to focus on certain information.