What does BERT Learn from A rabic Machine Reading Comprehension Datasets? (2021)

Abstract: In machine reading comprehension tasks, a model must extract an answer from the available context given a question and a passage. Recently, transformer-based pre-trained language models have achieved state-of-the-art performance in several natural language processing tasks. However, it is unclear whether such performance reflects true language understanding. In this paper, we propose adversarial examples to probe an Arabic pre-trained language model (AraBERT), leading to a significant performance drop over four Arabic machine reading comprehension datasets. We present a layer-wise analysis for the transformer’s hidden states to offer insights into how AraBERT reasons to derive an answer. The experiments indicate that AraBERT relies on superficial cues and keyword matching rather than text understanding. Furthermore, hidden state visualization demonstrates that prediction errors can be recognized from vector representations in earlier layers.

## introduction the rise of pre-trained language models (lm), including elmo @xcite , bert @xcite , gpt @xcite , and roberta @xcite , has shifted the field of natural language processing (nlp) from training complex neural models from scratch to fine-tuning on downstream tasks. these contextual embeddings are generated by training on a general language model task, such that these vectors encode the structure and meaning of each word in the context. this approach has replaced the effort needed to tailor complex neural models with a simple finetuning using a relatively small amount of data for a specific task. as the use of lms is growing in various nlp tasks, it is important to understand how they achieve high performance. this understand-ing can be used to leverage their strengths, identify their vulnerabilities, and rectify their weaknesses. morphologically rich languages (mrl) are languages in which significant information is expressed morphologically via word variation, such as in arabic, hebrew, and turkish, rather than syntactically, as in english @xcite . the diversity of word forms requires machine learning models to cope with the extreme data sparseness that follows from the complex word structure, which further complicates automatic semantic identification. furthermore, as each word can appear in multiple forms, some are unseen in the annotated data, which amplifies the out-of-vocabulary (oov) problem. recently, a considerable number of research studies have sought to analyze different lms in the context of the english language, while other languages have received limited attention. most studies, including @xcite , @xcite , and @xcite , use black-box external probes to assess model robustness and to identify its weaknesses. the present study follows this direction and proposes adversarial examples as external probes to investigate what underpins the reported performance of arabic pre-trained lms. this paper focuses on the task of arabic machine reading comprehension (mrc), which requires the model to extract a span from the text as the answer, given a question and a passage. our approach involves probing the fine-tuned arabert (version 0.1) @xcite model by appending distractors containing keywords to examples in the test set and, in turn, evaluating model robustness. the goal is to mislead the model into making incorrect predictions by adding uninterpretable information to the passage, noting that humans can still easily find the correct answer. to the best of our knowledge, no previous research study source: tydi qa dataset context: ‫ﺗ‬ ‫ﺸ‬ ‫ﯿ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﻲ‬ ‫ﺑ‬ ‫ﺮ‬ ‫ﯾ‬ ‫ﻤ‬ ‫ﻦ‬ ‫إ‬ ‫ﻋ‬ ‫ﺎ‬ ‫ر‬ ‫ﺗ‬ ‫ﮫ‬ ‫د‬ ‫ي‬ ‫ﺣ‬ ‫ﯿ‬ ‫ﺚ‬ ‫ﺳ‬ ‫ﯿ‬ ‫ﺘ‬ ‫ﻲ‬ 2012 ‫ﻋ‬ ‫ﺎ‬ ‫م‬ ‫إ‬ ‫ﺷ‬ ‫ﺮ‬ ‫ا‬ ‫ﻛ‬ ‫ﮫ‬ ‫ا‬ ‫ﻟ‬ ‫ﺒ‬ ‫ﻠ‬ ‫ﺠ‬ ‫ﯿ‬ ‫ﻜ‬ ‫ﻲ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺎ‬ ‫ﻧ‬ ‫ﺸ‬ ‫ﺴ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﺛ‬ ‫ﻢ‬ ‫ا‬ ‫ﻧ‬ ‫ﻀ‬ ‫ﻢ‬ ‫ﻓ‬ ‫ﻲ‬ 11 ‫ﻣ‬ ‫ﺘ‬ ‫ﻰ‬ ‫ﻧ‬ ‫ﺎ‬ ‫د‬ ‫ر‬ ‫ﻛ‬ ‫ﺎ‬ ‫ن‬ ‫ﺟ‬ ‫ﯿ‬ ‫ﻨ‬ ‫ﻚ‬ ‫ﺗ‬ ‫ﻢ‬ ‫إ‬ ‫ﻟ‬ ‫ﻰ‬ ‫د‬ ‫ي‬ ‫ﻧ‬ ‫ﺎ‬ ‫د‬ ‫ي‬ ‫و‬ ‫ﻓ‬ ‫ﺎ‬ ‫ز‬ ‫ا‬ ‫ﻧ‬ ‫ﻀ‬ ‫ﻢ‬ -‫ﻣ‬ ‫ﺴ‬ ‫ﯿ‬ ‫ﺮ‬ ‫ﺗ‬ ‫ﮫ‬ ‫ﺑ‬ ‫ﻠ‬ ‫ﻘ‬ ‫ﺐ‬ ‫ﺣ‬ ‫ﯿ‬ ‫ﺚ‬ ‫ا‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺒ‬ ‫ﺎ‬ ‫ر‬ ‫ﯾ‬ ‫ﺎ‬ ‫ت‬ ‫ا‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺤ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻓ‬ ‫ﯿ‬ ‫ﻦ‬ ‫ﻓ‬ ‫ﯿ‬ ‫ﺮ‬ ‫د‬ ‫ر‬ ‫ﺗ‬ ‫ﻤ‬ ‫ﺖ‬ ، ‫ا‬ ‫ﻹ‬ ‫ﻧ‬ ‫ﺠ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﺰ‬ ‫ي‬ ‫ا‬ ‫ﻻ‬ ‫ﻧ‬ ‫ﺠ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﺰ‬ ‫ي‬ ‫؟‬ ‫ﻛ‬ ‫ﯿ‬ ‫ﻔ‬ ‫ﻦ‬ ‫ﺑ‬ ‫ﺮ‬ ‫و‬ ‫ﯾ‬ ‫ﻦ‬ ‫د‬ ‫و‬ ‫ر‬ ‫ي‬ ‫ﻓ‬ ‫ﻲ‬ ، ‫إ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﻻ‬ ‫ﻋ‬ ‫ﺒ‬ ‫ﺎ‬ ً ‫ﺑ‬ ‫ﺪ‬ ‫أ‬ ‫ﻋ‬ ‫ﺎ‬ ‫د‬ ‫ﯾ‬ ‫ﺎ‬ ً 2010 . ‫ﺑ‬ ‫ﺪ‬ ‫أ‬ ‫د‬ ‫ي‬ ‫ﺑ‬ ‫ﺮ‬ ‫و‬ ‫ﯾ‬ ‫ﻦ‬ ‫ﻣ‬ ‫ﺴ‬ ‫ﯿ‬ ‫ﺮ‬ ‫ﺗ‬ ‫ﮫ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﺟ‬ ‫ﯿ‬ ‫ﻨ‬ ‫ﻚ‬ ، ‫ﺣ‬ ‫ﯿ‬ ‫ﺚ‬ ‫ﻛ‬ ‫ﺎ‬ ‫ن‬ ‫ﻻ‬ ‫ﻋ‬ ‫ﺒ‬ ‫ﺎ‬ ً ‫ﻋ‬ ‫ﺎ‬ ‫د‬ ‫ﯾ‬ ‫ﺎ‬ ً ‫و‬ ‫ﻓ‬ ‫ﺎ‬ ‫ز‬ ‫ﺑ‬ ‫ﻠ‬ ‫ﻘ‬ ‫ﺐ‬ ‫د‬ ‫و‬ ‫ر‬ ‫ي‬ ‫ا‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺤ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻓ‬ ‫ﯿ‬ ‫ﻦ‬ ‫ا‬ ‫ﻟ‬ ‫ﺒ‬ ‫ﻠ‬ ‫ﺠ‬ ‫ﯿ‬ ‫ﻜ‬ ‫ﻲ‬ 2010 -11 . ‫ﻓ‬ ‫ﻲ‬ ‫ﻋ‬ ‫ﺎ‬ ‫م‬ 2012 ، ‫ا‬ ‫ﻧ‬ ‫ﻀ‬ ‫ﻢ‬ ‫إ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﻧ‬ ‫ﺎ‬ ‫د‬ ‫ي‬ ‫ﺗ‬ ‫ﺸ‬ ‫ﯿ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﻲ‬ ‫ا‬ ‫ﻹ‬ ‫ﻧ‬ ‫ﺠ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﺰ‬ ‫ي‬ ، ‫ﺣ‬ ‫ﯿ‬ ‫ﺚ‬ ‫ﺗ‬ ‫ﻢ‬ ‫إ‬ ‫ﺷ‬ ‫ﺮ‬ ‫ا‬ ‫ﻛ‬ ‫ﮫ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ا‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺒ‬ ‫ﺎ‬ ‫ر‬ ‫ﯾ‬ ‫ﺎ‬ ‫ت‬ ‫ﺑ‬ ‫ﺸ‬ ‫ﻜ‬ ‫ﻞ‬ ‫ﻧ‬ ‫ﺎ‬ ‫د‬ ‫ر‬ ‫ﺛ‬ ‫ﻢ‬ ‫ﺗ‬ ‫ﻤ‬ ‫ﺖ‬ ‫إ‬ ‫ﻋ‬ ‫ﺎ‬ ‫ر‬ ‫ﺗ‬ ‫ﮫ‬ ‫إ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﻓ‬ ‫ﯿ‬ ‫ﺮ‬ ‫د‬ ‫ر‬ ‫ﺑ‬ ‫ﺮ‬ ‫ﯾ‬ ‫ﻤ‬ ‫ﻦ‬ . ‫و‬ ‫ﻗ‬ ‫ﻊ‬ ‫ﻣ‬ ‫ﻊ‬ ‫ﻓ‬ ‫ﻮ‬ ‫ﻟ‬ ‫ﻔ‬ ‫ﺴ‬ ‫ﺒ‬ ‫ﻮ‬ ‫ر‬ ‫ج‬ ‫ﻣ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺑ‬ ‫ﻞ‬ 18 ‫ﻣ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﻮ‬ ‫ن‬ ‫ﺟ‬ ‫ﻨ‬ ‫ﯿ‬ ‫ﮫ‬ ‫ا‬ ‫ﺳ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻟ‬ ‫ﯿ‬ ‫ﻨ‬ ‫ﻲ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﻋ‬ ‫ﺎ‬ ‫م‬ 2014 ، ‫و‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﻋ‬ ‫ﺎ‬ ‫م‬ 2015 ‫ﺣ‬ ‫ﺼ‬ ‫ﻞ‬ ‫ﻋ‬ ‫ﻠ‬ ‫ﻰ‬ ‫ﻟ‬ ‫ﻘ‬ ‫ﺐ‬ ‫أ‬ ‫ﻓ‬ ‫ﻀ‬ ‫ﻞ‬ ‫ﻻ‬ ‫ﻋ‬ ‫ﺐ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ا‬ ‫ﻟ‬ ‫ﺴ‬ ‫ﻨ‬ ‫ﺔ‬ ‫ﻓ‬ ‫ﻲ‬ ‫أ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺎ‬ ‫ﻧ‬ ‫ﯿ‬ ‫ﺎ‬ . ] 11 [ ‫ﻓ‬ ‫ﻲ‬ ‫و‬ ‫ﻗ‬ ‫ﺖ‬ ‫ﻻ‬ ‫ﺣ‬ ‫ﻖ‬ ‫ﻣ‬ ‫ﻦ‬ ‫ذ‬ ‫ﻟ‬ ‫ﻚ‬ ‫ا‬ ‫ﻟ‬ ‫ﻌ‬ ‫ﺎ‬ ‫م‬ ، ‫ا‬ ‫ﻧ‬ ‫ﻀ‬ ‫ﻢ‬ ‫إ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﻣ‬ ‫ﺎ‬ ‫ﻧ‬ ‫ﺸ‬ ‫ﺴ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﺳ‬ ‫ﯿ‬ ‫ﺘ‬ ‫ﻲ‬ ‫ﻣ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺑ‬ ‫ﻞ‬ 54 ‫ﻣ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﻮ‬ ‫ن‬ ‫ﺟ‬ ‫ﻨ‬ ‫ﯿ‬ ‫ﮫ‬ ‫ا‬ ‫ﺳ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻟ‬ ‫ﯿ‬ ‫ﻨ‬ ‫ﻲ‬ . question: ‫ﻣ‬ ‫ﺘ‬ ‫ﻰ‬ ‫ا‬ ‫ﻧ‬ ‫ﻀ‬ ‫ﻢ‬ ‫ا‬ ‫ﻟ‬ ‫ﻼ‬ ‫ﻋ‬ ‫ﺐ‬ ‫ﻛ‬ ‫ﯿ‬ ‫ﻔ‬ ‫ﻦ‬ ‫د‬ ‫ي‬ ‫ﺑ‬ ‫ﺮ‬ ‫و‬ ‫ﯾ‬ ‫ﻦ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺎ‬ ‫ﻧ‬ ‫ﺸ‬ ‫ﺴ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﺳ‬ ‫ﯿ‬ ‫ﺘ‬ ‫ﻲ‬ ‫ا‬ ‫ﻻ‬ ‫ﻧ‬ ‫ﺠ‬ ‫ﻠ‬ ‫ﯿ‬ ‫ﺰ‬ ‫ي‬ ‫؟‬ answer: 2015. analyzes the arabic pre-trained lm fine-tuned on mrc task. figure 1 shows an example of a distractor sentence appended to the passage. we construct an unreadable distractor by concatenating a variation of keywords from the question, answer, and a random sentence from the passage, and shuffling the word order in the input. our experimental results show a significant performance drop, implying that arabert (version 0.1) relies on statistical cues and keyword matching and, moreover, can be distracted easily when irrelevant information is added to the passage. although the task of arabic mrc has gained popularity in the research community, the question of what information must be captured by a model to achieve high performance remains unclear. we extend our analysis on adversarial examples by examining the hidden state vectors between encoder layers. at this end, we visualize vector transformations in each layer of the transformer for correctly-predicted and falsely-predicted answers. this visualization exposes incorrect predictions in the earlier layers and reveals the part of the context that the model regards as supporting facts. we summarize the main contributions of this research as follows: the rest of the paper is organized as follows: section 2 reviews the related work. section 3 discusses the methodology to construct adversarial examples and visualizes the transformer's hidden state vectors. we conduct several experiments and discuss our findings in section 4. finally, we conclude the paper in section 5. ## related work the interpretability and probing of models for nlp tasks fall into three areas: probing tasks to understand linguistic properties captured by the model; adversarial examples to understand the flaws and weaknesses of the model; and assessing dataset quality by conducting partial training and constructing challenging sets @xcite . we discuss the first two directions as they are the focus of our work. probing tasks the most common approach used to understand the linguistic properties captured by a model is to investigate its hidden states by designing probing tasks. the authors in @xcite designed a novel edge probing task to measure the effectiveness of a pre-trained model in capturing linguistic information, which involved probing core nlp tasks encompassing diverse syntactic and semantic phenomena. in @xcite , the authors discovered that bert follows a classical nlp pipeline in an interpretable and localizable way. the authors in @xcite applied a set of general and qa-specific probing tasks to discover information in each representation layer. an attention-based probing classifier was proposed by @xcite to demonstrate syntactic information and coreference captured by bert's attention. through structural probes, (hewitt and manning, 2019) suggested that syntactic information can be recovered from bert token representations. as multiple probing studies report that bert possesses syntactic, semantic, and world knowledge information, @xcite emphasized that the absence of linguistic patterns in a probing classifier cannot lead one to conclude that the information is not there, nor can the presence of such information reveal how it is used. furthermore, it is worth questioning the extent to which a probing classifier can be complex such that the recovered information is extracted from the original model rather than the probing classifier @xcite . to rectify these issues, @xcite proposed amnesic probing, which involves removing certain information from the model (e.g., part-of-speech tags) and evaluating the performance change. adversarial examples adversarial examples have gained immense popularity in the nlp field, particularly as a way to understand model failures. most research studies have focused on constructing black-box attacks due to the nature of the text @xcite . first, the discrete nature of the input complicates the task of measuring the distance between the original and the adversarial example. second, it is difficult to formulate minimizing the distance as an optimization problem for a discrete input. in @xcite , it was shown that appending a distractor sentence to the passage in an mrc task lowers the performance for state-of-the-art models. the authors in @xcite proposed a black-box genetic algorithm to construct semantically and syntactically similar examples, the aim being to fool sentiment analysis and textual entailment models. the performance of bert dropped when evaluated on unreadable adversarial examples, which were generated by appending keywords to candidate answers in multiple-choice reading comprehension datasets @xcite . recently, model-in-the-loop adversarial annotation, which has been investigated by @xcite , @xcite , and @xcite , is a new direction that has been applied to annotate challenging datasets using an adversarial model to retrieve hard examples. the annotators are allowed to interact with the adversary during the annotation process, and they can use model feedback to inform the generation process. only one study, that of (alshemali and kalita, 2019), has investigated adversarial examples in the arabic language. specifically, noun-adjective agreement in arabic was violated in order to construct a perturbed input text. this attack fooled a word-level bidirectional long short-term memory (bilstm) model and a word-level convolutional neural network (cnn) model, and it successfully reduced model performance when evaluated on a sentiment analysis task. transformer models our analysis focuses on arabert (version 0.1) (antoun et al., 2020a), an arabic pre-trained language model based on the bert-base architecture @xcite . bert is a stack of transformer encoder layers with multiple self-attention heads @xcite . in (antoun et al., 2020a), the authors trained the bert-base architecture on manually-scraped arabic news websites, the public-words arabic corpus, and the open source international arabic news corpus (osian). since then, several arabic transformer models have been proposed. safaya et al. (2020) introduced an arabic bert, a pretrained bert-base on arabic wikipedia dump and the arabic version of oscar (ortiz suárez et al., 2020). in arbert and marbert, (abdul-mageed et al., 2020) pre-trained bert model on modern standard arabic (msa) and dialects. antoun et al. (2020b) generated aragpt2 trained from scratch on large arabic corpora following the architecture and training procedure of gpt2 @xcite . to the best of our knowledge, no research study has been undertaken to analyze recent arabic pretrained language models. arabic adversarial examples were constructed for sentiment analysis task @xcite , but no previous study has examined adversarial examples for arabic mrc. ## methodology we focus our analysis on arabert (version 0.1) fine-tuned on the task of arabic mrc using four different arabic mrc datasets. we propose two approaches to investigate the model's failures and to study the transformation of hidden vectors in each layer. first, we construct an unreadable perturbed sentence and append it to the context to generate adversarial examples. following this, we qualitatively analyze the vector transformations by examining their position in vector space. ## conclusion this paper analyzed hidden state transformations through layers and adversarial examples to explore what an arabic pre-trained lm learns from mrc datasets. we proposed a simple yet effective method to construct adversarial examples that fool arabert, which resulted in a substantial performance drop. our analysis suggests that pre-trained lms achieve competitive performance simply by relying on superficial cues such as lexical overlap or entity type matching. qualitative analysis of the hidden states of transformers indicated that uninterpretable information can be used to understand the reasons that underpin model failure and weaknesses. we demonstrated that locating the correct supporting fact for mrc earlier in the middle layers contributes to correct predictions. our finding suggests that different layers solve different problems. we suggest examining part of the network or connecting non-adjacent layers based on the downstream task at hand. it would be worthwhile for future work to analyze the degree to which different tokenization methods (input representations) assist in the learning of better morphology and the modeling of infrequent words in arabic pre-trained lms. furthermore, we are interested in extending our adversarial examples to cover other arabic pre-trained lms