Evaluating Explanation Methods for Neural Machine Translation (2020)

Abstract: Recently many efforts have been devoted to interpreting the black-box NMT models, but little progress has been made on metrics to evaluate explanation methods. Word Alignment Error Rate can be used as such a metric that matches human understanding, however, it can not measure explanation methods on those target words that are not aligned to any source word. This paper thereby makes an initial attempt to evaluate explanation methods from an alternative viewpoint. To this end, it proposes a principled metric based on fidelity in regard to the predictive behavior of the NMT model. As the exact computation for this metric is intractable, we employ an efficient approach as its approximation. On six standard translation tasks, we quantitatively evaluate several explanation methods in terms of the proposed metric and we reveal some valuable findings for these explanation methods in our experiments.

## introduction neural machine translation (nmt) has witnessed great success during recent years @xcite @xcite . one of the main reasons is that neural networks possess the powerful ability to model sufficient context by entangling all source words and target words from translation history. the downside yet is its poor interpretability: it is unclear which specific words from the entangled context are crucial for nmt to make a translation decision. as interpretability is important for understanding and debugging the translation process and particularly to further improve nmt models, many efforts have been devoted to explanation methods for nmt @xcite @xcite @xcite . however, little progress has been made on evaluation metric to study how good these explanation methods are and which method is better than others for nmt. generally speaking, we recognize two orthogonal dimensions for evaluating the explanation methods: i) how much the pattern (such as source words) extracted by an explanation method matches human understanding on predicting a target word; or ii) how the pattern matches predictive behavior of the nmt model on predicting a target word. in terms of i), word alignment error rate (aer) can be used as a metric to evaluate an explanation method by measuring agreement between human-annotated word alignment and that derived from the explanation method. however, aer can not measure explanation methods on those target words that are not aligned to any source words according to human annotation. in this paper, we thereby make an initial attempt to measure explanation methods for nmt according to the second dimension of interpretability, which covers all target words. the key to our approach can be highlighted as fidelity: when extracting the most relevant words with an explanation method, if those relevant words have the potential to construct an optimal proxy model that agrees well with the nmt model on making a translation decision, then this explanation method is good ( §3). to this end, we formalize a principled evaluation metric as an optimization problem over the expected disagreement between the optimal proxy model and the nmt model( §3.1). since it is intractable to exactly calculate the principled metric for a given explanation method, we propose an approximate metric to address the optimization problem. specifically, inspired by statistical learning theory @xcite , we cast the optimization problem into a standard machine learning problem which is addressed in a two-step strat-egy: firstly we follow empirical risk minimization to optimize the empirical risk; then we validate the optimized parameters on a held-out test dataset. moreover, we construct different proxy model architectures by utilizing the most relevant words to make a translation decision, leading to variant approximate metric in implementation ( §3.2). we apply the approximate metric to evaluate four explanation methods including attention @xcite , gradient norm @xcite , weighted gradient @xcite and prediction difference @xcite . we conduct extensive experiments on three standard translation tasks for two popular translation models in terms of the proposed evaluation metric. our experiments reveal valuable findings for these explanation methods: this paper makes the following contributions: ## nmt and explanation methods different from rnn-search, which relies on rnn, transformer employs an attention network to define h, and two additional attention networks to define s t as follows: foot_0 ## experiments in this section, we conduct experiments to prove the effectiveness of our metric from two viewpoints: how good an explanation method is and which explanation method is better than others. ## related work in recent years, explaining deep neural models has been a growing interest in the deep learning community, aiming at more comprehensible and trustworthy neural models. in this section, we mainly discuss two dominating ways towards it. one way is to develop explanation methods to interpret a target black-box neural network @xcite . for example, on classification tasks, @xcite propose layer-wise relevance propagation to visualize the relationship between a pair of neurons within networks, and @xcite introduce a gradientbased approach to understanding the compositionality in neural networks for nlp. in particular, on structured prediction tasks, many research works design similar methods to understand nmt models @xcite @xcite . the other way is to construct an interpretable model for the target network and then indirectly interpret its behavior to understand the target network on classification tasks @xcite @xcite . the interpretable model is defined on top of extracted rational evidence and learned by model distillation from the target network. to extract rational evidence from the entire inputs, one either leverages a particular explanation method @xcite or an auxiliary evidence extraction model @xcite . although our work focuses on evaluating explanation methods and does not aim to construct an interpretable model, we draw inspiration from their ideas to design @xmath0. ( 6 ) for our evaluation metric. with the increasing efforts on designing new explanation methods, yet there are only a few works proposed to evaluate them. @xcite propose a paradigm to evaluate explanation methods for document classification that involves human judgment for evaluation. @xcite conduct the first human-independent comprehensive evaluation of explanation meth-ods for nlp tasks. however, their metrics are task-specific because they make some assumptions for a specific task. our work proposes a principled metric to evaluate explanation methods for nmt and our evaluation paradigm is independent of any assumptions as well as humans. it is worth noting that @xcite directly measure the performance of the target model p on the extracted words without constructing q to evaluate explanation methods for classification tasks. however, since translation is more complex than classification tasks, p trained on the entire context c t typically makes a terrible prediction when testing on the compressed context w k φ (c t ). as a result, the poor prediction performance makes it difficult to discriminate one explanation method from others, as observed in our internal experiments. concurrently, jacovi and goldberg (2020) make a proposition to evaluate faithfulness of an explanation method separately from readability and plausibility (i.e., human-interpretability), which is similar to our definition of fidelity, but they do not formalize a metric or propose algorithms to measure it. ## conclusions this paper has made an initial attempt to evaluate explanation methods from a new viewpoint. it has presented a principled metric based on fidelity in regard to the predictive behavior of the nmt model. since it is intractable to exactly calculate the principled metric for a given explanation method, it thereby proposes an approximate approach to address the minimization problem. the proposed approach does not rely on human annotation and can be used to evaluate explanation methods on all target words. on six standard translation tasks, the metric quantitatively evaluates and compares four different explanation methods for two popular translation models. experiments reveal that pd, ngrad, and attn are all good explanation methods that are able to construct the nmt model's predictions with relatively low perplexity and pd shows the best fidelity among them.