# Towards Understanding Neural Machine Translation with Attention Heads' Importance
## 2 Background  
### Interpreting Attention  
Interpretability lacks a unified definition. Some researchers perceive it as the extent to which people can understand the reasons behind decisions, while others believe that interpretability refers to an individual's ability to consistently and accurately predict the outcomes of model predictions [10]. Due to the complexity of widely used neural network models, their internal logic and working principles are difficult for users to understand. Users are unclear about what comprehensible linguistic knowledge the model has acquired through training and how the model processes language information. Consequently, we propose that interpretability involves bridging the gap between models that are unintligible to humans and the integration of human-comprehensible linguistic knowledge to elucidate model behavior.  
In the realm of NMT interpretability, numerous studies have concentrated on explaining the importance of various model components. These studies enable an intuitive observation of the elements crucial to model predictions, thereby enhancing the interpretability of models. Recent inquiries into attention interpretability have honed in on the functionalities of attentional mechanisms and the implications of attention weights, with research on attention heads gaining increasing traction [11]. There has been a significant emphasis on dissecting the influence of diverse components within models for enhancing interpretability [12]. Attention as the driving force behind the Transformer, the state-of-the-art NMT model, is garnering escalating scholarly attention. Previous investigations underscored the intrinsic value of attention. For example, [PERSON] [13] employed intermediate representation erasure to ascertain the impact of attentional mechanisms, while [PERSON] [14] explored the alignment effect of attention weights. However, as recent studies indicate, relying solely on attention weights to interpret attentional processes is inadequate [15]. The debate over the explicability of attention also constitutes a major discourse [16; 17]. Therefore, in our approach to interpreting attention, we pivot towards leveraging linguistic knowledge that is understandable to humans.  
The core of the attention layer is the multi-head attention, formed by concatenating the outputs of multiple attention heads. This architecture allows for the learning of diverse feature aspects. Recent studies have scrutinized the redundancy within multi-head attention mechanisms, revealing that attention heads vary in importance [8]. [PERSON] [18] utilized Layer-wise Relevance Propagation (LRP) [19] to ascertain the contribution of individual attention heads in each layer towards model predictions, further investigating the functions of critical attention heads. The importance of attention heads has also been explored in multilingual and cross-linguistic contexts [20]. Motivated by research into the information processed by attention mechanisms [21], we aim to analyze the multi-grain linguistic knowledge acquired by important attention heads, thereby elucidating the factors influencing model decisions [22].  
In our experiments, we analyze the relationship between attention heads and multi-granularity linguistic knowledge by masking each attention head in every layer of three different types of attention mechanisms: encoder self-attention, decoder self-attention, and encoder-decoder attention. We aim to explain the reasons and content behind the model's decisions using linguistic knowledge that humans can understand.