# Towards Understanding Neural Machine Translation with Attention Heads' Importance
## 2 Background
### Linguistic Knowledge  
Linguistics delves into the structure, function, and essence of human language, encompassing areas such as language structure, phonology, word formation, syntax, and semantics.  
In the realm of interpretability, linguistic knowledge is particularly valued for its ability to render model decisions understandable to humans. Many researchers advocate for using linguistic knowledge as a basis for elucidating models [23]. Recent studies have increasingly interpreted NMT models using linguistic insights, including aspects like morphology [6] and POS [24]. However, these studies primarily focus on the impact of individual word information in NMT. Consequently, we have explored the utilization of inter-word interactions and multi-word information to shed light on model decisions [25]. Understanding the interplay between words and leveraging multiple-word information can make translations smoother, achieving the \"faithfulness, expressiveness, and elegance\" standard of translation. Additionally, this approach can improve the accuracy and fluency of translations involving complex sentence structures, such as idioms and colloquialisms.  
This paper investigates the decisions made by the model's important heads at three different granularities of linguistic knowledge: POS, dependency relations, and syntactic trees. POS reflects the role of individual word information in a sentence in NMT and helps us understand how the model handles individual words. Dependencies reflect the dominant relationship between two words in a sentence and help us understand how the model handles inter-word relationships. The syntax tree reflects the relationships between words in a sentence and helps us understand how the model handles multiple words as well as syntactic structure.  
Finally, in Table 1, we compared two categories of interpretability: visualization and linguistic interpretation. We explained their strengths and limitations and provided explanations for some references and their main contributions in order to better understand the main contributions of our study.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Strengths** & **Limitations** & **References** & **Contributions** \\ \hline \hline \multirow{8}{*}{Visualization} & \multirow{8}{*}{1. Intuitive representation of textual data and processes; 2. Provides visually intuitive charts; 3. Accessible to non-experts.} & \multirow{8}{*}{1. Difficulty in explaining complex grammatical structures; 2. May overlook subtle nuances in text; 3. Limited in-depth linguistic analysis.} & \multirow{8}{*}{
\begin{tabular}{c} Understanding the internal workings of NMT models through attention-based visualization techniques. \\ \end{tabular} } \\  & & & \\ \cline{1-  
Numerous studies have explored the interpretability of NMT. In this paper, we link the importance of attention heads to linguistic knowledge, using visualization to understand the importance of attention heads in the model, and employing linguistic knowledge to understand the content of the model's decision making.