# Research Method for Ship Engine Fault Diagnosis Based on Multi-Head Graph Attention Feature Fusion
## 2 Related Work and Theory
### Graphical Attention Neural Network  
Both GANN and GCN studied in this paper are variants of GNN. Different from the Laplacian smoothing of all directly adjacent nodes at each node of GCN [33], GANN uses the attention mechanism to calculate the edge weight coefficient. The attention mechanism should be to the operation of aggregating neighbors of GCN. According to the three elements of the attention mechanism: query, source, and attention value [34], we will determine to set query, which is the feature vector of the current central node, source to the feature vector of all neighbors, and attention value to the new feature vector of the center node after the aggregation operation. Consider the feature vector corresponding to node \(x_{i}\) in layer 1 of the graph as \(h_{i}\Big{(}h_{i}\in R^{d^{(l)}}\Big{)}\), where \(d^{(l)}\) denotes the feature length of the node. After applying an aggregation operation with an added attention mechanism, the output is a new feature vector \(h_{i}^{{}^{\prime}}\Big{(}h_{i}^{{}^{\prime}}\in R^{d^{(l+1)}}\Big{)}\) for each node, with \(d^{(l+1)}\) denoting the length of the output feature vector. We refer to this aggregation operation as the Graph Attention Layer (GAL). Define the central node as \(x_{i}\), then the correlation from neighboring nodes \(x_{j}\) to \(x_{i}\) is as follows:  
\[e_{ij}=a\big{(}Wh_{i},Wh_{j}\big{)}, \tag{14}\]  
The weight parameter of the feature transformation of the nodes in the layer is denoted as \(W\in R^{d^{(l+1)}\times d^{(l)}}\). The function \(a(\cdot)\) calculates the correlation between two nodes, and in this paper, a single fully connected layer is chosen:  
\[e_{ij}=LeakyReLU\bigg{(}\overset{\rightarrow}{a}^{T}\bigg{[}W\overset{ \rightarrow}{h_{i}}\parallel W\overset{\rightarrow}{h_{j}}\bigg{]}\bigg{)}, \tag{15}\]  
The weight vector \(a\in R^{2d^{(l+1)}}\) is used along with the activation function LeakyReLU with a negative slope \(\alpha=0.2\). Here, \"\(\parallel\)\" denotes a vector splicing operation. To better assign the weights, the computed correlation is normalized using Softmax:  
\[\alpha_{ij}=\frac{\exp\big{(}e_{ij}\big{)}}{\sum\limits_{k\in N_{i}}\exp \big{(}e_{ij}\big{)}}, \tag{16}\]  
\(\alpha\) is the weight coefficient, which is calculated in the equation above to ensure that the aggregated weight coefficients of all neighboring nodes of the central node sum to one. Figure 1 illustrates the computation process. \(Wh_{i}\) and \(Wh_{j}\) are first parameterized by \(a\), then activated by LeakyReLU, and finally normalized by Softmax to obtain the weight parameters. Here's the complete formula for calculating the weight coefficients:  
\[\alpha_{ij}=\frac{\exp\bigg{(}LeakyReLU\bigg{(}\overset{\rightarrow}{a}^{T} \bigg{[}W\overset{\rightarrow}{h_{i}}\parallel W\overset{\rightarrow}{h_{j}} \bigg{]}\bigg{)}\bigg{)}}{\sum\limits_{k\in N_{i}}\exp\bigg{(}LeakyReLU\bigg{(} \overset{\rightarrow}{a}^{T}\bigg{[}W\overset{\rightarrow}{h_{i}}\parallel W \overset{\rightarrow}{h_{k}}\bigg{]}\bigg{)}\bigg{)}}, \tag{17}\]The update formula for the feature vector of the node \(x_{i}\) is calculated as follows based on the weight coefficients obtained from the above formula:  
\[h^{(l+1)}{}_{i}=\sigma\left(\sum_{k\in N_{i}}\alpha_{ij}Wh^{(l)}{}_{j}\right), \tag{18}\]  
where \(h^{(l+1)}_{i}\) is the updated feature vector of node \(x_{i}\) in layer \(l+1\); \(\alpha\) represents the activation function (commonly used is LeakyReLU); \(N_{i}\) denotes the neighbors of node \(x_{i}\); \(\alpha_{ij}\) is the weight coefficient calculated using the previously mentioned weight coefficient formula; and \(W\) is the weight matrix parameter. This formula calculates the updated feature vector for each node by aggregating information from its neighbors, with the aggregation weights determined by the attention mechanism. To enhance the expressive capacity of the attention layer, this paper employs the multi-head attention mechanism, similar to the approach utilized by [PERSON] et al. [35], to enhance the model's expressive ability.