# A ResNet1D-AttLSTM-Based Approach for Ship Trajectory Classification
## 2 Methods
### Construction of the ResNet1D-AttLSTM Model
#### 2.2.2 AttLSTM Module  
LSTM networks alleviate the vanishing gradient problem in traditional RNNs to a certain extent by introducing gating mechanisms, thereby improving the ability to model long-term dependencies. However, when processing sequential features, LSTMs adopt a sequential approach, capturing information only from the beginning of the sequence to the current time step, while underutilizing information beyond the current position. Moreover, LSTMs have limitations in extracting feature patterns jointly expressed by multiple non-adjacent regions. Specifically, LSTMs struggle to effectively capture complex patterns that are discontinuously distributed in the sequence. Although theoretically capable of capturing such patterns at the final step of processing, their modeling capability may degrade due to the loss of early information. To address these issues, this study constructs an AttLSTM module by integrating a multi-head self-attention mechanism into the final layer of the LSTM. The self-attention mechanism directly calculates the correlations between each position in the trajectory sequence and all other positions, enabling the effective capture of global feature patterns that are discontinuously distributed while simultaneously focusing on the joint behavior of multiple regions. The application of the multi-head mechanism further enhances the model's understanding of the entire sequence, thereby improving its ability to model complex patterns. As shown in Figure 4, the AttLSTM module consists of an LSTM layer and an attention layer. The input to this module is the preprocessed ship trajectory sequences. Due to the variable lengths of these input sequences, we statistically determine the maximum length \(L_{max}\) from all trajectory sequences in the dataset to standardize the input length for the LSTM layer. For trajectory sequences shorter than \(L_{max}\), zero-padding is applied to extend them to \(L_{max}\). First, the trajectory points in the sequence are fed into the LSTM layer in chronological order to initially capture the temporal variation characteristics of the trajectory sequences. Then, a multi-head self-attention mechanism is applied to further extract and emphasize information from key time points within the sequence. This enhances the model's focus on important temporal features. In the LSTM layer, the number of network layers is set to \(N\). Each LSTM unit within a layer contains three gates (input gate \(i_{t}^{(l)}\), forget gate \(f_{t}^{(l)}\), and output gate \(o_{t}^{(l)}\)) where \(l\) denotes the current layer index. Each LSTM unit performs the following process for time step \(t\): given the input \(x_{t}^{(l)}\), the previous hidden state \(h_{t-1}^{(l)}\), and the previous cell state \(c_{t-1}^{(l)}\), it calculates the current hidden state \(h_{t}^{(l)}\) and the current cell state \(c_{t}^{(l)}\). The update process is described by Equations (2)-(8), as follows:  
\[f_{t}^{(l)}=\sigma\Big{(}\textbf{W}_{f}^{(l)}\Big{[}h_{t-1}^{(l)},x_{t}^{(l)} \Big{]}+b_{f}^{(l)}\Big{)} \tag{2}\]  
\[i_{t}^{(l)}=\sigma\Big{(}\textbf{W}_{t}^{(l)}\Big{[}h_{t-1}^{(l)},x_{t}^{(l)} \Big{]}+b_{i}^{(l)}\Big{)} \tag{3}\]  
\[j_{t}^{(l)}=tan\ h\Big{(}\textbf{W}_{f}^{(l)}\Big{[}h_{t-1}^{(l)},x_{t}^{(l)} \Big{]}+b_{j}^{(l)}\Big{)} \tag{4}\]  
\[c_{t}^{(l)}=f_{t}^{(l)}*c_{t-1}^{(l)}+i_{t}^{(l)}*j_{t}^{(l)} \tag{5}\]  
\[o_{t}^{(l)}=\sigma\Big{(}\textbf{W}_{\sigma}^{(l)}\Big{[}h_{t-1}^{(l)},x_{t}^{ (l)}\Big{]}+b_{o}^{(l)}\Big{)} \tag{6}\]  
\[h_{t}^{(l)}=o_{t}^{(l)}*tan\ h\Big{(}c_{t}^{(l)}\Big{)} \tag{7}\]  
\[x_{t}^{(l+1)}\ :=h_{t}^{(l)} \tag{8}\]  
In the equations:  
* \(*\) denotes element-wise multiplication (Hadamard product). * \(:=\) denotes assignment. * \(\textbf{W}\) and \(b\) represent the weight matrices and bias parameters of the gate controllers, respectively. Figure 4: Architecture of the AttLSTM module. \(\sigma\) and \(\tan h\) denote the Sigmoid activation function and the hyperbolic tangent activation function, respectively.