# A Hybrid Model of Conformer and LSTM for Ocean Wave Height Prediction
## 2 Materials and Methods
### Algorithm Process  
The flowchart of our algorithm shows in the Figure 3.  
Figure 3: Algorithm flowchart. We used Spearman correlation analysis on buoy data to select feature variables with high correlation to wave height. The results are given in Tables 1 and 2. The correlation coefficient ranges from \([-1,1]\). The closer the absolute value of the coefficient is to 1, the higher the correlation; the closer to 0, the lower the correlation. A negative value indicates a negative correlation with the target feature. Experiments show that, in terms of wind speed, although GST has a higher correlation than WSPD, GST is an instantaneous value, while the target feature of wave height to be predicted is an average value. Therefore, WSPD is selected as the feature variable for wind speed; in terms of temperature, ATMP has a higher correlation than WTMP and DEWP, so ATMP is selected as the temperature feature variable; in terms of wave period, APD has a significantly higher correlation than DPD; finally, the atmospheric pressure value at sea level is selected as the pressure feature variable. Enter the Conformer, using multi-head self-attention mechanisms to transform the input X, generating different query (Q), key (K), and value (V) vectors for each head:  
\[\mathbf{Q}_{i} =\mathbf{X}\mathbf{W}_{i}^{Q} \tag{2}\] \[\mathbf{K}_{i} =\mathbf{X}\mathbf{W}_{i}^{K}\] (3) \[\mathbf{V}_{i} =\mathbf{X}\mathbf{W}_{i}^{V} \tag{4}\]  
where \(\mathbf{W}_{i}^{Q}\), \(\mathbf{W}_{i}^{K}\), and \(\mathbf{W}_{i}^{V}\) are learnable weight matrices, corresponding to the query, key, and value of the i-th head, respectively. Calculate the scaled dot-product attention for each head [27]:  
\[\mathbf{Attention}(\mathbf{Q}_{i^{\prime}}\mathbf{K}_{i^{\prime}}\mathbf{V}_{i})=\mathbf{softmax}( \frac{\mathbf{Q}_{i}\mathbf{K}_{i}^{T}}{\sqrt{\mathbf{d}_{k}}})\mathbf{V}_{i} \tag{5}\]  
Enter the one-dimensional convolutional layer, using a sliding window to extract the feature relationships between samples, and use a one-dimensional convolutional kernel with a length of 5 (the same as the number of feature values) to extract features. This method can focus on potential trends in the data within a short period. During the training of deep neural networks, the input distribution of each layer changes due to parameter updates in the previous layer. This phenomenon is called \begin{table}
\begin{tabular}{l l} \hline \hline
**Features** & **Explanation** \\ \hline WVHT & Significant wave height (meters) is calculated as the average of the highest one-third \\  & of all of the wave heights during the 20 min sampling period. \\ WSPD & Wind speed (m/s) averaged over an eight-minute period for buoys. \\ GST & Peak 5 or 8 s guest speed (m/s) measured during the eight-minute or \\  & two-minute period. \\ ATMP & Air temperature (Celsius) \\ WTMP & Sea surface temperature (Celsius). \\ DEWP & Dewpoint temperature taken at the same height as the air \\  & temperature measurement. \\ APD & Average wave period (seconds) of all waves during the 20 min period. \\ DPD & Dominant wave period (seconds) is the period with the maximum wave energy. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Explanation of buoy data. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Feature Name** & **WSPD** & **GST** & **ATMP** & **WTMP** & **DEWP** & **APD** & **DPD** & **PRES** \\ \hline Correlation Coefficient & 0.662 & 0.689 & \(-\)0.330 & 0.112 & \(-\)0.310 & 0.595 & 0.294 & 0.104 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Correlation coefficients based on multi-variable features at station 41040 in 2021, with the target variable being wave height. \"internal covariate shift.\" To reduce this shift, batch normalization normalizes the input of each layer, ensuring that their mean is 0 and variance is 1. This normalization helps the model converge faster and improves its generalization ability.