# Research on Fault Diagnosis of Marine Diesel Engines Based on CNN-TCN-ATTENTION
## 3 Experiment
### Ablation Experiment
#### 3.3.1 Ablation Experiment of Convolution Layer  
Ablation experiments are conducted to select the appropriate network structure by evaluating the number of convolution-pooling pairs, thereby avoiding issues such as overfitting due to excessive layers, reduced efficiency, and higher costs associated with a more complex structure. Multiple experiments are performed on the number of layers in the CNN and TCN, and the average results are taken to calculate the performance of the network models with different layer counts. The results are displayed in Table 7, and the accuracy and loss curves for the training and validation sets are depicted in Figure 13.  
From Table 7 and Figure 13, it can be observed that Model A has a lower accuracy in the test set, and its loss curve converges poorly. Even after the accuracy starts to converge, the loss curve continues to decrease. This is due to the relatively small number of convolutional layers in the CNN and TCN, which causes fluctuations in the network's stability and insufficient feature extraction capabilities of local structures. Moreover, Model A has 3100 trainable parameters and a training time of 23 s, which is advantageous compared to the other three models. However, due to its simpler network structure, its performance in terms of accuracy and convergence is suboptimal, leading to unsatisfactory diagnostic results. In contrast, Model C and Model D show higher accuracies in the test set, 99.43% and 99.72%, respectively. However, when compared to Model B, it is evident that as the number of convolution layers increases, the accuracy begins to decline. This is because an excessive number of layers leads to model overfitting due to the complexity of the network structure. Additionally, the training time increases to 30s and 33s, and the convergence rate is slower than that of Model B. Occasional small gradient explosion phenomena also occur during training. Gradient explosion is a common issue in deep learning, which can destabilize the network. Model C has 4574 trainable parameters, and Model D has 5374, which makes them more complex than the other models. Thus, both Model C and Model D are not ideal choices overall. When selecting Model B, it achieves the highest accuracy in the test set, reaching 100%. The number of trainable parameters in Model B is 3900, which is slightly more than Model A but more balanced compared to Models C and D. Moreover,  
\begin{table}
\begin{tabular}{l l l} \hline
**Layers** & **Description** & **Details** \\ \hline
1 & Input & input\_shape = (32,1,22) \\  & & Filters = 32, kernel\_size = 3, strides = 1, padding = 1, \\
2 & Conv & activation = ReLu \\
3 & Max pooling & Kernel\_size = 2, strides = 2, padding = 0 \\
4 & Conv1 & Filters = 16, kernel\_size = 3, strides = 1, padding = 1, \\  & & dropout = 0.5, activation = ReLu \\
5 & Conv2 & Filters = 16, kernel\_size = 3, strides = 1, padding = 1, \\  & & dropout = 0.5, activation = ReLu \\
6 & Conv1d & Filters = 16, kernel\_size = 3, strides = 1, padding = 1, \\  & & activation = ReLu \\
7 & Attention & (Query, Key, Value), filters = 16, activation = linear \\
8 & FC & layer\_size = 160, activation = linear \\
9 & FC & layer\_size = 5, activation = linear \\ \hline \end{tabular}
\end{table}
Table 6: Details of CNN-TCN–ATTENTION.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **CNN Conv** & **TCN Conv** & \begin{tabular}{c} **Test** \\ **Accuracy/\%** \\ \end{tabular} & \
\begin{tabular}{c} **Running** \\ **Time/s** \\ \end{tabular} &
\begin{tabular}{c} **Trainable** \\ **Parameters** \\ \end{tabular} \\ \hline A & 1 & 1 & 95.63 & 23 & 3100 \\ B & 1 & 2 & 100 & 29 & 3900 \\ C & 2 & 1 & 99.43 & 30 & 4574 \\ D & 2 & 2 & 99.72 & 33 & 5374 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Experimental results with different numbers of convolution layers.  
its training time is shorter, making it the most efficient model overall. Therefore, Model B's architecture is chosen as the foundation for the combined model, and the subsequent research is based on this model.