# Research on Fault Diagnosis of Marine Diesel Engines Based on CNN-TCN-ATTENTION
## 2 Deep Learning Network Models  
### Convolutional Neural Networks (CNNs)  
Convolutional neural networks (CNNs) are among the most crucial architectures in deep learning, known for their powerful feature extraction capabilities [29]. CNNs can extract both local and hierarchical features from input data, having fewer connections and parameters than traditional multi-layer perceptrons due to local connections, weight sharing, and pooling operations. This reduces the model's complexity and enhances training efficiency. The structure of the CNN is illustrated in Figure 1.  
The primary structure of a CNN includes the input layer, convolutional layer, pooling layer, fully connected layer, and Softmax layer. The convolutional and pooling layers serve as the core components of the CNN, usually appearing in pairs. Users can freely choose the number of convolution-pooling pairs based on the complexity of the task. This configuration can optimize the overall network structure and enhance the model's performance [30]. Input layer: the input layer of a CNN is responsible for converting the input image or data into a 2D matrix and storing this matrix for subsequent operations [31]. Convolutional layer: The convolutional layer contains multiple convolutional kernels, which perform convolution operations on the input data to extract features. This operation calculates the convolutional kernel and the dot product of the data, assigning higher values to areas in the data with features and lower values to regions without features. The sum of these dot products is assigned to the corresponding data value in the output matrix at the current position of the convolutional kernel. By moving and calculating with the convolutional kernel repeatedly, a new matrix is obtained [32]. This paper's suggested approach makes use of a 3 \(\times\) 3 convolutional kernel. Assuming the input data are 5 \(\times\) 5 and the sliding step is 1, a 3 \(\times\) 3 feature map can be obtained. The convolution process is illustrated in Figure 2. The formula is listed in Equation (1). \[\text{M}_{\text{t}}=(\text{m}+2\text{p}-\text{k})/\text{s}+1, \tag{1}\]  
Figure 1: Structure of CNN. where \(M_{t}\) denotes the feature map size, \(m\) denotes the input matrix size, \(k\) denotes the dimension of the convolutional kernel, \(s\) denotes the stride, and \(p\) denotes the number of padding layers. Activation layer: The activation layer commonly consists of an activation function that introduces a nonlinear transformation to the output of the convolutional layer, giving the feature map nonlinear characteristics. Common activation functions include ReLU, tanh, and sigmoid. The model presented in this paper employs the ReLU activation function, as defined in Equation (2). ReLU offers benefits such as rapid convergence and straightforward gradient calculation [33]. \[ReLU(x)=max(x,0), \tag{2}\]  
Pooling layer: This is commonly positioned following the convolutional layer and serves to down-sample the extracted features, reducing the spatial dimensions and computational complexity of the data. Pooling operations commonly involve max pooling and average pooling [34]. The model proposed in this paper uses max pooling. Pooling also involves a sliding kernel, which can be referred to as a sliding window. Assume that the sliding window size is \(2\times 2\) and the stride is \(2\). At each position of the window, the maximum value within the region is chosen as the output. The pooling process is illustrated in Figure 3, and the formula is displayed in Equation (3). \[X=f[aS(x)+b], \tag{3}\]  
where \(X\) represents the output of the pooling layer, \(f()\) represents the activation function, \(\alpha\) represents the sampling coefficient, and \(b\) is the bias term. Fully connected layer: Following multiple convolution and pooling layers, the resulting feature maps are flattened row by row and then concatenated to form a vector,  
Figure 3: Principle of the pooling process, with an outline highlighting the sliding region.