# Research on Fault Diagnosis of Marine Diesel Engines Based on CNN-TCN-ATTENTION
## 2 Deep Learning Network Models
### Temporal Convolutional Network (TCN)  
The temporal convolutional network (TCN) is specifically designed for time-series data, combining various convolutional methods. It primarily employs temporal 1D convolutions and uses residual connections to mitigate issues such as vanishing and exploding gradients [35]. In terms of structure, the TCN differs from the CNN primarily in that it consists of causal convolutions, dilated convolutions, and residual connection modules. The architecture of the TCN is illustrated in Figure 4.  
The input feature vector first undergoes dilated causal convolutions, followed by normalization using WeightNorm to accelerate model training. The ReLU activation function is then applied to introduce nonlinearity into the feature vector. Dropout is used to discard unnecessary neurons and prevent overfitting. A 1D fully convolutional layer is employed to ensure that the input sequence and output sequence of the TCN have the same length. The result is then added to the input vector to produce the final output. Causal convolution: Causal convolution is a fundamental component of the TCN, ensuring that the network maintains causality when processing time-series data. This means that the output at any given time step depends only on the current time step and the preceding elements in the input sequence [36]. Causal convolution is achieved by adding the appropriate amount of zero padding to the left side of the input sequence, ensuring that the convolution operation does not \"see\" future data. Causal convolution is a key requirement for handling time-series data, as it preserves the natural temporal order. The structure of causal convolution is illustrated in Figure 5, and its formula is illustrated in Equation (4). \[y_{t}=\sum_{k=0}^{k-1}\omega_{k}\cdot x_{t-k}, \tag{4}\]  
where \(y_{t}\) denotes the output of the convolution operation, \(\omega_{k}\) denotes the weights of the convolutional kernel, and \(x_{t-k}\) denotes the elements of the input sequence. Figure 4: Architectural principle of TCN. Dilated convolution: Unlike traditional convolution, dilated convolution allows for the spaced sampling of the input during the convolution process. The sampling rate is controlled by the dilation coefficient \(\mathrm{d}\), as shown in the figure. For the lower layers, where \(\mathrm{d}=1\), each point in the input is sampled. In the intermediate layers, where \(\mathrm{d}=2\), every second point is sampled as input. Generally speaking, the value of \(\mathrm{d}\) increases with the layer height. This indicates that as the number of layers increases, the effective receptive field's size increases exponentially. Thus, dilated convolution enables convolutional networks to achieve large receptive fields with fewer layers [37]. The model presented in this paper uses two layers of dilated convolution. The structure of dilated convolution is depicted in Figure 6. The formula is given in Equation (5). \[\mathrm{y_{t}}=\sum_{k=0}^{k-1}\omega_{k}\cdot\mathrm{x_{t-d\cdot k}}, \tag{5}\]  
where \(\mathrm{y_{t}}\) denotes the output of the convolution operation, \(\omega_{k}\) denotes the weights of the convolutional kernel, \(\mathrm{x_{t-d\cdot k}}\) denotes the elements of the input sequence, and \(\mathrm{d}\) denotes the dilation rate. Residual connection: In the TCN, although the introduction of causal and dilated convolutions effectively expands the receptive field, it inevitably increases the depth of the network. This necessitates the use of residual connections to address potential issues such as gradient vanishing or gradient decay [38]. This paper takes a residual module with  
Figure 5: Principle of causal convolution. Figure 6: Principle of dilated convolution. a convolutional kernel size of 3 and a dilation rate of 1 as an example. The structure is illustrated in Figure 7, and the formula is provided in Equation (6).