# An Attention-Based Residual Neural Network for Efficient Noise Suppression in Signal Processing
## 2 Methods and Evaluation Index  
### ResATnet Denoising Principle  
Noisy data y is the sum of clean data z and noise v, and separating these two components is essential for accurate signal interpretation. Here is the calculation formula:  
\[\mathrm{y}=\mathrm{z}+\mathrm{v} \tag{1}\]  
The foremost objective of deep learning is to reduce the divergence between the unpolluted signal and the approximated signal by creating intricate mappings that facilitate the recovery of a pristine, untainted signal from seismic data plagued by noise. The mathematical expressions are:  
\[\mathrm{R}(\mathrm{y};\mathrm{P})=z_{\mathrm{R}} \tag{2}\]  
where R stands for mapping, \(\mathrm{P}=\{\mathrm{W},\mathrm{B}\}\) is the parameter of the neural network, and \(z_{\mathrm{R}}\) denotes the signal retrieved from the noise by the neural network, which constitutes an estimation of the pristine signal, z. Once the mapping is established, the neural network is trained to traverse the loss function in reverse, with the objective of extracting meaningful clean signal features from the noisy data, which can be expressed as:  
\[\mathrm{L}(\mathrm{P})=\sum_{k=1}^{Q}\Big{|}\left|\mathrm{R}(\mathrm{y}_{k}; \mathrm{P})-z_{\mathrm{R}_{k}}\right|\Big{|}_{F}^{2} \tag{3}\]  
Where \(\mathrm{L}(\mathrm{P})\) represents the loss function, and Q is the batch size. Figure 1 shows the network structure of ResATnet. The apparent resemblance of its structure to that of the Resnet18 network is noteworthy because the method proposed in this paper is an obvious modification on the residual block of Resnet18. The original Resnet18 network comprises residual blocks, consisting of a pair of weight layers with shortcut connections, which enables the network to circumvent the vanishing gradient problem and attain improved performance in training deep neural networks (Figure 2). Shortcut connections pass the feature map before the current residual block directly to the features extracted by the weight layer. This practice propagates noise throughout the network, consequently impeding the denoising efficacy of the residual neural network. Motivated by the paper [25] and its innovative attention mechanism, we introduce an attention mechanism into the residual block of Resnet18 to selectively suppress extraneous noise features and amplify informative signal features, thereby elevating the denoising performance of the network. Specifically, the signal features extracted from the weight layer are enhanced using CBAM after the second weight layer of the residual block, and the noise is filtered using the CA on the shortcut connections. After that, the filtered features are added with the features extracted from the weight layer and finally processed by activation function ReLU to obtain the feature output (Figure 3). Figure 1: The network structure of ResATnet. Figure 2: Resnet’s residual block. The Convolutional Block Attention Module, a soft attention module proposed by [PERSON] [26], represents a straightforward, yet powerful, mechanism for selectively enhancing salient features within a neural network. It is worth noting that other soft attention modules, such as the spatial attention module and channel attention module [27; 28], also exist, with CBAM being a hybrid module encompassing the aspects of both. The CBAM is rooted in the notion that the individual channels of a given feature map undergo a preliminary process of weighting, which is subsequently followed by a further spatial weighting operation in order to obtain the ultimate weighted feature map. Specifically:  
The channel attention module (CAM) operates by extracting channel-weighted values via a pooling mechanism that compresses spatial dimensions while preserving the channel dimensions. In the CAM architecture, the feature map is first subjected to two parallel pooling layers, MaxPool and AvgPool, followed by compression and recovery operations via a multilayer perceptron (MLP) module.