# DIR-Net: Deep Residual Polar Decoding Network Based on Information Refinement
## 3 DIR-Net Method
### Attention-Based Denoising Subnetwork  
After comparing several neural networks, we choose the residual network as the backbone of the denoising subnetwork. A residual network is cascaded by some residual blocks with the same structure, and each block contains several layers. The core idea of the residual block is learning only the difference between the input and output features, which is much similar to the denoising process. With the signal with noise as input, the residual block learns to extract noise and subtracts it from the signal. On the other hand, residual network naturally has the advantages of easy training and preventing gradients vanish. For the design of the denoising subnetwork, we refer to the structure proposed in [34] which is each block contains two convolution layers and one activation layer. We connect an attention module behind each residual block. The reason is that we think the features extract by the network are not all helpful for denoising. Some features may be useless or even negative. The attention module takes the output of the residual block as input, generating a suitable weight for each feature. Multiply these weights by the corresponding features, the features will be enhanced or suppressed. For the design of the attention module, we refer to the structure proposed in CBAM [25]. A convolution layer and a global average pooling layer are used to generate spatial feature weights and channel feature weights, respectively. For the last attention module, all the features from previous residual block are involved in the weight calculation, which makes the generated weight more accurate. By feeding the weighted features to the following residual blocks, a better denoising effect will be obtained. Usually, signal denoising relies more on local features. In other words, the denoising of each information bit is more dependent on its neighbors than distant information bits. Therefore, a convolution network is constructed to perform the denoising task, which is powerful in extracting local structural features in computer vision tasks. The structure of our denoising subnetwork is shown in Figure 1.  
Considering that the signal is one-dimensional, a 1D convolutional layer is utilized to construct our network. At the beginning of the network, a convolutional layer is used to map the signal to the feature space. After that, a series of residual blocks and attention modules are utilized to denoise the signal in the feature domain. The residual blocks are responsible for feature extraction, and the attention modules are responsible for separating valid information and noise. Through this cascade, signals can be gradually refined. The number of residual blocks is set using a tradeoff between denoising quality and speed. Specifically, the structure of our residual block is shown in Figure 2.  
The residual block is composed of a convolution branch and a short connection branch. The convolution branch contains two 1D convolution layers and an activation layer. The output \(H(x)\) of the residual block is defined as  
\[H(s(t))=s(t)+f(s(t)), \tag{11}\]  
where \(s(t)\) is the input of both branches, and \(f(s(t))\) is the output of the convolution branch. The final output is the sum of the two branches. Here, leaky ReLU (LReLU) is utilized as activation function in our network. It is defined as follows:  
\[LReLU(k_{i}(s(t))= \begin{cases}k_{i}(s(t))&if\;k_{i}(s(t))>0\\ a\cdot k_{i}(s(t))&else\end{cases} , \tag{12}\]  
where \(a\) is a constant value that decides the slope of the activation function on the negative semiaxis, and \(k_{i}(s(t))\) is the feature of \(s(t)\) extracted by the \(i\)-th layer of the neural network. As is shown in Figure 3, LReLU can preserve negative values that are beneficial for our task, as the range of our data is \([-1,1]\). In this way, the neural network can be forced to learn the noise distribution and then remove the noise from the input signal via an addition operation.