# Dual Residual Denoising Autoencoder with Channel Attention Mechanism for Modulation of Signals
## 3 DRdA-CA Architecture Details
### The Encoding and Decoding Blocks  
The Squeeze-and-Excitation Deep Residual Network (SE-ResNet), as the champion model of Imagenet 2017, is very similar to the Deep Residual Network (ResNet). However, it adds an SE module so that the information between the channels can be exchanged, thereby improving the performance of the network. Figure 3a depicts the original SE-ResNet module. We modify the SE unit in SE-ResNet and use the improved SE-ResNet to establish the encoding and decoding blocks. That is, the fully connected layer (FC) in the original SE unit is replaced by a convolutional layer. This can be attributed as follows. (1) The SE-ResNet was originally designed and implemented under the Caffe framework, in which the FC supports the input of data dimension \(1\times 1\times C\). However, the deep learning framework Pytorch used in this paper does not support the input of data with the same dimension. When the FC is used in the model, both the input and output data require dimension transformation, i.e., three-dimensional data \(1\times 1\times C\) should be reduced and flattened into one-dimensional data C as the input of the fully connected layer. It will increase the computational complexity of the model and destroy the spatial structure of the \begin{table}
\begin{tabular}{c c} \hline \hline
**Layer Name** & **Output Feature Map Size** \\ \hline convolutional layer 1 & \(32\times 2\times 1024\) \\ encoding block 1 & \(32\times 2\times 1024\) \\ encoding block 2 & \(64\times 2\times 512\) \\ encoding block 3 & \(128\times 1\times 256\) \\ decoding block 1 & \(128\times 2\times 521\) \\ decoding block 2 & \(64\times 2\times 1024\) \\ decoding block 3 & \(32\times 2\times 1024\) \\ convolutional layer 2 & \(1\times 2\times 1024\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Denoising Autoencoder’s Parameters. Figure 2: Dual residual denoising autoencoder network framework with channel attention. data, leading to the loss of feature information. This problem can be addressed by replacing the FC with a \(1\times 1\) convolutional layer. (2) Both the convolutional layer and FC perform a dot product operation on the feature maps, thus they have the same functional form. That is, the \(1\times 1\) convolutional layer can be used to replace the FC without changing the function operation of the SE module itself. In addition, since the value of the modulation signals is bipolar with positive and negative numbers, we choose LeakyReLU as the activation function. The improved SE module, as shown in Figure 3b, mainly consists of the following two parts. (1) The squeeze part: Assume that the dimension of the original feature map is \(C\times H\times W\), where \(C\) represents the number of channels in the feature map, \(H\) represents the height of the feature map, and \(W\) is the width of the feature map. The task of the squeeze part is to compress the feature map dimension \(C\times H\times W\) to \(C\times 1\times 1\), which is equivalent to squeezing the matrix in the dimension of of \(H\times W\) into the matrix in the dimension of \(1\times 1\). The squeezed matrix can still obtain the previous global field of view and has a wider perception area. The squeeze operation is implemented using the Global Average Pooling (GAP) method, and its calculation is described as follows:  
\[g_{c}=F_{sq}(v_{c})=\frac{1}{H\times W}\quad\sum_{i=1}^{H}\sum_{i=1}^{W}v_{c}(i,j) \tag{3}\]  
where \(v_{c}\) is the feature map output by the third convolutional layer in the improved SE module. After GAP processing for \(v_{c}\), we get the feature map \(g_{c}\), and the same below. (2) The excitation part: By adding a convolutional layer above and below the nonlinear activation layers, we parameterize the gating mechanism to predict the importance of each channel in the feature map \(g_{c}\). Then the weight values of different channels are obtained and applied to the original feature map to complete the excitation. The excitation operation process is performed as follows.