# Improved CycleGAN for Mixed Noise Removal in Infrared Images
## 2 Methods
### Architecture of the Improved Network  
We have realized the mapping between noisy images to denoised images by redesigning the generators, discriminators and the cycle consistency loss function in the original CycleGAN. The specific architecture of the improved network is shown in Figure 1, where \(\mathbf{G}\) generates the fake noisy image from the input real clean image. \(\mathbf{F}\) is responsible for recovering the fake clean image from the input real noisy image. \(\mathbf{D}_{\mathbf{N}}\) is used to discriminate the fake noisy image from the true noisy image. \(\mathbf{D}_{\mathbf{C}}\) is used to determine whether the input image is a true clean image or not. The cycle consistency loss is used to ensure the similarity of non-noise information between the noisy images and the denoised images during the denoising process. **Generator Design:** In order to achieve a better removal effect for the mixed noise, the generator of the improved network adopts a structure based on encoder-feature learning module-decoder, which is shown in Figure 2a. At first, for a noisy infrared image with height \(h\) and width \(w\), the encoder downsamples it using three feature extraction modules consisting of convolution layer, InstanceNorm layer and ReLU function. The input image produces an h \(\times\) w \(\times\) 64 feature map after the first feature extraction module, and then the width and height dimensions of the feature maps produced by each module after that are halved and the number of channels is doubled. Finally, an h/4 \(\times\) w/4 \(\times\) 256 feature map is generated and put into the feature learning module. Feature learning module consists of several ResNet-E Blocks connected in series. The ResNet-E Block is a residue-like structure containing three branches we proposed; its specific structure is shown in Figure 2b, which inherits the two branches of the original ResNet Block, and additionally adds a new branch containing the EMA attention mechanism [26]. EMA is a multi-scale attention module with cross-spatial learning capability, which divides a given input feature map into N sub-feature maps along the direction of the channel dimension and extracts the attention weights of each sub-feature map by three parallel routes (where X denotes 1D horizontal global pooling and Y denotes 1D vertical global pooling); then, the extracted feature information is inputted together  
Figure 1: The architecture of the improved denoising network, where \(\mathbf{G}\) and \(\mathbf{F}\) represent generators and \(\mathbf{D}_{\mathbf{N}}\) and \(\mathbf{D}_{\mathbf{C}}\) represent discriminators. into cross-spatial learning block for establishing interdependencies between channels and spatial locations. For a given input feature map \(f\), the output feature map \(F\) obtained after processing by ResNet-E Block can be expressed as follows:  
\[F=f+Conv(Conv(f))+EMA(f) \tag{4}\]  
The feature maps outputted after the feature learning module are delivered to the decoder which consists of ConvTranspose layer and convolution layer for reconstructing the clean image without noise. At the same time, we also use the skip-connection structure to link the encoder with the decoder in order to realize the fusion of low-level features obtained after each convolution in the encoder with high-level features produced by the deconvolution of the decoder at the same resolution, so as to improve the utilization of feature information of each layer in the model. **Discriminator Design:** The structure of the discriminator is shown in Figure 3. Except for the last layer, which is a convolution operation with depth 1, all other layers are composed of 4 \(\times\) 4 convolution module, InstanceNorm layer and LeakyReLU function.