# Denoising Multiscale Back-Projection Feature Fusion for Underwater Image Enhancement
## 3 Methodology
### Multiscale Back-Projection Feature Fusion  
The encoder of the U-Net structure gradually reduces the size of the feature map during downsampling and expands the perceptual field to better visualize the global information of the feature map. However, the edge information and texture details are lost when all feature maps are transformed to the same scale for fusion. The underwater environment is complex and variable, and the blur and color bias of underwater images are also very uneven in terms of the different depths of objects in the images, which highlights  
Figure 2: Up- and downprojection units proposed by the DBPN. the drawbacks of the U-Net structure. The recovery effect is poor for underwater images with uneven distributions of color bias blur, and information loss leads to a lack of texture details in the restored images. The multiscale back-projection feature fusion mechanism (MBFF) continuously superimposes multiple scale feature maps to perform scale transformation and feedback errors iteratively. As shown in Figure 3, the input feature is \(F\), and the features of different scales \(F_{1},\dots,F_{m}\) are fused with the input from the back-projection feature fusion block. \(F_{m}\) is the feature map for the scale \(S_{m}\) with \(m\in\{1,\dots,M\}\). The process for one back-projection feature fusion is formally formulated as follows:  
For feature \(F_{m}\), the back-projection feature fusion block first upsamples/downsamples the original feature, \(F\), several times until it is the same size as \(F_{m}\). Then, it computes the difference, \(e_{m}\), between the intermediate map, \(F_{b}\), and \(F_{m}\). Then, the downsampling/upsampling process restores the residual map, \(e_{m}\), to its original size. Finally, the back-projecting map is added to the original feature, \(F\), to obtain the fused feature, \(F^{\prime}\). The process of fusion feature, \(F_{m}\), using MBFF is illustrated in Algorithm 1. This Algorithm handles situations where \(F_{m}\) has a larger size than \(F\), requiring upsampling followed by downsampling. If \(F_{m}\) is smaller than \(F\), the operations are reversed, with downsampling preceding upsampling. \[MBFF_{en}(F,F_{m}) =F+Con(Diff(F,Decon(F_{m}))), \tag{5}\] \[MBFF_{de}(F,F_{m}) =F+Decon(Diff(F,Con(F_{m}))),\]  
where \(Decon()\) is the deconvolution operator used to scale up \(F_{m}\) to the same size as \(F\), and \(Con()\) denotes the convolution operation that scales down \(F_{m}\) to the same size as \(F\). \(Diff(,)\) is the calculation of the residual between \(F\) and the scaled \(F_{m}\). ```
Input: Feature \(F\), Fusion feature \(F^{m}\) Output: The fused feature \(F^{\prime}\)
1 Calculate deconvolution and convolution times: \(Size_{in}\gets size(F)\) \(Size_{out}\gets size(F_{m})\) \(times=(Size_{out}//Size_{in})//2\) \(F_{b}\gets Decon_{times}(F)\) Calculate the differences between \(F_{b}\) and \(F_{m}\): \(e_{m}\gets Diff(F_{b},F_{m})\) Down-sampling the differences \(e_{m}\) with multiple convolutions: \(e_{m}\gets Con_{times}(e_{m})\) Add the back-projection feature with \(F\): \(F^{\prime}=F+e_{m}\)
2
3 end for
```  
**Algorithm 1**Feature fusion process of MBFF for \(F_{m}\). Figure 3: Details of the multiscale back-projection feature fusion mechanism. When taking the fourth encoder, \(EB^{4}\), as an example, it merges the output of the features from three separate encoders: \(EB^{1}\), \(EB^{2}\), and \(EB^{3}\). These features are denoted as \(F_{1}\), \(F_{2}\), and \(F_{3}\), with different sizes, as outlined in Table 1. They have the size of \(256\times 256\) (\(F_{1}\)), \(128\times 128\) (\(F_{2}\)), and \(64\times 64\) (\(F_{3}\)). The input feature, \(F\), for encoder \(EB^{4}\) has a size of \(32\times\ 32\times 128\). In order to fuse with \(F^{3}\), the input feature, \(F\), is upsampled once using deconvolution operations with a filter size of \(3\times 3\) and a stride of 2. After upsampling, \(F\) has a size of \(64\times 64\).