# An Unsupervised Weight Map Generative Network for Pixel-Level Combination of Image Denoisers
## 2 Related Work
### Consensus Neural Network  
In 2019, CsNet was proposed by [PERSON] et al. They observed that denoised images using different denoising methods are significantly complementary, and there may be a way to combine the denoised images to generate an excellent overall result. Therefore, they integrated different denoisers in the CsNet to generate a combined denoised image, which is better than those obtained by the individual denoising techniques. It mainly handles Gaussian noise. As can be seen in Figure 1, it can be divided into two stages: the combining stage and the boosting stage. The first stage is employed for preliminary estimates to obtain the combined image. For simplicity, we used the same notation as in [28]. A set of initial estimates \(\{\hat{x}_{1},\hat{x}_{2},\dots,\hat{x}_{K}\}\) can be derived using \(K\) image denoisers \(D_{1},D_{2},\dots,D_{K}\). To concatenate these initial estimates, the matrix \(\hat{X}=[\hat{x}_{1},\hat{x}_{2}\dots,\hat{x}_{K}]\in\mathbb{R}^{N\times K}\) is constructed. For image combination, CsNet focuses on the linear combination of estimators, which is used to keep the model unbiased and ensure lower variance. That is, given \(\hat{X}\), the linearly combined estimate can be obtained as  
\[\hat{x}=\sum_{k=1}^{K}w_{k}\hat{x}_{k}=\hat{X}w \tag{2}\]  
where \(w\overset{def}{=}[w_{1},w_{2},\dots,w_{K}]^{T}\in\mathbb{R}^{K}\) is the vector of combination weights. To identify the weights \(w_{1},w_{2},\dots,w_{K}\), [PERSON] et al. used a convex optimal combination framework based on the previous work of [PERSON] [30] to solve a convex optimization problem. Using the estimator \(M\), the CsNet method estimates a set of typical denoisers to generate their respective mean squared error (MSE) [31] of the initial denoised image. The MSE between the ground truth \(x\in\mathbb{R}^{N}\) and the combined estimate \(\hat{x}\in\mathbb{R}^{N}\) is  
\[MSE(\hat{x},x)\stackrel{{ def}}{{=}}\mathbb{E}[||\hat{x}-x||^{2}]= \mathbb{E}[||\hat{X}w-x||^{2}] \tag{3}\]  
According to the \(MSE\) of each denoised image, the optimal combination problem can be solved by obtaining the weight vector \(w\in\mathbb{R}^{K}\) and minimizing the \(MSE\). Subsequently, depending on the optimal weight, it composites the denoised images together. \[ \begin{array}{ll}\underset{\begin{subarray}{c}\mathbf{w}\\ \mathbf{w}\end{subarray}}{\text{minimize}}&\mathbb{E}\big{[}\|\hat{\mathbf{X}}w- \mathbf{x}\|^{2}\big{]}\\ \text{subject to}&\mathbf{w}^{T}\mathbf{1}=1,\quad\text{ and }\quad\mathbf{w}\succeq 0\end{array}  \tag{4}\]  
where the constraints \(w^{T}\mathbf{1}=1\) and \(w\succeq 0\) ensure that the total weights are \(1\) and the combined estimate is nonnegative, respectively. However, after the first stage, the image quality of the combined image is not excellent. To enhance the contrast and recover the lost features of the combined image, a deep learning-based booster was used at the end of the CsNet method, where variables were updated one by one in the iterations. Specifically, based on the current estimate \(\hat{x}^{(t)}\) and the observed image \(y\), the next estimate \(\hat{x}^{(t+1)}\) can be obtained by the equation  
\[\hat{\mathbf{x}}^{(t+1)}=\mathcal{B}_{t}\Big{(}\mathbf{y},\hat{\mathbf{x}}^{(t)}\Big{)}+ \hat{\mathbf{x}}^{(t)} \tag{5}\]  
Here, \(\mathcal{B}_{t}\) is a multi-layer convolutional neural network that implements nonlinear mapping. In the CsNet model, \(\mathcal{B}_{t}\) is composed of three convolutional layers and three deconvolutional layers, and the size of the convolution kernels is \(3\times 3\). The input of the network is the pair \(\Big{(}\mathbf{y},\hat{\mathbf{x}}^{(t)}\Big{)}\). In addition, [PERSON] et al. provided skip connections to ensure that information is not lost while passing through the network layer. To obtain a better improvement effect, the CsNet model uses five cascaded networks \(\mathcal{B}_{t}(t=1,2,\ldots,T)\), where \(T\) is \(3\).