# Conditional GAN-Based Two-Stage ISP Tuning Method: A Reconstruction-Enhancement Proxy Framework
## 3 Methods
### Loss Function  
The two stages of ReEn-GAN use different loss functions respectively, in order to obtain different optimization objectives and losses of images in the two stages. In the reconstruction stage, in order to capture the detail of the image, as well as to make \(G_{Re}\) faithfully reconstruct the scene--which contains genuine image structures from RAW image data--and the effect of \(P_{Re}\) on images, specific loss functions are required. In ISP restoration, it is necessary for the network to learn the detailed reconstruction features of the image. Therefore, based on \(\mathcal{L}_{GAN}\), the addition of \(\mathcal{L}_{1}\) can enable the network to learn the detailed features of the image without causing blurring like \(\mathcal{L}_{2}\). Meanwhile, by incorporating the perceptual loss of pre-trained network loss \(\mathcal{L}_{VGG}\), the generated images of the network can be perceived similarly rather than at the pixel-level. Therefore, the mutual compensation between \(\mathcal{L}_{1}\) and \(\mathcal{L}_{VGG}\) can enhance the learning ability of the network. As mentioned earlier, the use of \(\mathcal{L}_{patchGAN}\) can also improve the network's perception of details and enhance its overall generalization ability. Therefore, we use the combination of \(\mathcal{L}_{1}\), \(\mathcal{L}_{patchGAN}\), and \(\mathcal{L}_{VGG}\) as the loss function in the reconstruction stage. The loss functions are shown as follows:  
\[ \begin{cases}\mathcal{L}_{Re}=\mathcal{L}_{patchGAN}+\alpha_{1}\mathcal{L}_{1 }+\alpha_{2}\mathcal{L}_{VGG}\\ \mathcal{L}_{patchGAN}(G_{Re},D_{Re})=\mathbb{E}_{I,I_{GT}}[\log D(I,I_{GT})]+ \mathbb{E}_{I}(\log\{1-D[I,G(I)]\})\\ \mathcal{L}_{1}=\mathbb{E}_{I,I_{GT}}[||I-I_{GT}||_{1}]\\ \mathcal{L}_{VGG}(I,I_{GT})=\min\limits_{G_{Re}}\sum\limits_{i=1}^{5}\frac{1}{ C_{i}W_{i}H_{i}}\Big{[}||B_{i}(I)-B_{i}(I_{GT})||_{2}^{2}\Big{]}\end{cases}  \tag{7}\]  
where \(\alpha_{1}\) and \(\alpha_{2}\) control the importance of \(\mathcal{L}_{1}\) and \(\mathcal{L}_{VGG}\). \(\mathcal{C}_{i}\), \(W_{i}\) and \(H_{i}\) indicate the channels, width, and height of the corresponding output feature map, respectively. Using \(\mathcal{L}_{1}\) instead of \(\mathcal{L}_{2}\) is to avoid the blur effect. \(\mathcal{L}_{VGG}\) is based on a pre-trained VGGNet [38]. In the enhancement stage, in order to capture the color and global perceptual distribution of the image, as well as to make \(G_{En}\) faithfully reconstruct the scene--which contains genuine image color from RGB image data--and the effect of \(P_{En}\) on images, specific loss functions are also required. In the enhancement stage of ISP, it is necessary for the network to learn the overall information of the image, while also preserving some details of the image so that the proxy can more effectively control the influence of ISP parameters on the image. Therefore, on the basis of \(\mathcal{L}_{GAN}\), adding \(\mathcal{L}_{1}\) enables the proxy to learn the pixel-level information of the image more effectively, and by adding \(\mathcal{L}_{color}\)[39], the network can learn macro color information. Through \(\mathcal{L}_{color}\), the proxy can learn the influence of parameters on the chromaticity of the image more effectively. By integrating \(\mathcal{L}_{globalGAN}\) and \(\mathcal{L}_{color}\), as well as complementing \(\mathcal{L}_{1}\), the network can be trained more effectively and enhance overall generalization. Therefore, we use the combination of \(\mathcal{L}_{1}\), \(\mathcal{L}_{globalGAN}\), and \(\mathcal{L}_{color}\) as loss function in the enhancement stage. The loss functions are shown as follows:\[ \begin{cases}\mathcal{L}_{En}=\mathcal{L}_{globalGAN}+\beta_{1}\mathcal{L}_{1}+ \beta_{2}\mathcal{L}_{color}\\ \mathcal{L}_{globalGAN}(G_{En},D_{En})=\mathbb{E}_{I,I_{GT}}[\log D(I,I_{GT})]+ \mathbb{E}_{I}(\log\{1-D[I,G(I)]\})\\ \mathcal{L}_{1}=\mathbb{E}_{I,I_{GT}}[||I,I_{GT}||_{1}]\\ \mathcal{L}_{color}(I,I_{GT})=||I_{b}-I_{GTb}||_{2}^{2}\end{cases}