# Image Degradation Modeling for Real-World Super Resolution via Conditional Normalizing Flow
## 3 Materials and Methods
### Image Degradation Modeling via Conditional Normalizing Flow  
For blur degradation modeling, we parametrize the conditional probability distribution \(p_{y|\mathbf{x}}(\mathbf{y|x})\) of \(\mathbf{y}\) corresponding to \(\mathbf{x}\) with an invertible neural network \(f_{b,\theta}\). The image space of HR-LR image pairs is mapped via \(f_{b,\theta}\) into a latent space:  
\[\mathbf{z}_{b}=f_{b,\theta}(\mathbf{y};\mathbf{x}), \tag{3}\]  
where \(\mathbf{z}_{b}\sim p_{\mathbf{z}_{b}}(\mathbf{z}_{b})\) denotes the latent variable. Since \(f_{b,\theta}\) is required to be invertible, LR image \(\mathbf{y}\) can be generated from a sampled latent variable \(\mathbf{z}_{b}\) once given an HR image \(\mathbf{x}\) as  
\[\mathbf{y}=f_{b,\theta}^{-1}(\mathbf{z}_{b};\mathbf{x}). \tag{4}\]  
The distribution of latent space \(p_{\mathbf{z}_{b}}(\mathbf{z}_{b})\) can be simply assumed to follow a Gaussian distribution \(\mathbf{z}_{b}\sim\mathcal{N}(0,I)\). Thus, the probability density \(p_{y|\mathbf{x}}\) can be explicitly computed as:  
\[p_{y|\mathbf{x}}(\mathbf{y|x},\theta)=p_{\mathbf{z}_{b}}(f_{b,\theta}(\mathbf{ y};\mathbf{x}))\bigg{|}\text{det}\,\frac{\partial f_{b,\theta}}{\partial \mathbf{y}}(\mathbf{y};\mathbf{x})\bigg{|}. \tag{5}\]  
It is obtained by applying a variable change formula to the density, where the second term is the final volume scaling given by the determinant of Jacobian \(\frac{\partial f_{b,\theta}}{\partial\mathbf{y}}\). The Equation (5)  
Figure 1: Overall architecture of IDFlow for image degradation modeling. The blue solid lines correspond to the training procedure, and the blue dotted lines correspond to the inference procedure. allows to train the network and optimize the parameters by minimizing the negative log-likelihood(NLL) loss:  
\[\mathcal{L}(\theta;\mathbf{x},\mathbf{y})=-\log p_{\mathbf{z}_{b}}(f_{\theta}( \mathbf{y};\mathbf{x}))-\log\biggl{|}\det\frac{\partial f_{b,\theta}}{\partial \mathbf{y}}(\mathbf{y};\mathbf{x})\biggr{|}. \tag{6}\]  
Once the distribution is learnt, diverse realistic LR images can be generated by applying the inverse flow \(\mathbf{y}=f_{b,\theta}^{-1}(\mathbf{z}_{b};\mathbf{x})\). Furthermore, to ease the expression of the second term, the network \(f_{b,\theta}\) is decomposed into a combination of N cascaded invertible flow layers. The output of the \(n\)-th layer can be expressed as:  
\[\mathbf{h}^{n+1}=f_{\theta}^{n}(\mathbf{h}^{n};\mathbf{u}), \tag{7}\]  
where \(\mathbf{u}=g_{hr}(\mathbf{x})\) is encoding feature of the input HR image \(\mathbf{x}\) generated by an HR image encoder \(f_{encoder}\). Specially, \(\mathbf{h}^{0}=\mathbf{y}\) and \(\mathbf{z}_{b}=\mathbf{h}^{N}\). Then the Equation (6) can be expressed as:  
\[\mathcal{L}(\theta;\mathbf{x},\mathbf{y})=-\log p_{\mathbf{z}_{b}}(\mathbf{z} _{b})-\sum_{n=0}^{N-1}\log\biggl{|}\det\frac{\partial f_{\theta}^{n}}{\partial \mathbf{h}^{n}}(\mathbf{h}^{n};\mathbf{u})\biggr{|}. \tag{8}\]  
The second term can be efficiently computed by summing the log-determinant of Jacobian \(\frac{\partial f_{\theta}^{n}}{\partial\mathbf{h}^{n}}\) in each layer. Moreover, each layer is required to be invertible and allow a tractable Jacobian determinant. For real-world noise modeling, another invertible neural network \(f_{n,\theta}\) is utilized. Following [10], we collect noise patches from real-world images by setting max noise variance to a certain range and use the patches to train \(f_{n,\theta}\), the objective function can be expressed as:  
\[\mathcal{L}(\theta;\mathbf{n})=-\log p_{\mathbf{z}_{\mathbf{n}}}(\mathbf{z}_ {\mathbf{n}})-\sum_{n=0}^{N-1}\log\biggl{|}\det\frac{\partial f_{n,\theta}^{ n}}{\partial\mathbf{h}^{n}}(\mathbf{h}^{n};\mathbf{u}_{\mathbf{n}})\biggr{|}, \tag{9}\]  
where \(\mathbf{u}_{\mathbf{n}}=\overline{\mathbf{n}}\) is the global mean of input noise patch \(\mathbf{n}\) and is extended to have the same spatial resolution as the input, and \(\mathbf{z}_{\mathbf{n}}\) is assumed to follow \(\mathbf{z}_{\mathbf{n}}\sim\mathcal{N}(0,I)\) as well.