# Demosaicing by Differentiable Deep Restoration
## 2 Related Work
### Demosaicing  
Most of the traditional demosaicing methods are developed to deal with the raw image filtered by a certain CFA, e.g., Bayer pattern [1]. They can be roughly split into spatial domain methods and frequency domain methods.  
Among spatial domain methods, bilinear interpolation from the nearby pixels of the same color channel is the simplest one, which works well in flat image regions, but suffers from severe artifacts and losses in details. Some linear demosaicing methods [23; 24] working in a data-driven way by Linear Minimum Mean Square-Error Estimation (LMMSE) achieves good performance with fast running time. Rather than processing each color channel independently, some interpolation methods [25; 26] focused on inter-channel correlation based on the constant color difference assumption. The edge information was utilized in demosaicing techniques [27; 28; 29] to alleviate the artifacts caused by interpolation across edges. Some other properties like self-similarity [30] and data-redundancy [31] were exploited to improve the quality of reconstruction. Post-processing method [32] was also studied to improve the image quality of the initially demosaiced image.  
Among frequency domain methods, several techniques were developed to estimate the components in frequency domain to recover color image. In [33], the demosaicing method was designed for raw image generated by Bayer pattern, in which green color owns the most sample rate. Green component was estimated first by a diamond shape 2D filter in frequency domain, then the high-frequency information of green channel was added to improve the reconstruction of red and blue channels. Some demosaicing methods were proposed for more general situation, without restriction to one certain CFA. Luminance and chrominance components were directly estimated by appropriate frequency selection in [19], and the color image was recovered from these frequency components. We refer [34] to have a complete review of traditional demosaicing methods.  
Recently, deep neural networks present impressive improvements on the demosaicing task. [PERSON] et al. [9] proposed a joint approach for demosaicing and denoising by using CNN to directly regress the color image. [PERSON] et al. [35] designed a two-stage demosaicing architecture for Bayer pattern CFA. The green channel was recovered in the first stage and the red and blue channels were estimated by residual learning in the second stage. Based on the observation that the correlations between different channels are quite different, [PERSON] et al. [36] proposed a 3-stage CNN structure for demosaicing, in which two separated networks were used to reconstruct red and blue channels. [PERSON] et al. [37]designed an effective feature extraction network for joint demosaicing and denoising. [PERSON] et al. [38] exploited green channel and density map as guidance in network structure for joint demosaicing and denoising. In contrast to most traditional demosaicing methods, demosaicing methods with deep neural networks are more flexible with the ability to work on raw image filtered by any CFA.  
To the best of our knowledge, the methods of [PERSON] et al. [39; 40] is the only deep learning based demosaicing method that takes the degradation model into account. They trained an iterative network for joint demosaicing and denoising based on the given CFA. Thus, the demosaicing method was still optimized isolated from the design of CFA. In contrast, we go one step further, the CFA pattern is jointly learned in our method, which owns more benefits. Furthermore, our method is more convenient to train and more effective and efficient to execute by directly generating the color image in a closed-form solution other than in an iterative way.