# Wasserstein Generative Adversarial Network Based De-Blurring Using Perceptual Similarity
## 2 Preliminaries
### Perceptual Loss Function  
The perceptual loss function is used in image transfer tasks by extracting representations of a feature map. There are two types of perceptual losses which are the content loss function and the style loss function. Basic merit of the perceptual loss function is the ability to extract feature responses in layers of CNN. As the network goes deeper, the input image is changed to be representations of features, not of pixel values. That is because features of the high layer have large receptive fields, and it represents the actual content of the image and spatial structure. Therefore, the high layer of the network can capture high-level contents and perceptual information, as in Figure 7, and the feature response of high layers is called the content representation.  
To minimize the difference of content representation between the input and target images, the content loss function was proposed in [10], and it is represented as follows:  
\[\mathcal{L}_{\mathrm{content}}^{\varnothing,j}\ (x,\hat{x})=\ \frac{1}{C_{j}H_{j}W_{j}} \big{\|}\mathcal{O}_{\mathrm{j}}(\hat{x})-\mathcal{O}_{\mathrm{j}}(x)\big{\|} _{2}^{2}, \tag{8}\]  
where \(x,\hat{x},\ \mathcal{O}_{\mathrm{j}}\) and \(C_{j}\times H_{j}\times W_{j}\) are input image, target image, feature map of j-th layer and size of feature map, respectively. Usually, feature map is extracted in pre-trained networks by ImageNet datasets, such as VGGNet16 or VGGNet19. The content loss function is calculated by mean square error.  
In the de-blurring task, using the content loss function for a network encourages the output image to be perceptually similar to the sharp image, since it can capture the content of sharpness information. To capture more perceptual similarity and texture information, the style loss function was proposed in [11]. The style loss function is similar to the content loss function, which uses feature responses in layer of network. However, the style loss function consists of the correlations between different  
Figure 6: Graph of Equation (5).  
feature responses, and it extracts multiple feature maps of network. By considering the correlation of features, the style loss function obtains multi-scale representations. Correlation of features is given by a Gram matrix, which is a covariance matrix that represents the distribution of an image, and the Gram matrix is the inner product between different vectored feature maps. The Gram matrix for style loss is expressed as follows:  
\[\mathrm{G_{j}^{\varnothing}\left(x\right)}_{c,c^{\prime}}=\frac{1}{C_{j}H_{j}W_ {j}}\sum_{h=1}^{H}\sum_{w=1}^{W}\varnothing_{\mathrm{j}}(x)_{h,w,c}\varnothing_ {\mathrm{j}}(x)_{h,w,c^{\prime}} \tag{9}\]  
where \(\mathrm{G_{j}^{\varnothing}\left(x\right)}\), \(\varnothing_{\mathrm{j}}(x)_{h,w,c}\) and \(\varnothing_{\mathrm{j}}(x)_{h,w,c^{\prime}}\), and \(C_{j}\times H_{j}\times W_{j}\) are the Gram matrix of feature map in j-th layer, different feature maps with different channels, and the size of feature map, respectively.  
For the efficient calculation of the Gram matrix, feature maps are reshaped by changing the size of \(C_{j}\times H_{j}\times W_{j}\) to \(C_{j}\times H_{j}W_{j}\). After reshaping, the style loss function is calculated using difference between the Gram matrices of the target and the output images. It is minimized by using the Frobenius norm, and the formula of the style loss function is as follows:  
\[\mathcal{L}_{\mathrm{style}}^{\varnothing,j}=\frac{1}{C_{j}H_{j}W_{j}}\Big{\|} \mathrm{G_{j}^{\varnothing}\left(\hat{x}\right)-G_{j}^{\varnothing}(x)}\Big{\|} _{\mathrm{F}}^{2}. \tag{10}\]  
As a result, applying the syle loss function to WGAN can preserve more detailed edge information and capture the similarity of perceptual styles.