Fast generative adversarial networks model for masked image restoration

IET Image ProcessingVolume 13, Issue 7 p. 1124-1129 Research ArticleFree Access Fast generative adversarial networks model for masked image restoration Zhiyi Cao, Corresponding Author Zhiyi Cao 68545849@qq.com orcid.org/0000-0003-2698-8507 Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorShaozhang Niu, Shaozhang Niu Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorJiwei Zhang, Jiwei Zhang Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorXinyi Wang, Xinyi Wang Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this author Zhiyi Cao, Corresponding Author Zhiyi Cao 68545849@qq.com orcid.org/0000-0003-2698-8507 Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorShaozhang Niu, Shaozhang Niu Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorJiwei Zhang, Jiwei Zhang Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorXinyi Wang, Xinyi Wang Beijing Key Lab of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this author First published: 25 April 2019 https://doi.org/10.1049/iet-ipr.2018.5592Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The conventional masked image restoration algorithms all utilise the correlation between the masked region and its neighbouring pixels, which does not work well for the larger masked image. The latest research utilises Generative Adversarial Networks (GANs) model to generate a better result for the larger masked image but does not work well for the complex masked region. To get a better result for the complex masked region, the authors propose a novel fast GANs model for masked image restoration. The method used in authors' research is based on GANs model and fast marching method (FMM). The authors trained an FMMGAN model which consists of a neighbouring network, a generator network, a discriminator network, and two parsing networks. A large number of experimental results on two open datasets show that the proposed model performs well for masked image restoration. 1 Introduction Masked images are usually used to protect information, but they also bring difficulties in legally restoring the truth of the image. Therefore, using image inpainting technology to restore images legally and effectively is of great importance to forensic work. Masked image restoration refers to the use of computer technology, analysis of the causes of the mask, and possible results, restore the closest truth. It is well known that the generation and restoration of the mask is an irreversible process, so the restoration of the mask is a challenging research. The conventional masked image restoration algorithms take advantage of the correlation between the masked area and its neighbouring pixels to restore the truth of the image. For example, in Fig. 1d, if the mask is in a smaller area of Jordan's face, the neighbouring area provides enough pixels to restore the truth. However, once the scope of the mask has expanded, such as expanding to the global left face area, no relevant similar pixels can be found and the result of restoration will be distorted or wrong. Fig. 1Open in figure viewerPowerPoint Masked image restoration results. In each row from left to right (a) Masked image (96 × 96 pixels),(b) Masked image restoration resultsby FMM (96 × 96 pixels), (c) Masked imagerestoration results by our model (96 × 96 pixels),(d) Original image (96 × 96pixels). In the left row, the masked image is produced from the originalimage. In the bottom row we show masked image restoration results For the challenging research, the most effective solution is the face completion model (FACEGAN) [1], which takes advantage of generative adversarial networks (GANs) [2] model to restore face from a larger area. The FACEGAN model includes multiple loss and multiple networks. However, the model is time-consuming and some of the mask area itself is very unique or complex, such as Jordan's eyes area, the FACEGAN model is difficult to restore. In addition, the result of the restoration is whole-oriented rather than individual-oriented. For example, if given a red masked lip and the FACEGAN mode learns a lot of ordinary people's lips, we can only get a common colour lip rather than a red lip. This is a common problem of the CNN. Based on the above analysis, the FACEGAN model has a limit effect to generate a right result for the complex masked region. To solve this problem, we find that the neighbouring pixels provide material for generating better results. Further analysis proves that if given images like Fig. 1a, we can get images like Fig. 1b. However, for the complex task of masked image restoration, the generator loss of the conventional GANs is too small to generate the right images. Inspired by the FACEGAN model, we first propose using sqrt loss to increase the pixel loss between the generated image and the real image. Pixel loss is used to compensate for the numerical disadvantage of generator loss and to improve pixel-level removal. In addition, we propose increasing the content loss using sqrt loss to calculate the content loss between the generated image and the real image. The content loss is used to supplement the visually deficient of generator loss and improve the removal of deep features. The generator loss of the entire model consists of the adversarial loss, the pixel loss, and the content loss. The combination of three kinds of loss guarantees that the masked image restoration result is both numerically and visually reliable. The main contributions of this paper are summarised as follows: First of all, we propose a novel fast GANs model (FMMGAN) for complex masked image restoration, which including a neighbouring network, a generator network, a discriminator network, and two parsing networks. Second, we first use FMM to help the GANs model generating better restoration result with more neighbouring pixels. In addition, our model is very fast. Finally, unlike previous models, we first use the sqrt loss to calculate pixel loss and content loss and restore the masked image. We propose a new idea of measuring the stability and efficiency of the generation model, and mathematically verify the great advantage of FMMGAN model in increasing pixel loss and content loss. 2 Related work 2.1 Masked image restoration Early masked image restoration algorithms include a diffusion equation [3], texture synthesis algorithm [4], and an efficient patch matching algorithm [5]. Such algorithms are simple and fast, they perform well when similar patches can be found, they are likely to fail when the source image does not contain sufficient amount of data to fill in the unknown regions. This paper focuses on fast marching method (FMM) [6]. The algorithm starts from the boundary of this region and goes inside the region gradually filling everything in the boundary first. It takes a small neighbourhood around the pixel on the neighbourhood to be inpainted. This pixel is replaced by the normalised weighted sum of all the known pixels in the neighbourhood. Although masked image restoration results are not perfect, it provides more neighbouring pixels for the generation model. The latter masked image restoration algorithm is Shepard convolutional neural networks [7], it learns a convolutional network for masked image restoration. The best masked image restoration model is the FACEGAN model. It contains four networks: one generator network, two discriminators networks, and a parsing network. The corresponding loss also has four parts, the adversarial loss, the global discriminator loss, the local discriminator loss, and the parsing loss. This model effectively addresses the problem of images restoration. It can generate popular images, but cannot generate personalised images. 2.2 Image generation To our knowledge, the three popular models in image generation are VAEs, GANs, and the PixelCNN. Variational self-encoders (VAEs) [8] is a generation model, which is utilised to learn the true distribution and generate images. However, the quality of the generated image is not high. The GANs model is an unsupervised generation method. GANs model consists of two parts: one is the generator and the other is a discriminator. The generator is like a thief, and the discriminator is like a police, the purpose of the thief is to find ways to deceive the police, and the police aim is to find ways to not be deceived. The thieves and police are constantly optimising themselves to achieve their goals, while each other in the other's adversarial rising. The GANs model is the most popular generation model by far. Another generation model is PixelCNN [9]. They propose a model based on pixel convolution neural network (PixelCNN) architecture, which generates new images according to the change of conditions. However, the training process of PixelCNN model is very slow, and the quality of the generated images is not good as the GANs model. For the complex masked image, the pixel recursive super resolution (PRSR) [10] offers the inspiration of neighbouring pixels, which takes advantage of two deep convolution networks to obtain better pixel matching points. The global restoration process used two neural networks. First, adjust the network to compare low-resolution photos to high-resolution photos in the database. The next priority network will guess those details as a feature of high-resolution photos. The best guess of the last two neural networks will be integrated, eventually forming a pixel-level recovery results. However, the model is time-consuming and in the poor global visual sense. 3 FMMGAN model design In this section, we will introduce our proposed FMMGAN model for complex masked image restoration. Given a masked image, such as Jordan's mask image in Fig. 1a, the goal of the model is to use the generation model to recover it close to the output of the real image. Our model, shown in Fig. 2, includes a neighbouring network, a generator network, a discriminator network, and two parsing networks. Fig. 2Open in figure viewerPowerPoint Network architecture. It consists of one neighbouring, one generator, one discriminator and two parsing networks 3.1 Neighbouring network The goal of the neighbouring network is to generate better-masked images. The original masked images, as shown in Fig. 1a, are usually difficult to generate the same result as the real image. For the complex masked region, the GANs model learns global features and cannot generate personalised images. In this section, we will introduce FMM. If we want to restore the point p situated on the boundary of the masked region , take a small neighbourhood of size of the known image around p. The restoration of p should be determined by the values of the known image points close to p, i.e. in . We first consider grey value images, colour images as being a natural extension. For small enough, we consider a first-order approximation Iq(p) of the image in point p, given the image ∇I(q) and gradient I(q) values of point q: (1)Next, we restore point p as a function of all points q in by summing the estimates of all points q, weighted by a normalised weighting function w(p, q): (2)The weighting function w(p, q) is designed such that the restoration of p propagates the grey value as well as the sharp details of the image over . We design w(p, q) = dir(p, q) × dst(p, q) × lev(p, q): (3)The factor dir(p, q) ensures that the contribution of the pixels close to the normal direction N = ∇T (gradT in the code), i.e. close to the FMM's information propagation direction, is higher than for those farther from N. The factor dst(p, q) decreases the contribution of the pixels geometrically farther from p. The level set distance component lev(p, q) ensures that pixels close to the contour through p contribute more than farther pixels. Both dst and lev are relative with respect to the reference distances d0 and T0. In practice, we set d0 and T0 to the interpixel distance, i.e. to 1. Overall, the above factors model the manual restoration heuristics that describe how to restore a point by strokes bringing colour from a small region around it. The output of the neighbouring network is shown in Fig. 1b. 3.2 Generator network The goal of the generator network is to generate learned features. Unlike the original GANs model, the input of FMMGAN's generator network is the output of the neighbouring network. Generator network directly inputs 16 pieces of 96 × 96 FMM processed images. Generator network using a symmetrical encoding and decoding structure. The image is first reduced to 12 × 12 by the convolutional network and then enlarged to 96 × 96. With the help of this self-encoding process, we can get more hidden features of images to ensure the quality of the generated image. Here, the generator network is defined as G. 3.3 Discriminator network The goal of the discriminator network is not to be deceived by the output of the generator network. The results of the generator network are not targeted, which requires the discriminator network to ensure that. The input of the discriminator network is generated by the generator network output. Discriminator network needs to make a distinction between the result of the generator network and the real image, which is a two classification problem. Here, we define the network as D. 3.4 Parsing networks Without a parsing network, the model can be viewed as the deep convolutional adversarial networks (DCGAN) [11]. Experiments show that the DCGAN model has a limited effect on such complex tasks as masked image restoration. Even though we adopted the more stable adversarial model (WGAN) [12] and the upgrade of WGAN model [13] or LSGAN [14], the effect is still poor. Finally, inspired by FACEGAN model which proposed the parsing network for face completion, this paper proposes to extract the features by using the pre-trained VGG19 [15] network model. The goal of the parsing network is to extract the deep features of generated images and real images. Here, we use a pre-trained VGG19 model with a size of 96 × 96 to extract features. The parsing network first loads the parameters of the trained VGG19 model and then inputs the generated image. Then, through the VAE, get the features of the generated image. The feature extraction process of the real image is similar to the above. Here, the parsing results of the generated images and real images are defined as Pf and Pt. 3.5 FMMGAN objective function If we define the masked image as Im, the generated image as Ig and the real image as It, then the minimisation problem of FMMGAN model can be defined as follows: (4)The biggest improvement here is the calculation of generator loss. To explain the whole process in more detail, we introduce sqrt loss.Sqrt loss represents square root calculations loss of the current image X and reference image Y, and b, h, w and c represent the number, height, and width and the number of channels of the images, respectively. Here, X represents the generated image, Y represents the real image, then the mean square error formula can be defined as follows: (5)As part of the generator loss, we should minimise PIXELloss (LP) which represents the pixel loss and control it between 0 and 1 based on (5). The final LP is as follows: (6)If only increasing the pixel loss, the recovery effect is still not good enough due to the large visual difference between the generated image and the real image. Therefore, we proposed a CONTENTloss (LC) to represent the content loss. After the above parsing network, the parsing result of the generated image is defined as Pf, and the parsing result of the real image is defined as Pt. Therefore, the definition of LC is as follows: (7)In addition to the pixel loss and the content loss, we also add the adversarial loss ADVloss (LA). The adversarial loss is based on cross-entropy (C). In this formula, a is the output of the discriminator network, and y is 0 or 1. The final definition of C is as follows: (8)If we define the output of the discriminator network as Df, the final definition of LA is as follows: (9)Finally, if we define the variables r1, r2, and r3 and r1 + r2 + r3 = 1. Then, our generator loss (LG) is defined as follows: (10) 3.6 Why sqrt loss? For such a challenging task as masked image restoration, we focus on how to increase the generator loss. In order to find out the change rule of pixel loss for various generation models, we should study the objective function. First assume that we get the mathematical difference q which is between the generated image Ig and the real image It and it is distributed between 0.01 and 1. The generator network objective function of the GANs model is as follows: (11)The pixel loss calculated by the GANs model can be defined as follows: (12)In order to further control its output between 0 and 1, and to ensure that f (q) = 1 when q is equal to 1, the pixel loss calculated by the GANs model can be approximated as: (13)The generator network of DCGAN model is the same as GANs model, so the pixel loss calculated by the DCGAN model can be defined as follows: (14)The definition of the generator network objective function of the WGAN model is as follows: (15)Based on the above ideas, we can also control its output between 0.01 and 1, and ensure that when q is equal to 1, f (q) = 1, the pixel loss calculated by the WGAN model can be defined as follows: (16)Comparing the method of pixel loss calculation by the FACEGAN model and the PRSR model, and WGAN model, we will find that it is essentially the same. So the pixel loss calculated by them can be defined as follows: (17)As shown in Fig. 3, GANS model can be used to obtain larger pixel loss than WGAN model, but there is a collapse problem when the mathematical difference is very small. This is the famous question of GANs instability. At the same time, it is shown that WGAN is more stable than the original GAN, but get less pixel loss. Fig. 3Open in figure viewerPowerPoint Under the same mathematical difference condition, evaluate pixel loss of different generation models The objective function of generator network for LSGAN is as follows: (18)Based on the above ideas, the pixel loss calculated by the LSGAN model can be defined as follows: (19)As shown in Fig. 4, GANS model can be used to obtain larger pixel loss than LSGAN model, but there is a collapse problem when the mathematical differences are very small. It also shows that LSGAN is more stable than GANs but not very efficient. According to (5), the pixel loss calculated by the FMMGAN model can be defined as follows: (20) Fig. 4Open in figure viewerPowerPoint Under the same mathematical difference condition, evaluate pixel loss of different generation models As shown in Fig. 5, FMMGAN model can be used to obtain larger pixel loss than LSGAN model, but there is a small collapse problem when the mathematical differences are minimal, and FMMGAN model is more stable than GANs model and more efficient than LSGAN model and WGAN model. Fig. 5Open in figure viewerPowerPoint Under the same mathematical difference condition, evaluate pixel loss of different generation models 4 Experiment We carry on the experiment for masked image restoration on two open source datasets. First, using FMM and the DCGAN model for masked image restoration, it proves that they cannot accomplish this challenge. Then, the FMMGAN model is used to restore images, which proves the performance of this model. Finally, we compare the FMMGAN model with other generation models for masked image restoration [16-20]. 4.1 Datasets and pre-processing The ImageNet dataset [21] contains 14 million images. We use it to train a 96 × 96 VGG19 model before the experiment. The parsing networks in the following experiment will use this trained VGG19 model. The experiment was first conducted on the LFW [22] dataset, which contains 5749 people and a total of 13,233 face images. This paper uses 11,000 images to train model, 2233 images to test model. It was then conducted on the cars [23] dataset, which contains 196 categories, a total of 16,185 vehicle images. This paper uses 14,000 images to a training model, 2185 images to do the test. Before starting the experiment, we should resize the image to 96 × 96. 4.2 Experimental details We implement the FMMGAN model in the open source Tensorflow framework, which uses GTX1080Ti GPUs for both training and testing. Each batch of training randomly loads 16 images from each dataset as real images, then copies the images and processes them into masked images as input images. The next step uses the model structure shown in Fig. 2 to train the model parameters. Finally, the test images are processed into masked images, then only the generator network is used to generate the output result by using the trained model. The experimental results of the FMMGAN model on two datasets are shown in Fig. 6: Fig. 6Open in figure viewerPowerPoint Masked image restoration results. In each row from left to right (a), (d) Masked image (96 × 96 pixels),(b), (e) Masked image restorationresults by our method (96 × 96 pixels),(c), (f) Original image (96 × 96pixels). In the left row, masked image is produced from the originalimage. In the bottom row we show masked image restorationresults 4.3 Evaluation index In order to evaluate the quality of the generated images, we use a total of four evaluation indicators: PSNR, SSIM, MS-SSIM, TIME. PSNR (peak signal-to-noise ratio), which is the error-sensitive image quality evaluation. The unit of PSNR is dB, the larger the value, the smaller the distortion. SSIM [24] (structural similarity), which, respectively, measures the image similarity from three aspects: brightness, contrast, and structure. SSIM ranges from −1 to 1. MS-SSIM (multi-scale-structural similarity index) which, respectively, measures multi-level structural similarity. Model training time (referred to as TIME), it evaluates the performance of the model in terms of time complexity; the unit is the hour. 4.4 Experimental comparison For the LFW dataset, we compared the masked image restoration performance with four models: the FMMGAN model, the DCGAN model, the FACEGAN model, and the PRSR model. In order to be able to effectively compare with this model, the FACEGAN model [https://github.com/Yijunmaverick/GenerativeFaceCompletion.] and the PRSR model [https://github.com/nilboy/pixel-recursive-super-resolution.] have the same input and real image as FMMGAN model. We first train the model using training images and then compare the effects of different models on 2233 test images. The contrast effect of the LFW dataset is shown in Fig. 7: Fig. 7Open in figure viewerPowerPoint Masked image restoration results. In each row from left to right (a) Masked image (96 × 96 pixels),(b) Masked image restorationresults by DCGAN (96 × 96 pixels), (c)Masked image restoration results by FACEGAN (96X96 pixels),(d) Masked image restorationresults by PRSR (96 × 96 pixels), (e)Masked image restoration results by our method (96 × 96 pixels),(f) Original image (96 × 96pixels) From a visual point of view, in Fig. 7, we find that the FMMGAN model generates better images than the other models, which are closest to the real image. PRSR model generates relatively good images, these images are relatively clear. Although the FACEGAN model generates blurred images, the outline can still be discerned. The DCGAN model is almost nothing generated. For each model, we trained 100 times, respectively, and recorded the training time, then saved the model parameters. The average of training time is taken as the value of time complexity (TIME). The masked image restoration result is then generated using the model parameters. Finally, the results of PSNR, SSIM, and MS-SSIM indices are calculated by the generated images and the test images. We record the calculation results for each image, and the average of 2233 images is taken as the final result. The contrast effect on four evaluation indicators is shown in Table 1: From a numerical point of view, all the indicators of FMMGAN model in Table 1 are better than other models. Although images generated by the PRSR model are close to the real, it is very time-consuming. The average index value of FACEGAN model is medium and the time complexity is not high. The time complexity of DCGAN model is smallest, but the result is very poor. Table 1. Quantitative evaluations in terms of LFW dataset at four different models Model PSNR SSIM MS-SSIM TIME DCGAN 26.16 0.58 0.57 3 FACEGAN 31.03 0.88 0.87 5 PRSR 30.22 0.86 0.86 40 FMMGAN 33.25 0.92 0.91 3 For the cars dataset, we compare the masked image restoration performance with four models: the FMMGAN model, the DCGAN model, the FACEGAN model, and the PRSR model. The contrast effect is shown in Fig. 8 and Table 2: Table 2. Quantitative evaluations in terms of cars dataset at seven different models Model PSNR SSIM MS-SSIM TIME DCGAN 26.16 0.58 0.57 3 FACEGAN 30.22 0.87 0.87 5 PRSR 29.55 0.87 0.86 40 FMMGAN 32.36 0.90 0.90 3 Fig. 8Open in figure viewerPowerPoint Masked image restoration results (a) Masked image, (b) Restoration by DCGAN, (c) Restoration by FACEGAN, (d) Restoration by PRSR, (e) Our method, (f) Original image In conclusion, the FMMGAN model achieved better performance both visually and numerically. Our model is very fast. 5 Conclusion This paper studies the use of a novel fast GANs model for complex masked image restoration. First, the FMMGAN model is proposed. The model includes a neighbouring network, a generator network, a discriminator network, and two parsing networks. Second, we first use FMM to help the GANs model generating better restoration result with more neighbouring pixels. Our model is very fast. Third, unlike previous models, we first use the sqrt loss to calculate pixel loss and content loss and restore the masked image. We propose a new idea for measuring the stability and efficiency of the generation model, and mathematically verify the great advantage of FMMGAN model in increasing pixel loss and content loss. Experiments with other generation models demonstrate that our model achieves the best performance so far. Although our model is able to generate semantically plausible and visually pleasing images, it has some limitations. For the popular images, the masked image restoration result is best, but for some unknown images, the masked image restoration results are poor. It should be noted that we did not prove which contribution is greater in the pixel loss and the content loss for masked image restoration. These will be solved in the next study. 6 Acknowledgments This work is supported by National Natural Science Foundation of China (No. U1536121, 61370195) and BUPT Excellent Ph.D. Students Foundation (No. CX2018206). 7 References 1Li Y., Liu S., and Yang J. et al: ' Generative face completion'. arXiv preprint arXiv:1704.05838, 2017 2Goodfellow I., Pouget-Abadie J., and Mirza M. et al: ' Generative adversarial nets'. Advances in Neural Information Processing Systems, Montréal, Canada, 2014, pp. 2672– 2680 3Bertalmio M., Sapiro G., and Caselles V. et al: ' Image inpainting'. Proc. of the 27th annual Conf. on Computer Graphics and Interactive Techniques, New Orleans, USA, 2000, pp. 417– 424 4Bertalmio M., Vese L., and Sapiro G. et al: 'Simultaneous structure and texture image inpainting', IEEE Trans. Image Process., 2003, 12, (8), pp. 882– 889 5Barnes C., Shechtman E., and Finkelstein A. et al: 'Patchmatch: A randomized correspondence algorithm for structural image editing', ACM Trans. Graphics-TOG, 2009, 28, (3), p. 24 6Telea A.: 'An image inpainting technique based on the fast marching method', J. Graph. Tools, 2004, 9, (1), pp. 23– 34 7Ren J.S.J., Xu L., and Yan Q. et al: ' Shepard convolutional neural networks'. Advances in Neural Information Processing Systems, Montréal, Canada, 2014, pp. 901– 909 8Kingma D.P., and Welling M.: ' Auto-encoding variational Bayes'. arXiv preprint arXiv:1312.6114, 2013 9van den Oord A., Kalchbrenner N., and Espeholt L. et al: ' Conditional image generation with pixelcnn decoders'. Advances in Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 4790– 4798 10Dahl R., Norouzi M., and Shlens J.: ' Pixel recursive super resolution'. arXiv preprint arXiv:1702.00783, 2017 11Radford A., Metz L., and Chintala S.: ' Unsupervised representation learning with deep convolutional generative adversarial networks'. arXiv preprint arXiv:1511.06434, 2015 12Arjovsky M., Chintala S., and Bottou L.: ' Wasserstein gan'. arXiv preprint arXiv:1701.07875, 2017 13Gulrajani I., Ahmed F., and Arjovsky M. et al: ' Improved training of wasserstein gans'. arXiv preprint arXiv:1704.00028, 2017 14Mao X., Li Q., and Xie H. et al: ' Least squares generative adversarial networks'. 2017 IEEE Int. Conf. on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2813– 2821 15Simonyan K., and Zisserman A.: ' Very deep convolutional networks for large-scale image recognition'. arXiv preprint arXiv:1409.1556, 2014 16Kingma D., and Ba J.: ' Adam: A method for stochastic optimization'. arXiv preprint arXiv:1412.6980, 2014 17Hinton G., Srivastava N., and Swersky K.: ' Rmsprop: divide the gradient by a running average of its recent magnitude'. Neural Networks for Machine Learning, Coursera Lecture 6e, 2012 18Rasmus A., Berglund M., and Honkala M. et al: ' Semi-supervised learning with ladder networks'. Advances in Neural Information Processing Systems, Montréal, Canada, 2015, pp. 3546– 3554 19Ioffe S., and Szegedy C.: ' Batch normalization: accelerating deep network training by reducing internal covariate shift'. Int. Conf. on Machine Learning, Lille, France, 2015, pp. 448– 456 20Dosovitskiy A., Fischer P., and Springenberg J.T. et al: 'Discriminative unsupervised feature learning with exemplar convolutional neural networks', IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (9), pp. 1734– 1747 21Fei-Fei L.: ' Imagenet: crowdsourcing, benchmarking & other cool things'. CMU VASC Seminar, Pittsburgh, USA, 2010, vol. 16, pp. 18– 25 22Huang G.B., Mattar M., and Berg T. et al: ' Labeled faces in the wild: A database forstudying face recognition in unconstrained environments'. Workshop on Faces in 'Real-Life' Images: Detection, Alignment, and Recognition, Marseilles, France, 2008 23Krause J., Stark M., and Deng J. et al: ' 3d object representations for fine-grained categorization'. Proc. of the IEEE Int. Conf. on Computer Vision Workshops, Sydney, Australia, 2013, pp. 554– 561 24Wang Z., Bovik A., and Sheikh H. et al: ' Image quality assessment: from error visibility to structural similarity', 2004 Citing Literature Volume13, Issue7May 2019Pages 1124-1129 FiguresReferencesRelatedInformation