# Throwaway Shadows Using Parallel Encoders Generative Adversarial Network
###### Abstract  
Face photographs taken on a bright sunny day or in floodlight contain unnecessary shadows of objects on the face. Most previous works deal with removing shadow from scene images and struggle with doing so for facial images. Faces have a complex semantic structure, due to which shadow removal is challenging. The aim of this research is to remove the shadow of an object in facial images. We propose a novel generative adversarial network (GAN) based image-to-image translation approach for shadow removal in face images. The first stage of our model automatically produces a binary segmentation mask for the shadow region. Then, the second stage, which is a GAN-based network, removes the object shadow and synthesizes the effected region. The generator network of our GAN has two parallel encoders--one is standard convolution path and the other is a partial convolution. We find that this combination in the generator results not only in learning an incorporated semantic structure but also in disentangling visual discrepancies problems under the shadow area. In addition to GAN loss, we exploit low level L1, structural level SSIM and perceptual loss from a pre-trained loss network for better texture and perceptual quality, respectively. Since there is no paired dataset for the shadow removal problem, we created a synthetic shadow dataset for training our network in a supervised manner. The proposed approach effectively removes shadows from real and synthetic test samples, while retaining complex facial semantics. Experimental evaluations consistently show the advantages of the proposed method over several representative state-of-the-art approaches. shadow removal; image restoration; image reconstruction; partial convolution 202202010 January 202202010 January 2022010  
traditional algorithms for removing shadow from scene images, they are unable to remove shadows from facial images due to the complex nature of face semantics. In this work, instead of improving or modifying the previous deep learning based shadow removal model for removing shadow from face images, we took a totally different approach by using an image inpainting approach as a shadow removal model. Image inpainting is the method used to reconstruct lost or damaged parts of an image. The current state-of-the-art deep learning based image inpainting methods [7, 8, 9, 10, 11] reconstruct the damaged region by providing the mask of the damaged part. Refs. [7, 8] fill the missing pixel by copying similar patches from the surrounding region. Ref. [10] used some guidance information to reconstruct the corrupt part of the image while [12] use two discriminators to enforce global coherency. Some models [9, 11, 13] use two-stage networks by generating coarser results in the first stage and refine it in the second stage. All of the above mentioned deep learning works only use standard convolution as the backbone operation of a neural network. The standard convolution employs the same filter weights all over the image, nevertheless pixels are valid or affected. Eventually, it generates a well-incorporated structure under the affected area but fails to remove visual artifacts, particularly at the boundary of the affected and valid area as mentioned in [14, 15]. In the shadow removal problem, this issue becomes more severe because most of the time the shadow area is large and an irregular shape. To overcome these issues, many researchers use extensive post processing steps and/or additional refinement stages as in [8, 11]. To incorporate with irregular shape recovery and limitations of standard convolution, improved convolution--called partial convolution [14]--is proposed. In partial convolution, convolution is only employed on a valid pixel area and is re-normalized. A segmentation mask is used to locate a valid pixel area [14]. The valid pixel mask is updated after each iteration to compute new valid pixels. Additionally, our approach does not have any post processing or refinement stages.