  We propose a novel deep learning method for shadow removal. Inspired by
physical models of shadow formation, we use a linear illumination
transformation to model the shadow effects in the image that allows the shadow
image to be expressed as a combination of the shadow-free image, the shadow
parameters, and a matte layer. We use two deep networks, namely SP-Net and
M-Net, to predict the shadow parameters and the shadow matte respectively. This
system allows us to remove the shadow effects from images. We then employ an
inpainting network, I-Net, to further refine the results. We train and test our
framework on the most challenging shadow removal dataset (ISTD). Our method
improves the state-of-the-art in terms of root mean square error (RMSE) for the
shadow area by 20\%. Furthermore, this decomposition allows us to formulate a
patch-based weakly-supervised shadow removal method. This model can be trained
without any shadow-free images (that are cumbersome to acquire) and achieves
competitive shadow removal results compared to state-of-the-art methods that
are trained with fully paired shadow and shadow-free images. Last, we introduce
SBU-Timelapse, a video shadow removal dataset for evaluating shadow removal
methods.
