###### Abstract
[23] utilized the pixel-level information, local region-level information and global-level information to remove the shadow. [PERSON] et al. [25] extracted the grey level, color composition, and gradient information of pixels in a video frame to construct a respective shadow mask corresponding to the same video frame, and then they discriminated shadow pixels by minimizing the likelihood of missing a shadow pixel in each of the shadow masks. [PERSON] et al. [26] jointly used the color, texture and gradient by exploiting local neighboring information and designed an adaptive mechanism to estimate threshold parameters for detecting shadows. [PERSON] et al. [27] integrated the chromatic and gradient information with the image hypergraph segmentation and used a stochastic majority voting scheme to shadow regions. Without loss of generality, after extracting features, the above methods detect shadows according to the parameter assumptions and thresholds tuning in the classification stage. Apparently, it is difficult to acquire the appropriate thresholds of parameters for various environments such as indoors and outdoors. Recently, learning-based methods have been popular in shadow detection [28, 29, 30, 31, 32, 33, 34], which apply a classifier constructed by various shadow features to discriminate shadows from the foreground. For example, [PERSON] et al. [30] extracted a set of features derived from characteristic differences in color and edges, then employed support vector machines (SVM) and the co-training algorithm for classification. [PERSON] et al. [31] extracted various features based on illumination, color and texture, then employed partial least squares (PLS) and logistic discrimination (LD) to classify shadows from their moving objects. [PERSON] et al. [33] introduced the binary patterns of local colour constancy (BPLCC), light-based gradient matching (LGM) and intensity-reduction histogram (IRH) to construct two over-complete dictionaries from image patches, then performed the sparse representation classifier to discriminate shadows and objects. [PERSON] [34] designed a multi-layer pooling scheme (MLPS) to integrate the features in a local region and to reduce the dimension of extracting features, then used a random forest algorithm as the ensemble decision scheme. Although the above methods require the label information from a part of ground truth to train the classifier model, they can adapt to various environments very well without turning thresholds or parameters. Instead of handcrafted feature extraction, machine learning techniques have been developed to automatically learn features using deep neural networks [35; 36; 37; 38; 39; 40; 41; 42]. First, researchers mainly treated the convolutional neural network (CNN) as a powerful feature extractor and made significant performance improvement with the powerful deep features. For example, [PERSON] et al. [35] first extracted shadow edges via a structured CNN, and then solved shadow recovery as an optimization problem. [PERSON] et al. [36] first applied the CNN to shadow detection. They utilized a 7-layer CNN to extract features from superpixels and then feed the features to a conditional random field (CRF) model to smooth the detection results. Then, end-to-end CNN models were proposed due to the emergence of fully convolutional networks (FCN) [37]. For example, [PERSON] et al. [40] presented a semantic-aware stacked CNN model to extract the semantic shadow prior and then refined the output by a patch-based CNN. [PERSON] et al. [41] formulated the direction-aware attention mechanism in a spatial recurrent neural network (RNN) and recovered direction-aware spatial context (DSC) for detecting shadows. The CNN-based methods can learn features from the image, while they need a sufficient data for training and require to turn some parameters, such as the number of convolutional layers, the detection window size of each convolution layer, learning rate and the number of iterations. However, most of the methods mentioned were presented to detect shadow in single image.