  Inpainting arbitrary missing regions is challenging because learning valid
features for various masked regions is nontrivial. Though U-shaped
encoder-decoder frameworks have been witnessed to be successful, most of them
share a common drawback of mask unawareness in feature extraction because all
convolution windows (or regions), including those with various shapes of
missing pixels, are treated equally and filtered with fixed learned kernels. To
this end, we propose our novel mask-aware inpainting solution. Firstly, a
Mask-Aware Dynamic Filtering (MADF) module is designed to effectively learn
multi-scale features for missing regions in the encoding phase. Specifically,
filters for each convolution window are generated from features of the
corresponding region of the mask. The second fold of mask awareness is achieved
by adopting Point-wise Normalization (PN) in our decoding phase, considering
that statistical natures of features at masked points differentiate from those
of unmasked points. The proposed PN can tackle this issue by dynamically
assigning point-wise scaling factor and bias. Lastly, our model is designed to
be an end-to-end cascaded refinement one. Supervision information such as
reconstruction loss, perceptual loss and total variation loss is incrementally
leveraged to boost the inpainting results from coarse to fine. Effectiveness of
the proposed framework is validated both quantitatively and qualitatively via
extensive experiments on three public datasets including Places2, CelebA and
Paris StreetView.
