  For image inpainting, the convolutional neural networks (CNN) in previous
methods often adopt standard convolutional operator, which treats valid pixels
and holes indistinguishably. As a result, they are limited in handling
irregular holes and tend to produce color-discrepant and blurry inpainting
result. Partial convolution (PConv) copes with this issue by conducting masked
convolution and feature re-normalization conditioned only on valid pixels, but
the mask-updating is handcrafted and independent with image structural
information. In this paper, we present an edge-guided learnable bidirectional
attention map (Edge-LBAM) for improving image inpainting of irregular holes
with several distinct merits. Instead of using a hard 0-1 mask, a learnable
attention map module is introduced for learning feature re-normalization and
mask-updating in an end-to-end manner. Learnable reverse attention maps are
further proposed in the decoder for emphasizing on filling in unknown pixels
instead of reconstructing all pixels. Motivated by that the filling-in order is
crucial to inpainting results and largely depends on image structures in
exemplar-based methods, we further suggest a multi-scale edge completion
network to predict coherent edges. Our Edge-LBAM method contains dual
procedures,including structure-aware mask-updating guided by predict edges and
attention maps generated by masks for feature re-normalization.Extensive
experiments show that our Edge-LBAM is effective in generating coherent image
structures and preventing color discrepancy and blurriness, and performs
favorably against the state-of-the-art methods in terms of qualitative metrics
and visual quality.
