  Bridging distant context interactions is important for high quality image
completion with large masks. Previous methods attempting this via deep or large
receptive field (RF) convolutions cannot escape from the dominance of nearby
interactions, which may be inferior. In this paper, we propose treating image
completion as a directionless sequence-to-sequence prediction task, and deploy
a transformer to directly capture long-range dependence in the encoder in a
first phase. Crucially, we employ a restrictive CNN with small and
non-overlapping RF for token representation, which allows the transformer to
explicitly model the long-range context relations with equal importance in all
layers, without implicitly confounding neighboring tokens when larger RFs are
used. In a second phase, to improve appearance consistency between visible and
generated regions, a novel attention-aware layer (AAL) is introduced to better
exploit distantly related features and also avoid the insular effect of
standard attention. Overall, extensive experiments demonstrate superior
performance compared to state-of-the-art methods on several datasets.
