  Learning to capture dependencies between spatial positions is essential to
many visual tasks, especially the dense labeling problems like scene parsing.
Existing methods can effectively capture long-range dependencies with
self-attention mechanism while short ones by local convolution. However, there
is still much gap between long-range and short-range dependencies, which
largely reduces the models' flexibility in application to diverse spatial
scales and relationships in complicated natural scene images. To fill such a
gap, we develop a Middle-Range (MR) branch to capture middle-range dependencies
by restricting self-attention into local patches. Also, we observe that the
spatial regions which have large correlations with others can be emphasized to
exploit long-range dependencies more accurately, and thus propose a Reweighed
Long-Range (RLR) branch. Based on the proposed MR and RLR branches, we build an
Omni-Range Dependencies Network (ORDNet) which can effectively capture short-,
middle- and long-range dependencies. Our ORDNet is able to extract more
comprehensive context information and well adapt to complex spatial variance in
scene images. Extensive experiments show that our proposed ORDNet outperforms
previous state-of-the-art methods on three scene parsing benchmarks including
PASCAL Context, COCO Stuff and ADE20K, demonstrating the superiority of
capturing omni-range dependencies in deep models for scene parsing task.
