  We present Region Similarity Representation Learning (ReSim), a new approach
to self-supervised representation learning for localization-based tasks such as
object detection and segmentation. While existing work has largely focused on
solely learning global representations for an entire image, ReSim learns both
regional representations for localization as well as semantic image-level
representations. ReSim operates by sliding a fixed-sized window across the
overlapping area between two views (e.g., image crops), aligning these areas
with their corresponding convolutional feature map regions, and then maximizing
the feature similarity across views. As a result, ReSim learns spatially and
semantically consistent feature representation throughout the convolutional
feature maps of a neural network. A shift or scale of an image region, e.g., a
shift or scale of an object, has a corresponding change in the feature maps;
this allows downstream tasks to leverage these representations for
localization. Through object detection, instance segmentation, and dense pose
estimation experiments, we illustrate how ReSim learns representations which
significantly improve the localization and classification performance compared
to a competitive MoCo-v2 baseline: $+2.7$ AP$^{\text{bb}}_{75}$ VOC, $+1.1$
AP$^{\text{bb}}_{75}$ COCO, and $+1.9$ AP$^{\text{mk}}$ Cityscapes. Code and
pre-trained models are released at: \url{https://github.com/Tete-Xiao/ReSim}
