  Geo-localization is a critical task in computer vision. In this work, we cast
the geo-localization as a 2D image retrieval task. Current state-of-the-art
methods for 2D geo-localization are not robust to locate a scene with drastic
scale variations because they only exploit features from one semantic level for
image representations. To address this limitation, we introduce a hierarchical
attention fusion network using multi-scale features for geo-localization. We
extract the hierarchical feature maps from a convolutional neural network (CNN)
and organically fuse the extracted features for image representations. Our
training is self-supervised using adaptive weights to control the attention of
feature emphasis from each hierarchical level. Evaluation results on the image
retrieval and the large-scale geo-localization benchmarks indicate that our
method outperforms the existing state-of-the-art methods. Code is available
here: \url{https://github.com/YanLiqi/HAF}.
