  The drone navigation requires the comprehensive understanding of both visual
and geometric information in the 3D world. In this paper, we present a
Visual-Geometric Fusion Network(VGF-Net), a deep network for the fusion
analysis of visual/geometric data and the construction of 2.5D height maps for
simultaneous drone navigation in novel environments. Given an initial rough
height map and a sequence of RGB images, our VGF-Net extracts the visual
information of the scene, along with a sparse set of 3D keypoints that capture
the geometric relationship between objects in the scene. Driven by the data,
VGF-Net adaptively fuses visual and geometric information, forming a unified
Visual-Geometric Representation. This representation is fed to a new
Directional Attention Model(DAM), which helps enhance the visual-geometric
object relationship and propagates the informative data to dynamically refine
the height map and the corresponding keypoints. An entire end-to-end
information fusion and mapping system is formed, demonstrating remarkable
robustness and high accuracy on the autonomous drone navigation across complex
indoor and large-scale outdoor scenes. The dataset can be found in
http://vcc.szu.edu.cn/research/2021/VGFNet.
