Learning Visual Localization of a Quadrotor Using Its Noise as Self-Supervision
Journal: IEEE Robotics and Automation Letters
Year: 2022

We introduce an approach to train neural network models for visual object localization using a small training set, labeled with ground truth object positions and a large unlabeled one. We assume that the object to be localized emits sound, which is perceived by a microphone rigidly affixed to the camera. This information is used as the target of a cross-modal pretext task: predicting sound features from camera frames. By solving the pretext task, the model draws self-supervision from visual and