We introduce a general self-supervised approach to predict the future outputs of a short-range sensor (such as a proximity sensor) given the current outputs of a long-range sensor (such as a camera). We assume that the former is directly related to some piece of information to be perceived (such as the presence of an obstacle in a given position), whereas the latter is information rich but hard to interpret directly. We instantiate and implement the approach on a small mobile robot to detect obstacles at various distances using the video stream of the robot's forwardpointing camera, by training a convolutional neural network on automatically-acquired datasets. We quantitatively evaluate the quality of the predictions on unseen scenarios, qualitatively evaluate robustness to different operating conditions, and demonstrate usage as the sole input of an obstacle-avoidance controller. We additionally instantiate the approach on a different simulated scenario with complementary characteristics, to exemplify the generality of our contribution. Index Terms-Range sensing, computer vision for other robotic applications, deep learning in robotics and automation. Videos, datasets, and code to reproduce our results are available at: https://github.com/idsiarobotics/learning-long-range-perception/
Nava et al. REF introduce a self-supervised approach to predict future outputs of a short-range proximity sensor based on the current outputs of a long-range sensor, thus, training a Convolutional Neural Network (CNN) to accurately predict obstacles in the camera images.
Learning Long-Range Perception Using Self-Supervision From Short-Range Sensors and Odometry
IEEE Robotics and Automation Letters
IEEE Robotics and Automation Letters
Computer Science Engineering