FlightGoggles: Photorealistic Sensor Simulation for Perceptiondriven Robotics using Photogrammetry and Virtual Reality FlightGoggles is a photorealistic sensor simulator for perceptiondriven robotic vehicles. The key contributions of FlightGoggles are twofold. First, FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry. Second, it provides the ability to combine (i) synthetic exteroceptive measurements generated in silico in real time and (ii) vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in flight in a motioncapture facility. FlightGoggles is capable of simulating a virtualreality environment around autonomous vehicle(s) in flight. While a vehicle is in flight in the FlightGoggles virtual reality environment, exteroceptive sensors are rendered synthetically in real time while all complex dynamics are generated organically through natural interactions of the vehicle. The FlightGoggles framework allows for researchers to accelerate development by circumventing the need to estimate complex and hardtomodel interactions such as aerodynamics, motor mechanics, battery electrochemistry, and behavior of other agents. The ability to perform vehicleintheloop experiments with photorealistic exteroceptive sensor simulation facilitates novel research directions involving, e.g., fast and agile autonomous flight in obstaclerich environments, safe human interaction, and flexible sensor selection. FlightGoggles has been utilized as the main test for selecting nine teams that will advance in the AlphaPilot autonomous drone racing challenge. We survey approaches and results from the top AlphaPilot teams, which may be of independent interest. FlightGoggles is distributed as opensource software along with the photorealistic graphics assets for several simulation environments, under the MIT license at http:flightgoggles.mit.edu.

Flexible Trinocular: Nonrigid MultiCameraIMU Dense Reconstruction for UAV Navigation and Mapping In this paper, we propose a visualinertial framework able to efficiently estimate the camera poses of a nonrigid trinocular baseline for longrange depth estimation onboard a fast moving aerial platform. The estimation of the timevarying baseline is based on relative inertial measurements, a photometric relative pose optimizer, and a probabilistic wing model fused in an efficient Extended Kalman Filter (EKF) formulation. The estimated depth measurements can be integrated into a georeferenced global map to render a reconstruction of the environment useful for local replanning algorithms. Based on extensive realworld experiments we describe the challenges and solutions for obtaining the probabilistic wing model, reliable relative inertial measurements, and visionbased relative pose updates and demonstrate the computational efficiency and robustness of the overall system under challenging conditions.