Using ROS and Gazebo to
Safely Validate and Verify
Autonomous Systems
Joshua P. Hecker
Staff Software Engineer
ROSCon 2019

LOCKHEED MARTIN ▪ Autonomous Systems

1

ROS-related Projects @ LMAS
• Multiple projects built on and/or extending
ROS as a core autonomy architecture:– Transport and mining truck fleets– Subsurface vehicles– Multiple aerial platforms• Significant investment in vehicle andsensor modeling in Gazebo:– Segway RMP 440 LE (off-road platform)– Caterpillar 777F (mining haul truck)– Neptec OPAL (3D lidar sensor)• ROS/Gazebo commonly requested bycustomers, agencies, proposal calls, etc.
LOCKHEED MARTIN ▪ Autonomous Systems

Lockheed Martin
Autonomous Systems(LMAS)
Littleton, CO, USA2

Autonomy Validation and Verification• Validating and verifying autonomy software is essential to ensuring
specification fulfillment, proper functionality, and safety criticality• Performance of autonomous robots is notoriously difficult to guarantee:– How can we test all reasonable real-world scenarios?
– Do tests “fail” or do they produce “new, unexpected behavior”?
– Learning adds additional layers of complexity, time, and possible outcomes• Pure simulation has drawbacks:– Computational requirements– Time vs. Quality vs. Cost
Verification– Reality gap
Are we building
the product right?

LOCKHEED MARTIN ▪ Autonomous Systems

Unit Tests
Integration
Tests
Automated
Tests

Regression
Tests

Customer
Acceptance
Tests

Validation

System Tests
Beta Tests

UsabilityTests

Are we building
the right product?

3

A Solution: Live/Virtual/Constructive Simulation• Run ROS autonomy software on live asset– Minimize reality gap• Represent live asset in Gazebo using virtual avatar– Minimize computational requirements (only model essentials)– Minimize development costs (only model essentials)• Map live asset and virtual avatar into unified constructive environment– Requires accurate tracking of live asset• Provides for rapid testing of real robots in challenging environments• Simulates realistic collisions with minimal danger to physical systems

LOCKHEED MARTIN ▪ Autonomous Systems

4

Example LVC Instantiation

LOCKHEED MARTIN ▪ Autonomous Systems

5

Example LVC Instantiation
Physical Asset

1. Swarmie robotic hardware platform(NASA/UNM/Swarming Technologies)

LOCKHEED MARTIN ▪ Autonomous Systems

6

Example LVC Instantiation
Physical Asset

2. Vicon Vantage system(24 V16 cameras)
LOCKHEED MARTIN ▪ Autonomous Systems

Motion Capture
7

Example LVC Instantiation
Physical Asset

Movement

3. Infrared cameras
detect passive markers

Motion Capture
LOCKHEED MARTIN ▪ Autonomous Systems

8

Example LVC Instantiation
Virtual Avatar

Physical Asset

4. Unactuated Gazebo model withrequired sensor simulators
Movement

Motion Capture
LOCKHEED MARTIN ▪ Autonomous Systems

9

Example LVC Instantiation
Virtual Avatar

Physical Asset

Movement

Position Data

5. Gazebo/ROS plugin maps
real robot pose to avatar

Motion Capture
LOCKHEED MARTIN ▪ Autonomous Systems

10

Example LVC Instantiation
Virtual Avatar

Physical Asset

6. Real robot
gets sensed
Gazebo data
Sensor Data

Movement

Position Data

Motion Capture
LOCKHEED MARTIN ▪ Autonomous Systems

11

Example LVC Instantiation
7. Real robot
trustssimulatedsensors andmakes
decisions as
if it were
insidesimulatedenvironment

Virtual Avatar

Physical Asset

Sensor Data

Movement

Position Data

Motion Capture
LOCKHEED MARTIN ▪ Autonomous Systems

12

Example LVC Instantiation
Virtual Avatar

Physical Asset

Sensor Data

Movement

Position Data

Motion CaptureLOCKHEED MARTIN ▪ Autonomous Systems

13

Implementation Details• ROS 2D Navigation Stack on-board Swarmie (i.e. move_base):– Odometry input produced by encoders and IMU– Lidar input produced by Velodyne VLP-16– SLAM (gmapping package) generates occupancy grid map (offline)– AMCL localizes against stored occupancy map (online)• Gazebo avatar model (unactuated)– VLP-16 sensor (velodyne_simulator package) replicates real lidar– Avatar set to static with gravity disabled– gazebo::physics::Entity::SetWorldPose() used to map real pose to sim

LOCKHEED MARTIN ▪ Autonomous Systems

14

Demonstration Video Screenshots

LOCKHEED MARTIN ▪ Autonomous Systems

15

Discussion/Future Work• Low-cost, low-risk, prototype demonstration of LVC architecture• Opening up new possibilities for autonomy validation and verification:
Limitations
Lidar fusion brute force; Depends on SLAM

Improvements
Intelligently fuse real and sim. point clouds

Only lidar implemented; Restricts learning, etc. Phys./Sim. camera fusion with realistic render

Vicon needs stable, indoor space; Small robots Investigate DGPS, others, for outdoor tracking
Demo tests only entry-level ROS functionality

Extend to production-level ROS codebases

Evaluations are hand-designed, one-off

Configure automated (Monte Carlo) testing

LOCKHEED MARTIN ▪ Autonomous Systems

16

Wrap-Up/Credits
Questions?

Thanks:Ryker Chute (Video Production/Editing)
LMAS (General Support)
LOCKHEED MARTIN ▪ Autonomous Systems

17