Simulate robots likenever before withOpen 3D Engine
Adam Dąbrowski (Robotec.ai)

The talkPart 1: Finding your way in robotic simulators zoo.
• Why are game engines included?
• What does my use-case match?
• Demo: O3DE, scale, beauty and ROS.

Part 2: Building simulations with O3DE.
• Getting ahead with project templates.
• Resources at hand: robots, assets, demos.
• What is unique about O3DE integration with ROS?
• Demo: 3 minutes to simulate your robot!

Part 3: Features, future and you.
• Let's look at sensors, components and tooling for robotics.
• Where do I start and how can I contribute?

Finding your way in robotic simulators zoo.

Complete-package, general-purpose simulators. I excluded:• Dead / stale projects.
• Domain-specific sims such as Automotive (AWSIM, CARLA).
• Physics engines alone (though they matter).

Comparing
simulators

Fair comparison would be highly multidimensional and exhaust thetalk limit.
Simulator, use-case, user skill, platform, version, plugins, (..)
Instead, let us look at some key insights.

Simulators,engines

Game & simulation engines
Why game engines?
• Large interactive worlds -> content creation, high performance sensor / physics simulation• Telling good visual stories -> photorealism, generating synthetic data.
• Automotive: no great fit among robotic simulators -> driving automotive simulation towards game engines.

These aspects are becoming more important in other branches ofrobotics, because:Game &simulationengines• Multi-robot use / complexity of environments are growing.
• Fundamental problems are still being solved but focus on
applications and productivity is increasing.
Game engines can excel at simulation of integrated robotic usecases, especially where the focus is on perception, behaviors,scenarios and productivity.

Open 3D
EngineGreat fit for ROS simulation:• Open source and free.
• ROS 2 native:• C++ & CMake.
• No bridges, write ROS code in O3DE!
• Not just a connector: sensors, control, tooling.

• High modularity.
• Atom: a modern, photorealistic renderer.

My need is..
.. to simulate a task which heavily relies on accurate simulation ofphysics of { grasping of soft bodies, legged joint systems,hydrodynamics of submersible, deformable ground, …}

My need is..
.. to simulate a task which heavily relies on accurate simulation ofphysics of { grasping of soft bodies, legged joint systems,hydrodynamics of submersible, deformable ground, …}

My need is..
.. quite simple, I want a wheeled robot in a relatively small indoor area
for a quick research project.

My need is..
.. quite simple, I want a wheeled robot in a relatively small indoor area
for a quick research project.

Most will do, consider:My need is..
.. to produce large amounts of high-quality synthetic sensor datalabelled with ground-truth to train my neural networks for {detection, segmentation, tracking, ...}

My need is..
.. to produce large amounts of high-quality synthetic sensor datalabelled with ground-truth to train my neural networks for {detection, segmentation, tracking, ...}

Game engines and NVIDIA Isaac Sim might be the best fit:My need is..
.. industrial simulation with ROS robotic arms and AMRs. Scene is quite large, and I

want to see robot interactions in presence of humans and each other to achieve
productivity goals. All simulation has to be physical and integrated with real systemsuch as robot deliberation packages, providing them with realistic sensor data. game engines that are well-integrated with ROS will typically do a better job at this.

My need is..
.. industrial simulation with ROS robotic arms and AMRs. Scene is quite large, and I

want to see robot interactions in presence of humans and each other to achieve
productivity goals. All simulation has to be physical and integrated with real systemsuch as robot deliberation packages, providing them with realistic sensor data. game engines that are well-integrated with ROS will typically do a better job at this.

O3DE might be the best fit:How to build with O3DE?

Simple robotics project: Ros2ProjectTemplate• Hello world with O3DE and nav2 stack.
• Husarion ROSBot XL in a Warehouse.
• Simple assets with smaller footprint.

ProjectTemplates

Fleet of indoor robots: Ros2RobotFleetTemplate• Start with a fleet of AMRs in a warehouse, guided by nav2.
• Learn how to build and customize environment.
• Amazon Proteus robot model.

ProjectTemplates

Robotic arms: Ros2ManipulationTemplate• Panda Franka and UR10 arms.
• First scene is meant for vision and grasping R&D.
• Second scene includes conveyor belt and palletizing.

ProjectTemplatesCustom messages? Build & source workspace, add one line:
target_depends_on_ros2_package(${MyTarget} my_msgs REQUIRED)

Designed with ROS 2 developers in mind, reflects concepts such as
namespaces, frames and transforms.

Build withO3DE
You can import your robot from URDF or SDFormat.

Resources

Features, the future and youSummary of Features
O3DE supports all ROS interfaces:
topics, services, actions.

Joints & grippers

Importer&utilities

Dynamics & steeringSensors

Learn through:• Documentation, starting with setup and build, including
API reference for ROS 2 Gem.
• Short tutorials for each template (with nav2 / MoveIt2)• Running one of three demos.

Learning andcontributing

Joining the community:• Contribute to the engine or the ROS 2 Gem project.
• Build and share Gems with your sensors and robots.
• Develop a new use-case or demo and tell us about it!
• Say hello on O3DE Discord.
Visit Robotec.ai booth to ask me anything, import yourrobot or run the demo!

Questions?
Thank you!
adam.dabrowski@robotec.ai

dabrowskiadam
adamdbrw