Up, Up, and Away: Adventures

in Aerial Robotics

Introduction
Kimberly McGuire
Robotics EngineerRamón Roche
General Manager

Some words from Kim“I’m very sorry I couldn’t come toROScon! I was looking forward toit all year but my health got in theway… I hope you enjoy this talkprepared by us and I really hope
to see you at next year’s ROScon
or at one of the Aerial-ROSmeetings!”

Kimberly McGuire
Robotics Engineer, BitcrazeWhere do Drones
come from?
Everything went by very quicklyIn the last 20years
We went from this10 years agoSelfie?

TodayTodayTodayToday$33.7bCurrent Drone Market size in 2023

$54.6b
Drone Market size by 2030

But the journey to success for manyhas been one of multiple challenges

We started working together to fix
the frustrations of our own journey’s
into Aerial Robotics

A cautionary tale
The journey of a robotics
developer in the aerial space

Introducing
Mary
Mary finished her MSc in Robotics
She knows about ROS, simulation, perception,and has build some ground robots herself.

The quest
She just got hired at a company that wants tobuild their own fully autonomous UAV for indoornavigation.
Mary hasn’t worked with UAVs before but how
hard can it be?

??

Copters

Decisions, decisions
Unmanned aerial vehicle○○○

Needs to be off the ground
It needs to be controllable
No human should be on board

Propeller
Wings

It usually consist of○○○MotorsPropellers
Wings/Flaps/balloons

Fixed-wing

Flaps
Air

Flapping wing

Many types available○

What type of platform do we need?
BlimpsMary thinks she needs a Multicopter
Main advantage: Can hold position!
Allows for precise control
Very common platform
Many examples available

Multicopter options

Let’s go with Quadcopters
Mary picks a quadcopter, as that is a very
common vehicle for this type of task.

But what is necessary to fly aquadcopter?

Identifying the required components

A few hours of google later…

Pilot on thegroundOn-boarda DronePixhawk 6X

Flight Management Unit (FMU)
Crazyflie Bolt

VOXL 2

Turns out Mary needsa Flight Controller
A flight controller or also flight management unit (FMU)
is crucial
It handles the control, navigation andtelemetry/communication necessary to fly!
Brains of the operation

Flight Management Unit (FMU)
Typical components of an FMU (minimum)●●●●

Microcontroller STM32 or similar
IMUBarometer
Magnetometer

The central unit of a drone
●
●
●
●
●

Usually realtime running an RTOS
Controls the I/O
Runs navigation
Handles telemetry/communication
And more…

How to choose the right one? Mary isn’t sure

Mary made a priority list for her project
1.
2.
3.
4.
5.

$$$ Price
Capabilities
Allow Custom firmware
Documentation
Well supported by Manufacturer

Mary found a manufacturer
that had what she was looking
for plus shipped to her country
at reasonable prices.

Mary needs to decide which firmware to run
She starts to understand the basics of aquadcopter and the necessary components
But compared to the ground robots, quadcopters
are unstable!
Does she need to write the controllers and drivers
from scratch?

Imposter syndrome kicks in…

Back to google…

So many options

And more…

How to choose?
Mary made another helpful list to help herdecide. The firmware needed to have:…The list narrowed down…

1. Support my Hardwarea.

After the huge expense

2. Friendly Licensea.

Mary needs to get to market

3. Maintainer Activitya.

Active Open Source Community

4. ROS Supporta.

Mary has a Masters she really wants to beable to use in this project??

Mary saw a youtube
video and chose PX4

Short Commercial

Mary is able to fly manually withRC control
Not super easy but it’s fun!
But… it’s not really autonomous
She needs telemetryPilot on thegroundOn-board aDrone

Communication
Telemetry: the real-time transmission of data collected onboard a drone about its flight dynamics, status, and operating
environment.
Control: Either RC attitude control, or high level commands like
waypoint navigation
Messaging protocol options:
MAVlink (ardupilot, PX4, ROSflight)
Pprzlink (Paparazzi UAV)
Multiwii Serial Protocol (Betaflight)
Crazy Real-Time Protocol (Crazyflie)Back to google…

Autonomy● Mary also implements on her quadcopter:
○
○
○

Lidar
RGB Camera
Flow sensor and height sensor○○○

Write something completely from scratch?
Try to fit existing ROS packages to aerial vehicles
Look for Aerial Autonomy stacks?

● She is experienced with several autonomy
packages in ROS● But she can’t use those out-of-the-box aseasily…
● Does she:

ROS Autonomy stacks for aerial vehicles
ROS packages examples
○○○
○
○
○

Aerostack(2)Aerial Core
Kumar Robotics
Crazychoir
Crazyswarm(2)And many more..

Don’t missthe talk
today!

Adapted from M. Fernandez-Cortizas, M. Molina, P. Arias-Perez, R. Perez-Segui, D. Perez-Saura, and P. Campoy, 2023, “Aerostack2: A software framework fordeveloping multi-robot aerial systems” , ArXiv DOI 2303.18237.
https://github.com/ROS-Aerial/aerial_robotic_landscape/blob/main/aerial_autonomy_stacks.md

How does ROS talk to the FMU?

ROS and FMU communication
Flight management Unit (FMU) to ROS Running on Laptop

Intercommunication

FMU

Intercommunication options:○○○

FMU <-> MAVlink <-> MAVros <-> ROS packages
FMU <-> DDS / Zenoh <-> ROS packages…….

LaptopBut running ROS in a laptop is not the end
goal we need an additional companion
platform on board the vehicle

Intel NUC

Raspberry pi 4

Companion Computers
Nvidia Jetson Xavier

Mary breaks down levels of control
FMU

High level goals
CompanionFast
Attitude control

SlowPosition control
Trajectory control

As many of us Mary had a RPi4 in her deskcollecting dust and she’s going to try thatfor now

Let’s not forget the assignment Mary!
Mary implements an additional:○ Companion computer (Raspberry Pi 4)

So now… finally… Mary is able to use someof her past expertise in robotics!

Safety andsimulation● Mary has spent close to $5k USD in herproject● The drone is a bit heavy now… 2 kg..
● Mary finds it a bit scary to fly, specially whencrashing is a real possibility.
● Mary decides to start using a simulator first!
● It needs to be able to simulate the drone in amore or less realistic way

Back to google…

Aerial Robotics Simulation
Simulators
○
○
○
○
○

Gazebo (new and classic)
WebotsMicrosoft AirSim
Nvidia Isaac Sim
And more…

Choices??
○○○

Dynamics versus vision rendering?
Hardware or software in the loop?
Collision models?

Mary saw another youtube
video and chose Gazebo

Now the real fun begins (?)

Now Mary is ready to try out things!
She’ll use the simulator to initial development
And implements it to the real drone, tries it out in the lab, and iteratively
develops the autonomous drone.
But…

Now the real fun begins (?)

Now Mary is ready to try out things!
She’ll use the simulator to initial development
And implements it to the real drone, tries it out in the lab,
and iteratively develops the autonomous drone.
But…

Mary ran out of time
The project was supposed to befinished in 3 months… she only has aweek left now to finish her
assignment…

What went wrong for Mary, and many others?
●●●●●The learning curve is still too steep
You almost need to be an expert in everything
Many need to custom build their own drone
Too many choices in software and hardware
Mary’s story is very common for many starting in aerial
robotics, both in academia and industry!

NOT MARY’S FAULT!

Mary could be youA common frustration● We have personally been through thisexperience● Almost everyone we meet has gone
through a similar experience● If we are going to complain weshould at least do something to help
● No group for aerial vehicles?
● Let’s start one!
● First meeting @ ROSCon 2022

Let’s document everything
and make a landscape
Our goals

The Journey so far
6 x Scientific meetings
○
○
○
○
○
○

Aerostack2
Drone2bets
Vehicle Gateway
Duckietown’s Duckiedrone
Education overview
NXP Semiconductors

6 x discussion meetings
○
○
○
○
○
○

Hardware, components and dev kits
Message standards
Swarm management
Tutorial and education
Aerial autonomy stacks
Simulation

Aerial Robotic LandscapeIt’s not yet complete
WE NEED YOUR HELPConclusions of our meetings so far
Education
Simulation
Standardized drone
Autonomy packages

Conclusions about Simulation
Feedback: Many use Gazebo but not yet thenew one
Many are confused about choices fordynamics models, vision engines and thedifferences between them
There are just so many simulators out there!
Also, don’t miss the talk on ‘Multi-drone
simulation with deep q-learning’ 3:30 PMtoday

Conclusions about Education
A lot of education available for aerial robotics
Only a handful that also cover ROS, even less
ROS 2
The “Duckiedrone” by duckietown
Feedback: better docs, more tutorials

Conclusions about Standardized Drone
We are missing a Turtlebot equivalentThe ‘Turtledrone’Conclusion Autonomy packages
Aerostack2 will present their work today as well at 10
AM today!
Many Autonomy packages that exist
But unclear for many which to use?
And should there be an specific aerial autonomy
package?
What is missing from those for ground robots?
Feedback: Better visibility, and end-to-end integration

Help us map the aerial robotics landscape
Aerial Robotics community working group meetings every 2 weeks
Soon we will be starting with specific development goals!
Let's make it easier for all the Mary’s to get started in Aerial Robotics!
Feedback: Your participation!

Thanks for your time!

Aerial Robotics Work Group

Kimberly McGuire - @bitcraze_se (X prev Twitter)kimberly@bitcraze.io
Ramón Roche - @mrpollo (X prev Twitter)rroche@linuxfoundation.com

www.github.com/ros-aerial