A True Zero-Copy RMW Implementation for ROS2

ROSCon 2019
Karsten Knese (BOSCH LLC)
karsten.knese@us.bosch.com
Michael Pöhnl (BOSCH GmbH)michael.poehnl@de.bosch.com

A True Zero-Copy RMW Implementation for ROS2
The Use CaseSensors

CR/RTC-HMI4 | 2019-09-21

Algorithms

Actuators

automated driving is a data processing chain with a sensor input of up to 10 GB/s

© 2019 Robert Bosch LLC and affiliates. All rights reserved.

A True Zero-Copy RMW Implementation for ROS2
The Problem
Node 1
Middleware

write(de)serializex copiescopy

copy

read

copy

read

A typical middleware…
 copies when passing messages from the publisher to the middleware
 copies when passing messages from the middleware to the subscriber does internally even more copies and/or serialization/deserialization
 does at least n+1 copies for an inter-process-communication with n subscribers

3

CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

Node 2
Node 3

A True Zero-Copy RMW Implementation for ROS2
The Problem
Node 1
Middleware

write(de)serializex copies

copy

copy

read

copy

readNode 2
Node 3A typical middleware…
 copies when passing messages from the publisher to the middleware
 copies when passing messages from the middleware to the subscriber does internally even more copies and/or serialization/deserialization
 does at least n+1 copies for an inter-process-communication with n subscribers

No time to copy and serialize n GB/s while driving!
4CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

A True Zero-Copy RMW Implementation For ROS2
The Solution: True Zero-Copy Inter-Process-Communication
Node 1

Node 2Node 3

Node 4write

readreadread

Memory Pool

SharedMemory

True zero-copy means…
 it is an end-to-end zero-copy approach from publishers to subscribers, based on shared memory
 the publisher directly writes to a chunk of memory provided by the middleware
 the middleware passes message references to subscribers and manages their livelinesszero-copy communication is a must-have for automated driving!
5CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

A True Zero-Copy RMW Implementation for ROS2
The Solution: Eclipse iceoryx™
Eclipse iceoryx
 Shared memory inter-process-communication with zero-copy support
 Written in modern C++ with support for Linux and QNX
 Just launched as Eclipse incubation project with Apache 2.0 license

rmw_iceoryx – the iceoryx RMW implementation for ROS2
 First version available that supports publish/subscribe, the ROS2 CLI and a bridge

 Zero copy support for fixed size messages, slim serialization for dynamic messages

6

CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

A True Zero-Copy RMW Implementation for ROS2
The Changes to ROS2 – Loaning Messages

publisher

subscription#2
publish_loaned_message()#3
take_loaned_message()#4#1borrow_loaned_message()

CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

rmw_iceoryx_cpp

return_loaned_message()

A True Zero-Copy RMW Implementation for ROS2
The Constraints for Zero-Copy
dynamic_size_message.msg

fixed_size_message.msg

int32 one_int
float64 one_floatchar[] char_array

int32 one_int
float64 one_floatchar[100] char_array[4 byte | 8 byte | 24 byte]
dynamic size

[4 byte | 8 byte | 100 byte]
fixed sized (POD)(heap allocation)
The topic is not allowed to use heap-based data structures (e.g. STL containers with default allocators)
CR/RTC-HMI4 | 2019-09-21© 2019 Robert Bosch LLC and affiliates. All rights reserved.

enough said …

getting started
https://github.com/eclipse/iceoryx
https://github.com/ros2/rmw_iceoryx
https://github.com/karsten1987/fixed_size_ros2_demo