aibo with ROS
Tomoya Fujita
R&D CenterSony Corporation

R&D Center System Technology Development Division Base System Development Department
Copyright 2018 Sony Corporation

What is “aibo”?

aiboERS-1000

Coming up in US
Ver y soon

Comes close to you&Makes interaction

Intelligent
Processing（Cloud）

Edge Computing System

Intelligent Processing（Edge）

Situational
Awareness

Behavior
Planning

Recognition

Behavior
Control

Mechatronics
OUT

HardwareIN

Sensing

User/Environment

Feeling Area
aibo can feel someone is there.

!

Feeling Area：5ｍ
Attraction Area：3m
Communication Area：1.5m

Attraction Area
aibo tries to get attentionaibo

Communication Area
aibo communicates with user/owner❤

OLED x2

Speakers

Camera x2

Head x3

ToF Sensor

Mouth x1

Touch Sensor x3

Application
Processor (SOC)

Microphone x4

MotionProcessor

PSD Sensors x2
Motion Detector

Neck x1

Qualcomm®
Snapdragon™ 820(APQ8096)

Sensing Devices

Waist x1
Front Legs x6
Hind Legs x6

Light Detector
IMU (head)

Ears x2

RAM 4GBROM 32GB

Tail x2
LTE/Wi-Fi

IMU (body)

Actuators

Sensor

Location

PerposeImage (fisheye camera)

Front Face

Face / Body / Hand / Object /Color / QR code

Image (fisheye camera)

Back

SLAM(SimultaneousLocalization and Mapping)ToF (Time of Flight)
PSD (Position SensitiveDetector)

Front Face

Object Detection

ChestCliff Detection

Touch Pressure

BackPatting / Slapping Detection

Touch Capacitive

Head, Chin

Patting Detection

6 axis (gyro x accelerometer)

Head

Lift / Cliff Detection

6 axis (gyro x accelerometer)

Body

Human Detect
Ambient LightGround

Body
Body
Paw

Lift Detection
Walk Balance Calibration
Human Detection
Environment Prediction
9Floor ContactDetection

Mic Holes10

131415Message From Brain

Body
Controller（ROS）
ROS border

MotionProcessor

EyeControl
DaemonOLEDOLED

Soundmixer
DaemonSpeaker

Main Processor (Linux)

Actuator
Actuator
Actuator
Actuator
Actuator
Actuator
Actuator
Actuator

CSTechLatest Robot Tech

MassProduction

ROSMobile
Wireless Tech
Camera Tech
Sensor Tech
Security Tech

MotionTech

Deep Learning
AI Reinforcement
Learning

Cloud
IoT
AWS IoTAWS Cloud

Mech

Inspection

ROS Embedded Optimization

ROS Transport Overview
XMLRPC:
configuration information
TCPROS/UDPROS:
data payload such as topic and services.

Computation Robot

Laptoprosmaster
XMLRPCXMLRPC

Image
ProcessXMLRPCTCPROS/UDPROSXMLRPC

CameraNodeDisplay
NodeROS Transport Overview
 Countermeasure Preconditions

 Whatʼs the problem?

• No change required for applications.

• CPU resource stress• Good affinity with TCP/UDP• Latency and Throughput.

• small latency and high throughput.
LaptopComputation Robotrosmaster

XMLRPCXMLRPCXMLRPC

Extend UDSROS layer with TCPROS/UDPROS,
but nothing else will be changed.
Our Scope

Image
Process

TCPROS/UDPROSXMLRPC

CameraNodeDisplay
NodeExpected Improvement as System
 Source Code• https://github.com/tomoyafujita/ipc-bench Environment• skylake(amd64) / hikey(arm64)• CPU frequency governor is “performance”• 100byte data payload / 10000 test loop average Performance Improvement for System
Category

Skylake

Hikey

Latency

UDS(STREAM)

1.82 us[usec]UDS(DGRAM)

2.14 us19.9 us

TCP/IP

3.14 us87.7 us

UDP/IP

2.78 us67.2 usUDS(STREAM)

1.95 Gbps

UDS(DGRAM)

1.98 Gbps

92 Mbps

TCP/IP

0.43 Gbps

12 MbpsThrough-Put

x1.7 ↓x4 ↑44.4 us127 Mbpsx2 ↓x10 ↑Extended Transport Overview
topic/service

ROS node

Software StackHardware Stack

System-A(OS instance)

System-B(OS instance)rosmasterNode(py)Node(cpp)
Node(py)Node(cpp)

roscppNode(py)Node(cpp)rospy

Node(py)Node(cpp)

roscpprospy

System Internal

UDSROS

System Internal

TCPROS/UDPROS

Network

System External

UDSROSTCPROS/UDPROS
System Boundary Aware, so itʼs automatically whichconnection should be used.

Improvement Result
 HelloWorld Benchmark• Publisher:Subscriber=1:1• Synchronous test loop using shared memory.
• Latency is the time window between publish and callback entrance.
• Environment: amd64(skylake) and arm64(hikey)

 Result Comparison
Platform
Skylake
Hikey

ROS
Transport

Duration [nsec]
minmaxaverage

TCP/IP

46,582

287,173

70,393

UDS

42,877286,130

64,796

TCP/IP

552,224

6,040,312

1,145,881

UDS

435,833

10,075,833

872,981

8% ↓

24% ↓

Github Repository
https://github.com/fujitatomoya/ros̲comm/tree/topic-kinetic-devel-uds-support
Pull Request
https://github.com/ros/ros̲comm/pull/1510
Tomoya Fujita <Tomoya.Fujita@sony.com>
Barry Xu <Barry.Xu@sony.com>

Direct I/O with rosbag
 Reference https://github.com/osrf/rosbag̲direct̲write

Process
page

Process

page

page

page
Data in userland will be stored
directly to the storage device
without CPU workload.

Page Cache(*) memory alignment constraints

File System
inodeinode

Block I/O

Epoll What is the problem?
• The more ROS topic connection, the more CPU stress with ros::spin()• Waiting for the stimulus during ros::spin without any message transmission.

• Poll system call is not good enough to take care of many file descriptors.

 Countermeasure• Using epoll instead, to reduce CPU stress.
• Already introduces as following commit, so backport the fix into 1.12.7 ros̲comm.
commit 9c0db37d9231003a7f162857e4aeb45675839609
Author: Mike Purvis <mike@uwmike.com>
Date: Thu Dec 21 11:31:08 2017 -0500
Topic subscription scalability fix (#1217)
Switching to using epoll system calls to improve performance of the topic polling code by a factor of 2.
This required disabling the addDelMultiThread test.

Polling Frequency
 ROS Threads
Thread Name

Use

Main

Main thread

PollManager

Publish/Subscribe I/O Polling

XMLRPCManager

Talk to rosmaster for connectivity map and services

ROSOutAppender

Logging

internalCallbackQueue

Connection drop, data store etc…

Polling Frequency Tunable for each processes, less CPU stress. (default freq is 100msec)

No Logging thread, less CPU stress. (instead of that, using own log system)

For ROS2
ROS2 Application
can be switched the RMW
implementation with environmental
variable.

Own RMW

Own
Implementation

Special Thanks to ROS Community

SONYはソニー株式会社の登録商標または商標です。
各ソニー製品の商品名・サービス名はソニー株式会社またはグループ各社の登録商標または商標です。その他の製品および会社名は、各社の商号、登録商標または商標です。