Sharron LIU, Yu YAN
Nov. 1st 2019

Who We Are
PerformanceSecurity• The Intel Open source Technology
Center (OTC, https://01.org)• The Robotics Software Engineering
Team delivers ROS2 packages
optimized on IA platforms

Edge
ServerVision
Robotics
Technologies

FunctionalSafety

AI

Real Time2

Why Do We Need Acceleration for Robotics?
CNN* based AI methods become popularin robotics systems•

Github
CNN ROSdeep learning ROS
IEEE

Number of repos(by 09/2019)

30
71
Number of papers(by 09/2019)

CNN Robot729
deep learning Robot 1788

CNN inference•••

Computational complexity•

Latency•

CPU loadsTime critical requirements from robotic
applications
https://www.shutterstock.com/image-photo/agriculture-technology-artificial-intelligence-concepts-farmer-734412016

CNN: Convolutional Neural Network

3

OpenVINOTM (Intel® Open Visual Inference &Neural network Optimization toolkit)

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html•100+ open source and public
pre-trained models•

Various inputs subject to theCNN model itself•

Common API across multiple
Intel® platforms

4Our works to connect OpenVINOTM with Robotics

5

ROS2 OpenVINOTM Toolkit• Deliver ROS2 topics andservices• Support OpenVINOTM
accelerated open models• People: face, emotion,age, gender, head pose,
person reidentification• Objects: detection,segmentation

Photo: Intel Robotics Software Engineering Team

Photo: Intel Robotics Software Engineering Team

• Execute on CPU, GPU, or
Movidius VPU
6

OpenVINOTM Grasp Pose Detection
1. Perpare a deployment model
Grasp Pose Detection developed in NEU

2. Run Model Optimizer to convert the model
python3 mo_caffe.py--input_model <path_to_.caffemodel>--input_proto <path_to_.prototxt>--output_dir ./fp32--data_type FP32 --log_level DEBUG

3. Use Inference Engine API to create classifier

Andreas ten Pas, Marcus Gualtieri, Kate Saenko, and Robert Platt. Grasp Pose
Detection in Point Clouds
The International Journal of Robotics Research, Vol 36, Issue 13-14, pp. 1455 1473. October 2017

OpenVINOTM classifier
Useful resources•

Model optimization FAQ, inference application examples, OpenVINOTM Toolkit documentations
7

ROS2 Grasp Library
Subscribed Topics•

RGB image from sensor(sensor_msgs::msg::Image)•PointCloud2 topic from RGBD sensor(sensor_msgs::msg::PointCloud2)•

Segmented object topic(people_msgs::msg::ObjectsInMasks)

Delivered Services•

plan_grasps (moveit_msgs::srv::GraspPlanning)

8

ROS2 Grasp Library
Grasp applications (random picking, recognition picking)Grasp Poses
Plan Grasps(obj_name = “”)Grasp Planning
OpenVINOTM Grasp
Detection

camera-robot
TF

Pick, Place

Robotstate

Robot Interface
Hand-Eye Calibration
Robot driver
Gripper driver9Intelligent Visual Manipulation Applications

Random picking
OpenVINOTM Grasp Detection @GPU + MoveIt

https://www.youtube.com/playlist?list=PLxCmGJeiLgoxq3uqcCVSYnSJ9iQk1L9yP

Recognition & picking
OpenVINOTM Object Segmentation +OpenVINOTM Grasp Detection +Hand-Eye calibration and transformation
10

Summary• CNN based AI methods become popular in intelligent robots• OpenVINOTM toolkit optimize and deploy deep learning solutions acrossmultiple Intel® platforms• 4 deep learning frameworks, 100+ pre-trained models, one API for all devices• Use converted open models – ROS2 OpenVINOTM Toolkit• Convert your own models – like what we did in the industrial robot• OpenVINOTM Grasp Detection model and classifier• ROS2 Grasp Library connect OpenVINOTM and the MoveIt framework• Intelligent visual manipulation applications running on real robot
11Resource Links and Contacts• https://github.com/intel/ros2_grasp_library• https://github.com/intel/ros2_openvino_toolkit

Sharron LIU

Yu YAN

Chao LI

Lewis LIU
12

Disclaimers
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness
for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, orusage in trade.
This document contains information on products, services and/or processes in development. All information provided here is
subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications androadmaps.

The products and services described may contain defects or errors known as errata which may cause deviations from publishedspecifications. Current characterized errata are available on request. No product or component can be absolutely secure.
Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-5484725 or by visiting www.intel.com/design/literature.htm.
Intel, the Intel logo, Intel RealSense, OpenVINO, and the OpenVINO logo are trademarks of Intel Corporation or its subsidiaries inthe U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© Intel Corporation14