Using ROS2 for Vision-Based
Manipulation with Industrial Robots

Adam Allevato
allevato@utexas.edu

Karsten Knese
karsten.knese@us.bosch.com

QoS

DDS
Auto-Discovery

Cross Platform
ament_resource_index

Multi-NodeProcesses
Composition
Namespacing
Security
ROS1 Bridge
Parameters
Lifecycle Nodes
Cross Language

Intra Process
Communication

QoS

DDS
Auto-Discovery

Cross Platform
ament_resource_index

Multi-NodeProcesses
Composition
Namespacing
Security
ROS1 Bridge
Parameters
Lifecycle Nodes
Cross Language

Intra Process
Communication

The big picture
Joint State Controller
Robot HAL// publish joint states
update()while(true){read_joints()
controller_manager.update()
write_joints()}

Joint Trajectory Controller// listen to traj msg// set joint states
update()More details from Jackie Kay and Adolfo Rodriguez Tsouroukdissian at ROScon 2015
https://roscon.ros.org/2015/presentations/RealtimeROS2.pdf

Controller Manager
ControllerManager::load_controller(package_name, class_name, controller_name){auto library_path = get_ament_resource(package_name, class_name);
auto loader = std::make_shared<class_loader::ClassLoader>(library_path);

HAL

auto controller =loader->createInstance<controller_interface::ControllerInterface>(class_name);
controller->init(controller_name);
executor_->add_node(controller->get_node_base_interface());

Controller}

int main(int argc, char ** argv){
cm.load_controller(“ros_controllers”, “ros_controllers::JointTrajectoryController”, “trajectory_controller/left”);
cm.load_controller(“ros_controllers”, “ros_controllers::JointTrajectoryController”, “trajectory_controller/right”);}

https://github.com/ros2/ros2/wiki/Composition

Ament Resource Index<ros2_ws/install>`-- share|`-- ...

# Other, non-plugin related, stuff|-- ament_index

HAL|-- resource_index|-- packages|`-- controller_manager|`-- ...

Controller|-- controllers|`-- ros_controllers# containing controller info

$ cat /ros2_ws/install/share/ament_index/resource_index/controllers/ros_controllers
ros_controllers::JointStateController;bin/default_controllers.dll
ros_controllers::JointTrajectoryController;bin/libdefault_controllers.dll

https://github.com/ament/ament_cmake/blob/master/ament_cmake_core/doc/resource_index.md

Lifecycle Controller Nodes
class ControllerInterface : public LifecycleNodeInterface{
virtual rcl_lifecycle_transition_key_t
on_configure(const State & previous_state);
virtual rcl_lifecycle_transition_key_t
on_activate(const State & previous_state);
virtual rcl_lifecycle_transition_key_t
on_deactivate(const State & previous_state);
virtual rcl_lifecycle_transition_key_t
on_cleanup(const State & previous_state);

HALController

virtual rcl_lifecycle_transition_key_t
on_shutdown(const State & previous_state);
virtual rcl_lifecycle_transition_key_t
on_error(const State & previous_state);};
http://design.ros2.org/articles/node_lifecycle.html

Lifecycle Controller Nodes
class MyController : public ControllerInterface{// do lifecycle controller stuff
on_activate()...
};$ ros2 service list
/my_controller/change_state/my_controller/get_available_states/my_controller/get_available_transitions/my_controller/get_state

HALController$ ros2 service call /my_controller/get_state lifecycle_msgs/GetState
lifecycle_msgs.srv.GetState_Response(
current_state=lifecycle_msgs.msg.State(id=1, label='unconfigured'))https://github.com/ros2/ros2/wiki/Managed-Nodes

The Picky Robot: Motivation●●Show that ROS2's core features are in place
Do some integration but also add some new functionality

Let's build a Pickand Place demo!

Architecture
Astra Depth
CameraAstra Camera
DriverObjectDetector(LINEMOD)Shared Memory for Intraprocess Communication

https://github.com/ros2/ros2/wiki/Intra-Process-Communication

Architecture
Astra Depth
CameraAstra Camera
DriverObjectDetector(LINEMOD)

Intraprocess
Communication

Architecture
Astra Depth
CameraAstra Camera
DriverObjectDetector(LINEMOD)

Intraprocess
Communication

6DOF
Pose

Robot
Interface

UR5 Robot

Architecture
Astra Depth
CameraAstra Camera
DriverObjectDetector(LINEMOD)

RobotInterface

Intraprocess
Communication
ROS 2 launch file (Python)

UR5 Robot

Architecture
Astra Depth
CameraAstra Camera
DriverRViz

ROS2 bridge

ObjectDetector(LINEMOD)

Robot
Interface

Intraprocess
Communication
ROS 2 launch file (Python)

UR5 Robot

Architecture
Astra Depth
CameraAstra Camera
DriverRViz

ROS2 bridge

ObjectDetector(LINEMOD)

Robot
Interface

Intraprocess
Communication

Docker Container

ROS 2 launch file (Python)

UR5 Robot

http://www.trossenrobotics.com/Shared/images/PImages/widegripper/rg-wide-gripper-a.jpg

X

http://www.trossenrobotics.com/Shared/images/PImages/widegripper/rg-wide-gripper-a.jpg

Videohttps://vimeo.com/228848126

QoS

DDS
Auto-Discovery

Cross Platform
ament_resource_index

Multi-NodeProcesses
Composition
Namespacing
Security
ROS1 Bridge
Parameters
Lifecycle Nodes
Cross Language

Intra Process
Communication

QoS

DDS
Auto-Discovery

Cross Platform
ament_resource_index

Multi-NodeProcesses
Composition
Namespacing
Security
ROS1 Bridge
Parameters
Lifecycle Nodes
Cross Language

Intra Process
Communication

Special Thanks to ...

Calvin Ngan - Karsten's Boschsummer intern

Mikael Arguedas - Adam's Open
Robotics summer mentor

Thank you!
Code available:● https://github.com/kukanani/picky_robot● https://github.com/ros-controls/ros2_control● https://github.com/ros-controls/ros2_controllers

Adam Allevato
allevato@utexas.edu

Karsten Knese
karsten.knese@us.bosch.com