An Introduction to Team ViGIR’s
Open Source Software and DRC
Post Mortem

Stefan Kohlbrecher, Alberto Romay, Alexander Stumpf, Achim
Stein, Oskar von Stryk

Simulation, Systems Optimization and Robotics (SIM) Group, TU Darmstadt, Germany

Philipp Schillinger
Robert Bosch GmbH, Corporate Research, Department for Cognitive Systems. Stuttgart, Germany

Spyros Maniatopoulos, Hadas Kress-Grazit
Verifiable Robotics Research Group, School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, USA

David C. Conner
Christopher Newport University, Newport News, United States
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 1

Outline Intro• The DRC
• Team ViGIR (ROS based) Infrastructure System Overview
 DRC Finals
 Lessons Learned

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 2

The DARPA Robotics Challenge...close study of the disaster's first 24 hours, before the cascade of failures carried reactor
1 beyond any hope of salvation, reveals clear inflection points where minor differences
would have prevented events from spiraling out of control.
IEEE Spectrum, November 2011 pg. 36. (online version)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 3

DRC – Tasks and Rules

IEEE Spectrum
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 4

The “C” in DRC
 Uneven terrain, stairs andladders
 Motions with multiple
contacts (e.g. getting out
of a vehicle)

Versatile and robust(Loco-)Motion

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 5

The “C” in DRC
Versatile and robust

Perception

Versatile and robust(Loco-)Motion

 Perceive Environment for locomotion
 Perceive objects for manipulation
 Ability to acquire new objects and theirpotential purposes on the fly Robustness to different lightning
conditions

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 6

The “C” in DRC
Versatile and robust

Versatile and robust

Perception

Manipulation

Versatile and robust(Loco-)Motion

 Many different tools, only few exactly
known in advance Acquiring new manipulation modes Ability to coordinate manipulation,
locomotion & active perception

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 7

The “C” in DRC
Versatile and robust

Versatile and robust

Perception

Manipulation

Versatile and robust

Efficient interaction with

(Loco-)Motion

Human Supervisor

LimitedWireless Communication
bandwidth, delays, losses

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 8

DRC - The Meta-Challenges

 Highly compressed timeline Multiple competition events• VRC (June 2013)• Trials (Dec 2013)• Finals (June 2015)

 Systems integration
 High reliabilty• Only few attempts at tasks

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 9

Team ViGIR
International collaboration, Track B
Atlas team. VIrginia Germany
Interdisciplinary Robotics• TORC Robotics (Blacksburg, VA)• TU Darmstadt (Darmstadt,
Germany)• Virginia Tech (Blacksburg, VA)• Cornell University (Ithaca, NY)• Leibnitz Universität Hannover(Hannover, Germany)• Oregon State University (Corvallis,
OR)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 10

Team ViGIR

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 11

Team ViGIR Track B team, DRC participation from day one• Virtual Robotics Challenge (VRC)• DRC Trials• DRC Finals

 Software available: github.com/team-vigir• Exceptions:- Robot controller- Comms bridge

 Other teams using ViGIR software at DRC Finals• HECTOR (SIM, TU Darmstadt)• VALOR (TREC, Virginia Tech)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 12

Open Source Efforts by other
DRC competitors
 MIT:• Pronto State Estimator (pronto-distro github)• Drake Planning and Control (drake github)• Director UI (director github) IHMC:• IHMC Controller• SCS Simulator(ihmc_ros bitbucket)

 JSK:• Extensive ROS-based Software (jsk-ros-pkgs github)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 13

Hardware Boston Dynamics (BDI)
Atlas robot• Hydraulically actuated Our Atlas nicknamed“Florian” (after patron
saint of firefighters)
 API provided by BDI• Walking/Stepping• Balancing➔

Upper body planningdecoupled from low levelbalance control

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 14

Hardware – Atlas Versions
 Atlas V3 (2013–Nov 2014)• Tethered• 6DOF arms

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 15

Hardware – Atlas Versions
 Atlas V3 (2013–Nov 2014)• Tethered• 6DOF arms

 Atlas V4 (Feb 2015-Mar
2015)• Untethered• Onboard Computing• 6DOF arms

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 16

Hardware – Atlas Versions
 Atlas V3 (2013–Nov 2014)• Tethered• 6DOF arms

 Atlas V4 (Feb 2015-Mar
2015)• Untethered• Onboard Computing• 6DOF arms

 Atlas V5 (Apr 2015-Aug
2015)• As V4, but 7DOF arms

(lower 3 joints electric)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 17

Infrastructure Use of ROS from the beginning• Prior experience• Great community• A lot of useful software• Integration with DRCsim

 Private git(lab) repos• Now moved to github Project management via Redmine• Every task in issue tracker• Hundreds of Wiki-pages

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 18

Timeline with a Focus on
Infrastructure
Oct 12 Jan 13

Jan 14VRC

DRC Trials

Atlas V3
Atlas V4
Atlas V5
Ubuntu 12.04
14.04

ROS

FuerteGroovyHydro
Indigorosbuild
catkinOctober 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 19

Jan 15DRC Finals

Jan 16

Infrastructure – Managing
Robot Variability
 Many variants:• 3+ Atlas versions• 4 Hand types

 Could use args/params• Unwieldy to forward Use environments
variables Generate robot model(and onboard software
setup) at launch-timethrough launch files

github.com/team-vigir/vigir_atlas_common/blob/master/atlas_description/robots/vigir_atlas.urdf.xacro
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 20

Infrastructure – Deployment tomultiple machines Complex system• 3 Onboard Computers• 1 Field Computer• 4 OCS Computers Fast development cycles• Build and deploy quickly and consistently Remotelaunch scripts• Build using catkin (install)• Deploy using rsync• Start using ssh/screen

github.com/team-vigir/remotelaunch
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 21

Infrastructure – Deployment tomultiple machines
Launch scripts for each machine

Common environment
setup executed on eachmachine

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 22

Infrastructure – Simulation
Options
 No single solution that can do everything
currently available (as open source)• IHMC controller/Atlas with Gazebo integration to bereleased this fall

Simulator/Robot

Locomotion

Manipulation

Remarks

Atlas/BDI/DRCsim

(Yes)

No

Only with BDI lib

Atlas/IHMC/SCS

Yes

No

Atlas/IHMC/DRCsim

(Yes)

(Yes)

Coming soon :)Valkyrie/IHMC/DRCsim

(Yes)

(Yes)

Coming soonThor-Mang/Gazebo4

No

Yes

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 23

Components - Controls

Hardware

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 24

IMU

Controls Use of BDI supplied library• Walk (dynamic stability)• Step (static stability)• Manipulate (balance while standing) Provided as binary• Black box, no source (also for DRC

teams)• Not available to general public :( Effort to integrate IHMC Whole
Body controller• Use in competition prevented bytime constraints/delays• Coming soon

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 25

Components – State Estimation

Hardware

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 26

IMU

Components – State Estimation

State Estimation
State Estimator

Hardware

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 27

IMU

State Estimation
 Provide state (pose) estimate forrobot
 Fuse• Leg Kinematics• IMU

 Continuous but drifting estimate• Low drift with good sensors

 Use MIT's pronto• Tuned for Atlas system• pod build system• LCM communications

 LIDAR use dangerous in non-static
environment
pronto-distro (ViGIR fork)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 28

Components – Perception

State Estimation
State Estimator

Hardware

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 29

IMU

Components - Perception

Perception
World model
serverState Estimation

LIDAR
Filter

HardwareState Estimator

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 30

IMUPerception
 Provide situationalawareness for operator(s) Provide world state
estimate for robot• Footstep planning• Manipulation

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 31Perception LidarOctomapUpdater Environment octomap
updated in real-time Provide collision model forplanner Also provide filtered LIDAR
data for overall system• Annotate with transform
information as tf
prohibitive overconstrained comms
github.com/team-vigir/vigir_manipulation_planning/tree/master/vigir_lidar_octomap_updater
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 32

Perception – Compressing
LIDAR Data
 Standard scan too big for1500 Byte UDP limit

github.com/team-vigir/vigir_perception/tree/master/vigir_filtered_localized_scan_utils
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 33

Perception – Compressing
LIDAR Data
 Standard scan too big for1500 Byte UDP limit Compress:• Split (3 separate scans)• Distances to uint_16• Intensities to uint_8• Self filter bit Add start/end global
transform info Can reconstruct on OCS
side• Every compressed scan

usable standalone

github.com/team-vigir/vigir_perception/tree/master/vigir_filtered_localized_scan_utils
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 34

Perception – World Model
Server Collect LIDAR data
 Provide services• Pointcloud ROIs
• Octomap ROIs
• Gridmap slice ROIs
• Distance queries Two instances• Onboard• OCS Sync via compressed scans
github.com/team-vigir/vigir_perception/tree/master/vigir_worldmodel_server
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 35

Situational Awareness using
Fisheye Cameras
 Fisheye cameras
provide high FOV
 Hard to interpretfor humans
 Calibrate Fisheye
cam using theocamlib toolbox
 Virtual pinhole
camera that follows
tf frames
github.com/team-vigir/vigir_wide_angle_image_proc

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 36

Mesh Visualization
 Latest image data texture mapped onto mesh
• Depth image-based: Fast update rate, low range• LIDAR-based: Low update rate, high range

Rviz Mesh
DisplayPlugin

RBG Image

Depth Image

DepthImg to
Mesh Nodeshape_mgs/Mesh

World model
serverLIDAR cloud

Cloud to Mesh
Node

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 37

Mesh Visualization

Rviz Mesh
DisplayPlugin

shape_mgs/Mesh

DepthImg to
Mesh Nodegithub.com/team-vigir/vigir_perception/tree/master/vigir_point_cloud_proc
github.com/team-vigir/vigir_ocs_common/tree/master/vigir_ocs_rviz_plugins/vigir_ocs_rviz_plugin_mesh_display_custom
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 38

Components – Footstep Planner

Perception
World model
serverState Estimation

LIDAR
Filter

HardwareState Estimator

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 39

IMU

Components – Footstep Planner

Perception

Planning

World model
serverFootstep
PlannerState Estimation

LIDAR
Filter

HardwareState Estimator

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 40

IMU

Footstep Planner Based on work by Hornung et. al. [1]• A*-search-based planning approach

Discrete Foot Placements

Successor Set[1] Hornung et.al. Anytime Search-Based Footstep Planning with
Suboptimality Bounds, Humanoids 2012
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 41

Footstep Planner Complex Locomotion:• 3D perception and modeling• Safe sequences of foot placements• 6DOF foot placements• Obstacle avoidance• Balance control

 Divide and conquer• Terrain Model Generator• 3D Footstep Planning• Robot Controller
github.com/team-vigir/vigir_footstep_planning_core
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 42

Terrain Model Generator
 Only point clouds required• Octree as back-end• Incremental updates• Stand-alone ROS package- Usable in other domains

github.com/team-vigir/vigir_terrain_classifier
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 43

Terrain Model Generator
 Online Generation• Surface Normals (left)• Height Map (right)

Video: Terrain Model Generator

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 44Footstep Planner: 3D Planning Extension to 3D

 States: Become full 6 DOF
 Actions: Remain the same
 Roll, pitch and step height are constrained by underlying terrain
 Search space does not enlarge
 No expensive branching tree!

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 45

Footstep Planner: 3D Planning Ground contact estimation• Sampling of foot surface• Estimate contact situation of eachsample using height map• More flexible collision checking
model• Allows overhanging steps

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 46

Footstep Planner: Plugins

 Plugins used forcustomization of
all relevant systemaspects
 Setups for 3 robots
already available:• Atlas• Thor-Mang• ESCHER

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 47

Footstep Planner: Example

Video: Traversing Chevron Hurdles

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 48

Footstep Planner: Interactivity

Video: Interactive Planning

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 49

Components – Footstep Planner

Perception

Planning

World model
serverFootstep
PlannerState Estimation

LIDAR
Filter

HardwareState Estimator

Robot
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 50

IMU

Components - Manipulation

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation
OT
ServerState Estimation

Manipulation
Controller

LIDAR(s)State Estimator

Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 51

IMU

Motion Planning - Requirements
 Manipulation• Collision free planning• Cartesian Paths• Manipulation in contact with environment• Maintain stability

 Sliding Autonomy:• Operator/OCS-based (Teleop)• Operator/Object template based (Task level)• Behavior Executive (Autonomous)➔

Use MoveIt! as back-end

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 52

Motion Planning – Robot Setup Different robot variants
 Different hand variants Combinatory explosionof configs• Do not want to run setup

assistant for every(possible) combination

 Solution:• Use of xacro macros tochange configs

github.com/team-vigir/vigir_atlas_planning/tree/master/vigir_atlas_moveit_config

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 53

Motion Planning - Overview

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 54

Planning - Capabilities
 Additional move_groupcapability• Different types of motion

requests- Joint goal
- Cartesian goal
- Cartesian Path (waypoints)- Circular motion• Specify planning referencepose relative to endeffector• Constrain joint limits
selectively at run-time

TYPE_FREE_MOTION
TYPE_CARTESIAN_WAYPOINTSTYPE_CIRCULAR_MOTION
geometry_msgs/Pose[] target_poses
string reference_point_frame
geometry_msgs/Pose reference_point

github.com/team-vigir/vigir_manipulation_planning/tree/master/vigir_move_group
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 55

Planning – Object Templates On top of vigir_move_group Operator places objects
 Planning relative to
instantiated objects templates
 Object template library• Geometry• Mass/Inertia• Grasps• Stand poses
github.com/team-vigir/vigir_object_template_manager
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 56

Planning - “Ghost” robot
 Pre-plan motions with virtual“Ghost Robot”
 Additional capabilities
compared to start/goal statevisualization in MoveIt! Rviz
plugin• Snap endeffectors to objects• Move to stand poses relative toobject templates• Constrain IK joint limits• Send low-bandwidth planning
request directly from OCS

github.com/team-vigir/vigir_manipulation_planning/tree/master/vigir_ocs_robot_model

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 57

Manipulation Pipeline Example

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 58

Manipulation Pipeline Example

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 59

Manipulation Pipeline Example

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 60

Manipulation Pipeline Example

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 61

Manipulation example

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 62

Manipulation – Drake
Integration
 Switch between MoveIt! andMIT's Drake planning framework
on a per plan request basis• Whole Body Motions• Usinggithub.com/tu-darmstadt-ros-pkg/r
osmatlab

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 63

Manipulation – Reaching motion
using Drake Integration

Video: Reaching Motion

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 64

Components – FlexBE Behavior
Executive

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation
OT
ServerState Estimation

Manipulation
Controller

LIDAR(s)State Estimator

Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 65

IMU

Components – FlexBE Behavior
Executive

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation

Behavior
Executive

OT
ServerState Estimation
FlexBE

Manipulation
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 66

State Estimator

IMU

Behavior Executive - High-Level
Approach Communication constraints
 Limited time Complex robot system

Motivates high degreeof robot autonomy

Flexible Robot-Operator Collaboration

 Unstructured environment Complex tasks
 Robustness important
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 67

Motivates high degreeof operator supportBehavior Executive - High-Level
Approach SMACH, XABSL, etc.
• Focused on pure autonomy• Pre-defined robot behavior

 Required features:• Allow multiple degrees of autonomy• Support and restrict robot when in low autonomy• Adapt behavior to unforeseen situations• Abstraction of complex behavior design• Robust against runtime failure

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 68

Behavior Executive - FlexBE
 “Flexible Behavior Engine”• Based on SMACH → Hierarchical state machines• Adds robot-operator collaboration• Available on GitHub: github.com/team-vigir/flexbe_behavior_engine

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 69

FlexBE – States
 Interface basic robot capabilities / actions
 Executed periodically Event-based lifecycle (simplified):

on_enter

executeon_exitSend command(s), eg.
- publish message- actionlib call

Check conditions andevaluate results→ Determine outcome

Clean up

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 70

FlexBE – Autonomy Level Behavior runs with explicit Autonomy Level• Can be changed any time during execution

 State outcomes define required autonomy• High enough → Autonomous execution• Too low → Operator confirms or rejects

 Operator can force outcomes any time

Close Fingers

success
missedOctober 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 71

Lift Object

Try Alternative

Autonomy
Off→ Low ←High
Full

FlexBE – Data Input
 Behavior can request required data from operator
 Integrated into operator control station

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 72

FlexBE – Runtime Changes
 Behavior is locked in a specific state Modifications are sent to the onboard executive
 New version is generated and imported
 Active state is transferred• Extracted from old, running version• Integrated into new version

 Old version is stopped
 New version is executed→ Arbitrary adaptation
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 73

FlexBE – User Interface
 Facilitates behavior development
 Automated code generation
 Integrated operator interaction→ Is prerequisite for operator-robot collaboration- Behavior re-definition during runtime feasible- Transparent robot decision-making- Send context-dependent high-level commands

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 74

FlexBE – Editor
 Drag&Drop state composition
 Configuration of state properties Detailed documentation of states Dataflow graph and verification

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 75

FlexBE – Runtime Control

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 76

FlexBE – Beyond DRC
application

Video: FlexBE Demo

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 77Components – Comms Bridge

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation

Behavior
Executive

OT
ServerState Estimation
FlexBE

Manipulation
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 78

State Estimator

IMU

Components – Comms BridgeComms BridgeComms Bridge

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation

OCS Master
Onboard Master
Comms Bridge

Behavior
Executive

Comms Bridge

OT
ServerState Estimation
FlexBE

Manipulation
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 79

State Estimator

IMU

Comms Bridge
 Single ROS Master infeasible• Unreliable connection between operator

and robot

 Dual Master approach• OCS• Onboard Prioritization
 Special treatment of high rate statedata• Compress using domain knowledge

 Other data compressed using
blob_tools• Bz2 compression per default

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 80

Comms Bridge

OCS Master
Onboard Master
Comms Bridge

Components - OCS
Comms BridgeComms Bridge

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

Hardware

Robot
Controller

Template
Manipulation

OCS Master
Onboard Master
Comms Bridge

Behavior
Executive

Comms Bridge

OT
ServerState Estimation
FlexBE

Manipulation
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 81

State Estimator

IMU

Components - OCS
OCSMain View

World model
serverCamera View

Comms Bridge

OT
ServerMap View

Perception

Planning

World model
serverFootstep
Planner

LIDAR
FilterManipulation
Planner

HardwareComms Bridge

Robot
Controller

Template
Manipulation

OCS Master
Onboard Master
Comms Bridge

Behavior
Executive

Comms Bridge

OT
ServerState Estimation
FlexBE

Manipulation
Controller

LIDAR(s)Camera(s)October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 82

State Estimator

IMU

OCS
 3D visualization based on librviz• Map View (Top Down)- Rectangle selection (query sensor data ROI)
• Main View
• CameraView- Camera data visualization

 Multiple Qt widgets for general controls• “Ghost Control”• Pre-canned joint configurations

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 83Components – Install Install instructions for complete setups:
github.com/team-vigir/vigir_install/wiki• Waiting for Atlas IHMC/Gazebo integration for full capability(walk/manipulate) Atlas example• Thor-Mang example available for manipulation demo:
github.com/team-vigir/vigir_install/wiki/Install-thor-mang-v
igir-gazebo

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 84

Components – Tutorial video

Video: Open Source Tutorial

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 85

Work in Progress - Behavior
Synthesis

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 86Work in Progress - Behavior
Synthesis Compile formal Linear Temporal Logic (LTL)
specification from:• High-level task (goals and initial conditions)• Abstract description of the robot-plussoftware system, defined a priori (think
config files)
 The formalism treats theoutcomes of actions as
an adversarial environment
github.com/team-vigir/vigir_behavior_synthesis
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 87

Work in Progress
Automatically synthesize
a finite-state automatonthat is guaranteed tosatisfy the formal LTL
specification no matter
what the environment
does.

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 88Work in Progress Mapping from abstract symbols to low-levelsystem components (here, FlexBE states)
 Instantiation of symbolic automaton as an
executable state machine in FlexBE
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 89

Behavior Synthesis - Example

Video: Behavior Synthesis Demo

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 90

DRC Finals
 Decision not to do egress• Significant development effort• Risk of (catastrophic) damage to robot

 Limited testing under degraded comms conditions

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 91

DRC Finals – Day 1

Video: Day 1 Pt.1
Video: Day 1 Pt.2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 92

DRC Finals – Day 1

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 93

DRC Finals – Day 1

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 94

DRC Finals – Day 1

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 95

DRC Finals – Day 1

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 96

DRC Finals – Day 1
 Flawless Driving
 Comms bridge setup issue• Behavior control• Footstep planning Switch to teleop mode
 Slow but reliable

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 97

DRC Finals – Day 2

Video: Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 98

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 99

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 100

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 101

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 102

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 103

DRC Finals – Day 2

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 104

DRC Finals – Day 2
 Start delay due to arm hardware failure (Too) fast driving
 Reset after touching barrier Successful driving
 Door opened
 Pump shutdown• Possibly due to overheating Reset Fall while walking through door

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 105

DRC Finals Results
 3 Points (Day 1)
 Scored lower than would have been achievable andexpected• Achievable: 7 points (No egress)

 Missed chance at Day 1 due to comms issues
 Unknown cause for pump shutdown at Day 2
 Driving approach worked well on both robots that used it• ViGIR Florian (Atlas)• HECTOR Johnny (Thor-Mang)

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 106

Lessons Learned - ROS
 Workspace setup using wstool works well
• Few convenience scripts helpful Keeping pace can be painful• From rosbuild to catkin• From hydro to indigo (switching ROS distro and Ubuntu version

simultaneously) Using plain “catkin_make” in large projects bad idea
• Use catkin_tools

 Limited constrained comms capability
 Supporting different configurations feels more involved than it
should• Environment variables?

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 107

Lessons Learned – Big Picture Having a transatlantic, nine time zone team works• Right mindset and people• Tools

 DRC showed what is possible• Brilliant display of state of the art capabilities• Still a long way to go till robots can be useful for real DRC-like tasks

 Continuous Integration• Simulation-in-the-loop testing desirable

 Everybody wins
• Leap across wide range of capabilities• Open source developments (Gazebo, code releases..)• Incredible sportsmanship and cooperation across DRC teams

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 108

Conclusions
 DRC overview
 ROS infrastructure
discussion• Useful tools

 Intro to open sourcecomponents• Let us know what you'reinterested in DRC results discussion
 Lessons learned

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 109

Questions?

References Kohlbrecher et al. “Overview of team ViGIR's approach to the Virtual Robotics
Challenge”, IEEE SSRR 2013
 Kohlbrecher et al. “Human-Robot Teaming for Rescue Missions: Team ViGIR´s
Approach to the 2013 DARPA Robotics Challenge Trials” Journal of Field
Robotics, 2014
 Romay et al. “Template-Based Manipulation in Unstructured Environments forSupervised Semi-Autonomous Humanoid Robots”, IEEE Humanoids 2014
 Stumpf et al. “Supervised Footstep Planning for Humanoid Robots in RoughTerrain Tasks using a Black Box Walking Controller”, IEEE Humanoids 2014

 YouTube playlist with manipulation examples

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 110

Additional Material - Driving
ApproachVideo: Driving Approach

October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 111

Additional Material – DRC
Communication constraints
Uplink (to OCS)

Downlink (to robot)

Remarks

VRC

Total ~115 kB for 30
minutes. 500 ms
latency

Total, ~7 MB for 30
minutes. 500 ms
latency

Worst case (20% of
scenarios)

Trials

1 MB/s, 50ms latency

1 MB/s, 50 ms latency

Good comms

100 kB/s, 500 ms
latency

100 kB/s, 500 ms
latency

Bad comms

1.2 kB/s

1.2 kB/s

Finals300 Mbit/s

VRC Rules (pdf)
Trials Rules (pdf)
Finals Rules (pdf)
October 3rd, 2015 | Team ViGIR at the DRC | Stefan Kohlbrecher | 112

Switchevery
minuteOutages of 1-30
seconds after robot
traverses door