Robot programming, Simulation and Environment
using Choreonoid for Humanoid Beginner

2022/11/28
Yohei Kakiuchi
Toyohashi University of Technology

Outline• Robot competitions using Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

Outline• Robot competitions using Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

Robot competition using simulator• RoboCup https://www.robocup.org/– Soccer(sim) developed multi agent simulator, Rescue(sim),@Home, Industrial• DARPA Challenge– Grand(2005) / Urban(2007) Challenge– Robotics (Virtual2013, Trial2013, Final2015) Challenge– Subterranean Challenge (2017-2021) (Trial?)
https://en.wikipedia.org/wiki/DARPA_Grand_Challenge• JVRC (Japan Virtual Robotics Challenge) 2015
• WRS2020 (world robot summit)– Tunnel disaster challenge• HVAC (Humanoid Virtual Athletics Challenge) << This WS3

Robot competition using simulator• Why simulator is used at competitions?
• Difficulties in real robots– Hardware (preparations robot, maintenance, environment)• Wider variety of participants– Expert, Novice (experience)– Researcher, Developer and Hobbyist (profession)– Software, Hardware and Systems (speciality, interest)• Targets (organizer, participants)– Boosting humanoid robots research– Testing new algorithms– Testing new hardware– Testing new integrate system

4

Virtual Robotics Challenge 2013
• Using Gazebo as simulator– Gazebo running in server, participants connecting from their site

• Target of competition– Required to solve real tasks– Control system– Novel robot interface– Share autonomous algorisms and operator's input
Virtual Robotics Task 1

Virtual Robotics Task 2Virtual Robotics Task 3

Robot control environment using Gazebo (VRC)• Gazebo(http://gazebosim.org/)– Dynamics engine (ODE base)– Various environments (drcsim)– Highly compatible with ROS(gazebo-ros-pkgs)

RVIZ (ROS visualization andcontrolling by marker)
Model expression
on EusLisp

In Gazebox15

JVRC(Japan Virtual Robotics Challenge)• Computer Simulation Competition for Disaster Response
Robots– Target task: Disasters in tunnels– Oct 7〜10, 2015
– Simulator: Choreonoid– Participating Teams: 127

JVRC(Japan Virtual Robotics Challenge) Tasks••••Basic mobility in narrow areas
Manipulation (heavy objects)
Inspection
Searching for missing persons

Robot control environment using Choreonoid (JSK lab.)• Configured to use the same interface as the real robot• Almost same GUI using DRC
3DGUI

2DGUI

Robot control environment using Choreonoid

WRS(World Robot Summit)

Task T1
Traversing Obstacles

Task T2
Vehicle Inspection

Task T3
Rescue using Tools

Task T4
Secure the RouteTask T5
Fire Extinguish

Task T6
Shoring and Breaching

11

WRS2020 (Tunnel disaster challenge)• Tunnel disaster challenge– https://worldrobotsummit.org/wrs2020/challenge/disaster/tunnel.html

12

WRS2020 (Tunnel disaster challenge)• Stage Gate (Locomotion through rough terrain)

13

WRS2020 (Tunnel disaster challenge)• Stage Gate (Manipulation – Heavy Object / Tools )14

WRS2020• Stage Gate (Investigation using camera)

15

HVAC (Humanoid Virtual Athletics Challenge)• Whole body control for difficult environment• Operator set the target
Team Jaxon(2021)

x5
HVAC (Humanoid Virtual Athletics Challenge) (https://ytazz.github.io/vnoid/)Roundup of robot competitions using simulation• Simulator should be easy to use– Most important performance is speed (second is accuracy)– Well documented and many samples• How to increase participants– Good samples (correspondence with a thesis)– Allow for variety of purpose (few restrictions)– Increased complexity of task is a trade-off for the entry
barriers.

• My personal hope is that a real robot is a familiar target
17Outline• Robot competitions and Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

General Information (Choreonoid)• Choreonoid– Open source‣https://choreonoid.org/‣https://github.com/choreonoid– Integrated GUI platform for Robotics‣Choreography (for dancing)‣Simulation‣Visualization of sensors‣Remote control– Plugin system• From 2019, Start of business activities for commercial use••

「統合ロボットシミュレータChoreonoidの最新機能」 計測と制御2018年57巻10号p.700-705
https://www.jstage.jst.go.jp/article/sicejl/57/10/57_700/_pdf/-char/ja19

Choreonoid• Integrated GUI platform– Can be used for various purposes– GUI can be customized through the use of extensions– Using Libraries for robotic programming1. Robot simulator

3. Choreography

Used in JVRC (Japan virtualrobotics challenge)

Automatic balance adjustment function
allows choreography with the appearanceof a CG character.

2. Remote Operation Interface
Remote Operation Interface
in DRC Finals

20Choreonoid (Choreography)

Choreonoidを用いて作成したヒューマノイドロボットHRP-4Cの動作例
https://choreonoid.org/ja/about.html

21

Choreonoid (Choreography and Whole body dynamics)

General Information (Choreonoid)• Software structure of Choreonoid (3 main libraries)– Body (Robot model) library‣Robot structure – Body, Link, Sensor‣Kinematics‣Dynamics– Base system (GUI) library‣View (3D visualize view, etc.)‣Panel for Body, Link, Sensor‣Tool bar

– Utility library‣Robot model loader (using yaml)‣Matrix (Eigen), Algorisms– Python bindings
23General Information (Choreonoid)• Software structure of Choreonoid (3 main libraries)– Robot model library– Base system (GUI) library– Utility library– Plugin system
Tool Bar
PanelModel View

24

General Information (Choreonoid)• Software structure of Choreonoid– Plugin system‣User can add plugin‣Enable to integrate other libraries

Platform
UserBody Library

Util LibraryBase LibraryUser Library (dynamics , control)
Default Plugin for GUI system
User Plugin for GUI system
25

Outline• Robot competitions and Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

Layered software configuration for simulator and real robot
Hardwarecontrol

Real-time
Body control

Non real-time
Behavior control

Simulator

RealRobotRecognition

Useroperation (UI)

Humanoid robot control system in simulation• Humanoid robot control system– Body control
Right image (Binarization)
Left image 右カメラ2値化処理– Behavior control 左カメラ2値化処理– Recognition– Operator (UI)

カメラ画像
Image
fromCamera

Humanoid robot control system in simulation• Humanoid robot control system– Body control
Right image (Binarization)
Left image 右カメラ2値化処理– Behavior control 左カメラ2値化処理– Recognition– Operator (UI)

カメラ画像
Image
from
CameraHybrid locomotion (In simulator)
Moving over steps

Obstacle Avoidance Leg
Wheel Movement30

Outline• Robot competitions and Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

Robot assemble system on Choreonoid• Platform to configure a robot using actuatormodule• Support various series of actuator modules– By writing definitions file
KXR (kondo kagaku)

Dynamixel and Lego block32

Robot assemble system on Choreonoid• Platform to configure a robot using actuator module• Support various series of actuator modules– By writing definitions file
KXR (kondo kagaku)

Dynamixel and Lego block33

Robot assemble system on Choreonoid• Platform to configure a robot using actuator module• Support various series of actuator modules– By writing definitions file
KXR (kondo kagaku)

Dynamixel and Lego block34

Robot assemble system on Choreonoid• Platform to configure a robot using actuator module• Written as a Choreonoid plugin– https://github.com/IRSL-tut/robot_assembler_plugin②

Customized Tool Bar①①

Panel : parts tree②

Panel : parts selection
list for building robot③③

Panel : parts information
35

Robot assemble system on Choreonoid• Platform to configure a robot using actuator module• Verify configured robot in simulation and in real world

Robot assemble system on Choreonoid• Platform to configure a robot using actuator module• Verify configured robot in simulation and in real world
KXR (kondo kagaku)x2

Outline• Robot competitions and Simulation• General information of Choreonoid• Connecting to other system• Development system on Choreonoid• Learning Robot Programming

Education of Robot System using Choreonoid

ル群

ラ設定
ル

Interactive Robot Programming

Robot assembler(Building behavior while verifying the motion of the robot)(Rapid prototyping of configurable robots )

センサ設定
ファイル
シミュレーター

Verify motion by simulator

Verify motion by real robot

Assemble real robot

Education of Robot System using Choreonoid• 1 week experiential learning• For 3rd year undergraduate student– No familiarity with robot programming40

Education of Robot System using Choreonoid• 1 week experiential learning• For 3rd year undergraduate studentx4– No familiarity with robot programming

41

Interactive Robot Programming

42

Learning programming using Jupyter Notebook• Jupyter notebook– Interactive computing– Program execution and take a note
– Graphs and display views are also recorded– Providing the notebooks you made– Browser-based and enable to run in any environment– Available in a various languages

43

Learning programming using Jupyter Notebook• Jupyter notebook– Interactive computing– Program execution and take a note
– Graphs and display views are also recorded– Providing the notebooks you made– Browser-based and enable to run in any environment– Available in a various languages

44

Using Jupyter Notebook with Choreonoid• Implement Jupyter kernel using xeus
– https://github.com/jupyter-xeus/xeus– C++ interface library• Implement Choreonoid Plugin– https://github.com/IRSL-tut/jupyter_plugin– Learning interactive robot programming

45

Using Jupyter Notebook with Choreonoidx246