Graduate School of Information, Production and Systems (IPS), Waseda University

Introduction of Bio-Robotics andHuman-Mechatronics Laboratory
Bio-Robotics and Human-Mechatronics Laboratory
Graduate School of Information, Production and Systems, Waseda University
http://www.waseda.jp/sem-matsumaru/

Takafumi MATSUMARU
http://www.f.waseda.jp/matsumaru/
matsumaru@waseda.jp

2019.04. Grad School of IPS, Waseda University, Kitakyushu, Japan

Outline Introduction

 Biographical Information
 BR&HM Laboratory

 Past subjects

 HFAMRO: friendly amusing mobile robot
 IDAT: image-projective desktop arm trainer Touch Interaction
 Calligraphy-Stroke Leaning Support System
 Mobile robot companion
 RDB-D Visual SLAM Recent subjects

 Interactive Aerial Projection of 3D Hologram Object
 Pedestrian Navigation Robotic System Reinforcement Learning in Manipulator Control
 Real-time Remote Projection of 3D Human Image Short Range Finger Pointing Interface using Depth Camera
 Brand-Wise Random Picking of 500ml Plastic Bottle using RGB Image

 Closing remarks

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20192Outline
 Introduction

 Biographical Information
 BR&HM Laboratory

 Past subjects

 HFAMRO: friendly amusing mobile robot
 IDAT: image-projective desktop arm trainer Touch Interaction
 Calligraphy-Stroke Leaning Support System
 Mobile robot companion
 RDB-D Visual SLAM Recent subjects

 Interactive Aerial Projection of 3D Hologram Object
 Pedestrian Navigation Robotic System Reinforcement Learning in Manipulator Control
 Real-time Remote Projection of 3D Human Image Short Range Finger Pointing Interface using Depth Camera
 Brand-Wise Random Picking of 500ml Plastic Bottle using RGB Image

 Closing remarks

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20193

Biographical Information
 1985

B.S., Mechanical Engineering, Waseda University“Development of articulated manipulator aiming at force control”(Supervised by late Prof. I.Kato)
 1987 M.S., Mechanical Engineering, Waseda University“Basic theory of multi d.o.f. compliance control on articulated manipulator”
Ichiro Kato
(Supervised by late Prof. I.Kato)
1926-1994
 1987-99 Corporate Research & Development Center, Toshiba Corporation
 Research on robots for specialized operations
 Developing mechatronics systems using robotic technologies
 1998 Ph.D., Mechanical Engineering, Waseda University“Research on structure and control of working robot in a narrow space”(supervised by Prof. S.Sugano) 1999-2010 Associated Professor, Shizuoka University
 Education and Research on Bio-Robotics and Human-Mechatronics
 Invited Professor (2003). LSC - CNRS, Evry France, Visiting Fellow (2002).
Shizuoka Industrial Research Institute, Shizuoka Japan, etc.
 2010- Professor, Waseda University
 Research and Education on Bio-Robotics and Human-Mechatronics
Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20194

Bio-Robotics and Human-Mechatronics Laboratory
 Various themes between human and mechatronics systems(robots and other systems)
 To make mechanical systems more friendly / useful for users
 Developing new functions and producing real-world systems
 Integrating various
Humanknowledge andtechnologies into
Robotics &Bio-Engineeringsystems (selection /Ergonomics (interface, …)
Mechatronics
combination are
Medicine & biology
Mechanics (dynamics, …)(physiology,based on engineering Electronics (circuits, …)
biomimetics,...)
Control (software, …)
Psychology (emotion, ...)
sense)::
 Work on elemental
technologies by
Mechatronics systemourselves if desiredBetter interaction / relationship between human and robots
Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20195

Bio-Robotics and Human-Mechatronics Laboratory
 Career after graduation
 Domestic companies

 Seiko Epson Corp.
 Ascent Robotics Inc.
 KCM Corp.
 Nachi-Fujikoshi Corp.

 Oversee companies

 IDEALSEE [Sichuan, China]
 Hangzhou Hikvision Digital Technology Co.,Ltd. [China] HuaXia Bank [Beijing, China] Huawei Technologies Co.,Ltd. [Shenzhen, China] CT (Computer Technology) Asia Co.,Ltd. [Thailand] China Mobile Ltd.
 Da-Jiang Innovations Science and Technology Co.,Ltd. [China]
 Shanghai Mitsubishi Elevator Co.,Ltd. [China]
 Perfect World Co.,Ltd.

 Foreign university

 Universitas Pendidikan Indonesia [Bandung, Indonesia]
 State University of Malang [Malang, Indonesia]

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20196Outline
 Introduction

 Biographical Information
 BR&HM Laboratory

 Past subjects

 HFAMRO: friendly amusing mobile robot
 IDAT: image-projective desktop arm trainer Touch Interaction
 Calligraphy-Stroke Leaning Support System
 Mobile robot companion
 RDB-D Visual SLAM Recent subjects

 Interactive Aerial Projection of 3D Hologram Object
 Pedestrian Navigation Robotic System Reinforcement Learning in Manipulator Control
 Real-time Remote Projection of 3D Human Image Short Range Finger Pointing Interface using Depth Camera
 Brand-Wise Random Picking of 500ml Plastic Bottle using RGB Image

 Closing remarks

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 20197

Student: Asyifa Imanda SEPTIANA

Interactive Aerial Projection of 3D Hologram Object
 Target/purpose
 Interactive interface of 3D holographic

object image
 “Remote operation” of robotic systems

 3DAHII, three-dimensional aerial

holographic image interface
 Reconstruction and aerial projectionof 3D object jmage
 Pyramid-holography Parabolic reflector
 Interactive instruction
 Direct/real-time interaction with 3D
object image
 Operating procedure Reaction function
 End-tip control of robot manipulator

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., Japan

3DAHII [2017.11] (03:18)
3DAHII@iREX2017[2017.12] (00:13)
Apr. 20198

IPS, Waseda Univ.

Pedestrian Navigation Robotic System How to make mobile robots navigate inStart point

Student: Ahmed FARID

Street crossing

semi-structured outdoor environments
 Without pre-driving recording (i.e. SLAM) Using network-provided maps (i.e.

OpenStreetMaps)
 Contextual awareness for navigation (vision,location)

 Pedestrian path planning Build upon internet map processing approach Image processing on 2D maps to infer
 City blocks, Street crossings, Sidewalk path

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., Japan

Sidewalk Path
End point

Apr. 20199Student: Duc THANReinforcement Learning in Manipulator Control
 Reinforcement learning (RL) Policy S: environment and agent States A: Action of agent R: Reward Example data Supervised learning Unsupervised learning Deep RL(1) Object balancing
 Joint position
 Discrete action space (DAS) approach
before [2018.04]
 Deep Q-Network (DQN) algorithm(00:23)
 Continuous action space (CAS) approachafter [2018.04](03:55)
 Stochastic CAS (SCAS)
 Proximal policy optimization (PPO) algorithm Deterministic CAS (DCAS)
 Deep deterministic policy gradients (DDPG) algorithm

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., Japan(2) Object balancing
 Joint pos/spd
before [2018.04](00:17)after [2018.04](01:41)(3) Object catching Joint pos/spd
learning[2018.05] (02:19)

Apr. 201910

Student: Ami MORIKAWA

Real-time Remote Projection of 3D Human Image Real-time remote projection of 3D human image
 Image capturing
 Image processing
NEWSSEcameraW
N

450mm

500mm
500mm

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanMay. 2018 11

Student: Kazuki HORIUCHI

Short Range Finger Pointing Interface using Depth Camera
 Switch frequently between typing and cursor

Mouse operation

placing in tasks

Operatingplane Web browsing, Editing spread sheet and documents, Display
screenCoding Cursor placing using finger gesture No need of physically-interactive devices
 Easy finger operation
 Familiar with sign language
 Finger pointing is facing to display screen Intuitive

Display
screenAir pointing

Operatingplane Approacha) Ergonomic use of 3D space -- Posture Hand putting on deskb) Recognition accuracy of finger in near range -- Sensor
 Realsense SR300c) Real time operation -- Processing Accurate measurement of fingertip, Fast response
Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 201912

Student: Zhu CHEN

Brand-Wise Random

Picking of 500ml Plastic Bottle using RGB Image 500ml Plastic Bottles
 Condition
 Dry
 In water In water with ice Sensor RBG camera Procedure Segmentation
 Brand recognition
 Masked Image ROI image
 Mask R-CNN + Inception V3

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., JapanApr. 201913

Outline Introduction

 Biographical Information
 BR&HM Laboratory

 Past subjects

 HFAMRO: friendly amusing mobile robot
 IDAT: image-projective desktop arm trainer Touch Interaction
 Calligraphy-Stroke Leaning Support System
 Mobile robot companion
 RDB-D Visual SLAM Recent subjects

 Interactive Aerial Projection of 3D Hologram Object
 Pedestrian Navigation Robotic System Reinforcement Learning in Manipulator Control
 Real-time Remote Projection of 3D Human Image Short Range Finger Pointing Interface using Depth Camera
 Brand-Wise Random Picking of 500ml Plastic Bottle using RGB Image

 Closing remarks

Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., Japan

Apr. 201914

Message Let’s grow up together

developing a new ﬁeld at themeeting ground for people
who have a new way of
thinking and extraordinary
abilities regardless of areas
or aspects.

 領域や分野にとらわれない新しい考え方や，高い能力をもった人々が集う場で，新しい分野を開拓しながら，一緒に成長しましょう．
Bio-robotics & Human-mechatronics Lab., Grad School of IPS, Waseda Univ., Japan

Apr. 201915

Graduate School of Information, Production and Systems (IPS), Waseda University

Introduction of Bio-Robotics andHuman-Mechatronics Laboratory
Bio-Robotics and Human-Mechatronics Laboratory
Graduate School of Information, Production and Systems, Waseda University
http://www.waseda.jp/sem-matsumaru/

Takafumi MATSUMARU
http://www.f.waseda.jp/matsumaru/
matsumaru@waseda.jp

2019.04. Grad School of IPS, Waseda University, Kitakyushu, Japan