Human Centric Robotics
Understanding the neuroscience, andcontrol underlying physicalinteractions by human for thedevelopment of next generation
medical and social equipment androbots

Ganesh Gowrishankar
Le Centre
national de larecherche
scientifique1

Machine design for man

machine design is defined by human biomechanics and aesthetics.
However, designs sometimes require to consider ‘human movement
control’ and human behavior.
2Designing for control: golf driver (wood)light, flexible shaft helps store and release energy
shaft impedance is key!

shouldermhCurrent shaft impedance optimized toindividuals height, style and driver headsize
but arm impedance is neglected
Arm impedance determined by-• posture and online arm muscle
activations and dynamic• Human impedance control and adaptationhumans implicitly adapt their posture/muscle
activations depending on the tool and task

mdDeveloping (powered)smart golf drivers withimpedance control
Funded by JST (Japan),
3DFG (Germany)

Designing for control: golf driver (wood)shouldermh

Humans have ability to optimize
impact impedance - the ‘when’ and‘how’ of this process is little
understanding

understand human
impedance control
dynamics to optimize
design of sports equipment
for impact tasks

mdDeveloping (powered)smart golf drivers withimpedance control
Funded by JST (Japan),4
DFG (Germany)

Machine design for man
Design of machines interacting with humans can be
improved by understanding human movement
control

Human-robot
interactions are morechallenging
5

Robots are active interacting agents

Rehabilitation

Medical robotics
Social robotics• How should robots look/behave such that humans
feel comfortable, safe with them?
• benefit physically and psychologically from them?
6Complex human interaction dynamics
Humans interactions change with• Body dynamics, control(golf driver example)• Age• Disease• Cognitive factors– Anxiety– Fear– Mood

• Theory of mind
– What one expects of his partner?
– What interacting partner expects of him?
7

Humans- the ideal partner
Humans consider all thesefactors when interactingwith each other and with
their environment

Future robots need to do
the same!
8How to control human-robot
interactions?
• Solve human black box during
interaction
but humans are too complex!

• Interact utilizing predefinedmodels of human behavior

m4m1m2m4

m3 m5

m6

but we need the models first.

We require human centric robot control!

9Human centric Design and robotics
Human- motor
interactions

Machine-Humaninteractions

Cognitive factors
during interaction• How humans utilize• How control and• How the explicit
various visual and haptic
mechanical dynamics
mechanisms (Theory ofsignals to controlof humans andMind ) affect humanRequires
robotic
design, control
andinteractions
with research inmachinesinteract
robot interactions?
environment and otherwith eachother?
humanmotor
Neuroscience
humans?
• How do factors like• How the design/anxiety and competition• How interactions withaesthetics of robots beaffect motor control
other humans affects
improved?
/learning.
one’s own behavior?
• What robot behavior• How these can be used
• How human behaviors
will help humans
for a benefit ofchange with time, age,adapt faster, feel safe?
robotics?
10
disease ?

11

Robot-human control sharingbalance control= F(a robot, b human)a, b=?

Walking, balancing in a
exoskeleton

Re-learning to walk witha rehab-robot• least interference• compensate for each others mistakes• easy for human, robot to adjust to each other• promote learning between robot-human12Human –human interactions
Explicit and discrete audio, visualand haptic

Continuous physicalExplicit interactions : researched under Social Neuroscience and Theoryof Mind and applied in social robotics
Physical interactions• fundamentally different – implicit, involves multi-body dynamics• Research lacking, neural mechanisms still largely unclear
13

Neuroscience Questions• Does TOM act at low level of motor control?
Such that we have can models/ expectations
during physical interactions?
• Does our CNS recognize an interacting agent
to be human? Does it influence its behavior?
• What are the neural mechanisms underlying
physical interactions?

Interactive motor task design• Relatively simple yet rich
• Enables easy control and modifications of theinteraction characteristics• Enables recording of human movements• 2 robot system that subjects
can hold and manipulate• Visual feedback of cursor
position• Interaction forces generated
by robots
15

Interactive Motor task design
30 cm

target• Task: tracking a randomly moving
target• 80 subjects in total• Series of (1 min) trials; tworandom types

cursorWorkspaceboundary

VIDEO

SDSDSSDDSDSSDSDSDDSD….

single trial (S) and dual connection trial (D)16Single and Dual trials

K= 120 N/m, D=7 Ns/m17

Benefit of interaction

SDSDSSDDSDSSDSDSDDSD….

how the performance of each individual (better and worseperformer) is affected during interaction?
(Dperformance – Sperformance)18

improvement in performance
when connected with partner (m)Benefit of interaction
Partner worsePartner better

Motor Howinteraction
is mutually
better waspartner inbeneficial
single trial for(m)bothindividuals

19Error in single trial after the experiment

Learning: subjects skill improves byinteraction

Solo subject: train for the same time but never get connected to a partner20

Benefit of interaction

What should robots do toenable mutual benefits inrobot-human interactions?

21

How interaction benefits both partners?

1. Is the presence of ‘any’ interaction forces the key?
– Increased attention in connected trials– Increased general arm impedance hence system dynamics– ‘Follow the better’ strategy: eg. subject relaxes when external
disturbance helps task and fights against it if it disturbs task.

•Control experiment1- with recorded connection forces

22

Control1: force playback

recordedforces

Previous subjects

new set of subjects

SDSDSSDDSDSSD…

• magnitude, frequency of forces replicated• target velocity, position dependencies replicated23Control1: force playback

Novice – novice interaction
Force playback (Control 1)

Irrespective of partner performance, interaction leads to performance24
deterioration

How can interaction benefit both partners?
2. Additional feedback key for benefit?
(sensory integration)
If humans are
simply integratingfeedbacks:Vision

Any extra
feedback related
to task error
should help inbetter
performance
Forces from partner
performing the same task

25

How can interaction benefit both partners?
2. Additional feedback key for benefit?
(sensory integration)
Recorded trajectory of previous solo subjects

SDSDSSDDSDSSD…

•Control experiment 2- with recorded subject trajectory 26

Control2: trajectory-playback

Novice – novice interaction
Force playback (Control 1)
Traj. playback (Control2)

Performance better only if partner is better
27

How can interaction benefit both partners?
Whatconsistof?of?
andWhatdoes
doesinteraction
interaction
consistWhat was missing in the control experiments?
partner
subpartner
subPartner action/trajectory + Partner reaction

VisionSelf action+ Self reaction

Humans expect some specific partner reaction during interactive tasks
28Counterchecking humans expect areaction
Question:
What kind of partner reaction do individuals
expect?
Simulation theory of TOM [Csibra 2007, Gallesse 1998]:
Humans understand intentions of others by simulating thecurrent factors with their own system.

Hypothesis:
Humans model the partner to be similar tothemselves- We can change reaction and see if behavior is affected29

Qualitative reaction change :Connection to non-human agent• hand connected to a robot that tracks the target
perfectly• ‘optimal guidance’ but No reaction• similar to that used in many current rehab
applications

30

Expert-connect

Novice – novice interaction
Force playback (Control 1)
Traj. playback (Control2)

Robot –novice interaction

Performance with robot relatively worse than connection with
novice!

31

Qualitative response change :Connection with human expert

Task Expert

The difference in skill-level of the expert should
produce a qualitative change in the partner response..
So connecting to novice should be better thanconnection to expert!!
32Expert-connect

Novice – novice interaction
Force playback (Control 1)
Traj. playback (Control2)

Robot –novice interaction
Expert-novice interaction

Similar human > different human (even expert) > non human agent

Both partner performance and nature effects mutual benefit

33

To Summarize• We designed a human-human interaction experiment to understandhow humans share control, how this affects their behavior• Interaction can enable performanceimprovement during and after
interaction irrespective of partner
performance (mutual benefit)• Mutual benefit present only when
the partner actively tracks target(action) AND provides a appropriate
reaction

What did we learn for the improvement of human-robot
34
interactions?

Towards optimal Human-Robot control
sharing• Robots need to actively perform the task in parallel tohuman• Robots need to ‘consider’ human actions-provide
online reactions• Current robots provide Guidance or support (action)

Can we design a robot behavior with theseproperties?

35

Answering the Neuroscience questions• Does TOM act at low level of motor control? Suchthat we have an internal models of the interaction? Yes• Does our CNS recognize an interacting
agent to be human? Does it influence itsbehavior?

• What is are the neuralmechanisms underlying
physical interactions?

Yes, behavior is bestwith similar human

We do not know how humans
use the partner reaction toimprove performance….
BUT we know what kind ofreaction and behavior enables
mutual benefit!!

Human performing task

Robot performing task and giving
reactions to human behavior andmodifying its own behavior withhuman reactions

We designed a robot behavior with the algorithm37

We get human like benefit with robot!
Human as partner

robot as partnerwith online reaction

without online reaction
to the robot• Human performance consistently better with robot compared toworking alone!
• Human performance alone better AFTER interactive practice with38
robot compared to solo practice of task for the same timeError in single trial after the experiment

Learning: subjects skill improves byinteraction

Solo subject: train for the same time but never get connected to a partner
39

Concluding remarks• Designing for control : Machines/ Robots can be improved
tremendously by considering human movement control• This requires parallel and comprehensive research inrobotics and human motor neuroscience- Human CentricRobotics• Example of Human Centric Robotics :• Psychophysical human- human interaction
experiment• Discovered that humans can show mutual
performance improvement in interactivetasks!
40

Concluding remarks• Example of Human Centric Robotics :• Through experiments we isolated thekey features that make this possible• We implemented this on robots toshow similar (improved) humanbehavior during interactions with arobot
An illustration of the application of
mechanical engineering, robotics, logic tounderstand the human system and improve
human-robot interactions41

Dual Rehabilitation/training• Cheaper , subjects can get more
physiotherapy time and motivation•Sports training can be revolutionized•Occupational training canbe revolutionizedThis may explain what children learn
when they play with other children(rather than with adults)
42

Example works in
Human centric design and robotics
Human motorinteractions• Mechanisms ofphysical interaction
in humans. Ganesh et al.
Nature Sci. Rep. 2013.

• Investigation of thehierarchicalorganization ofhuman movement
learning• Understanding
human behavior
during collisions andimpact tasks.

Machine-Humaninteractions•

Cognitive factors duringinteraction•

Understanding motor control
deterioration due to anxiety.
Funding from Kakenhi Kiban B

Human optimization of hitting
tasks and application to robotics(with TUMunich) Funding
JST(JAPAN) and DFG (Germany)•Understanding how motor
observation affects ones own
motor performance. Ikegami &Improving BMI with machinelearning and neuroscience – withDr. Miyawaki of UEC Tokyo
Kakenhi ‘Houga’.

•Intelligent prosthetics project(with New castle Univ.) Pistohl etal. IEEE Rehab Eng. 2014•••

Optimizing design of golf gloves byunderstanding human motor control.

Ganesh Nature Sci. Rep. 2014

Immediate Tool induced effecton human body representation.
Ganesh et al Nature Com. 2014.

•Embodiment issues in humanoidavatars.
43

Thank you

Human CentricRobotics

Email: g.ganesh@aist.go.jp; gans_gs@hotmail.com
homepage: https://staff.aist.go.jp/g.ganesh/44