Adaptive Impedance Controller for Human-Robot Arbitration based on
Cooperative Differential Game Theory
Paolo Franceschi1,2 Nicola Pedrocchi1 and Manuel Beschi2
Abstract— The problem addressed in this work is the arbitration of the role between a robot and a human during physicalHuman-Robot Interaction, sharing a common task. The system
is modeled as a Cartesian impedance, with two separate external
forces provided by the human and the robot. The problem isthen reformulated as a Cooperative Differential Game, which
possibly has multiple solutions on the Pareto frontier. Finally,
the bargaining problem is addressed by proposing a solution
depending on the interaction force, interpreted as the humanwill to lead or follow. This defines the arbitration law andassigns the role of leader or follower to the robot. Experiments
show the feasibility and capabilities of the proposed control
in managing the human-robot arbitration during a sharedtrajectory following task.
Index Terms— physical Human-Robot Interaction, Role Arbitration, Differential Cooperative Game Theory, Adaptive
Control

I. INTRODUCTION
With the large spread of collaborative robots, collaborative
applications involving a human operator and a robot working
together to achieve a common goal represents the industry’s
most recent trend. In contrast to coexistence (when thehuman and the robot are in the same environment but do
not interact), synchronization (when they work in the sameworkspace, but at different times) and cooperation (whenthey work in the same workspace at the same time, thougheach focuses on separate tasks), collaboration happens when
the human operator and the robot must execute a task together, and the action of the one has immediate consequenceson the other [1].
Collaboration requires communication, which typically
happens through interaction forces, leading to physicalHuman-Robot Interaction (pHRI) [2]. As discussed in [3],impedance control is a ubiquitous technique to manage thepHRI. Initially proposed by Hogan [4], impedance control
has found an increasing interest due to numerous researchworks aiming at making it adaptive, in a way such that the
robot behavior adapts to the human one. Such adaptation
happens mainly in two manners, adaptation of the impedance
set-point and adaptation of the mass-spring-damper parameters. To the first category belong [5], and [6], where thedamping is made variable since it allows fast/slow motion
depending on its value. Examples of the second category
1 Paolo Franceschi and Nicola Pedrocchi are with the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing of
the National Research Council of Italy (CNR-STIIMA), via Alfonso Corti
12, 20133, Milano, Italy. {name.surname}@stiima.cnr.it
2 Paolo Franceschi and Manuel Beschi are with the Università di Brescia,
Dipartimento di Ingegneria Meccanica ed Industriale, via Branze 38, 25123,
Brescia, Italy manuel.beschi@unibs.it

can be found in [7] and [8]. Moreover, hybrid controllers
exist, which modify both impedance parameters and setpoint. This happens in [9], [10], where the updated law isdefined by fuzzy logic, in [11], where the parameters areonline updated via Model-Based Reinforcement Learning,
and in [12], where neural networks (NN) updates the desired position, and the impedance parameters are updated tomaintain stability.
In the previous examples, and many other works, thehuman-robot interaction is described by a static leaderfollower paradigm, where the human gives input, and therobot responds consequently, but vice versa never happens.
A step ahead is to consider Human-Robot Arbitration, which
can be defined as the mechanism that assigns task control
to either the human or the robot [13]. Such an issue isaddressed, with different approaches, in [14] [15] [16] [17]As discussed in [18], game theory provides useful tools
to analyze complex interactive behaviors involving multiple
agents. It provides mathematical models (cooperative, noncooperative, multi-stages, etc.) of strategic interaction among
rational decision-makers and provides the players with ”optimal” policies to minimize their objectives, taking into
account interaction. Thanks to its capabilities of describing
interactions in an ”optimal” way, controllers for the pHRI
based on game-theoretic formulations are investigated in
the literature. In [19], [20], and similarly, in [21] for theshared control with an exoskeleton, the continuous role
adaptation is investigated for a Differential Non-Cooperative
game. In these works, the concept of Nash Equilibrium isused to update the robot cost function according to theinteraction force. The result is a variable impedance control
with damping and stiffness updated online. An extensionof such approaches, also involving position update, can befound in [22]. The same differential non-cooperative gametheoretic problem is addressed in [23], [24], with a different
approach for the solution based on policy iteration. Finally, ageneral framework for differential game-theoretic modelingof Human-Robot interaction is presented in [25] for the two
agents game and extended to multiple agents in [26]. An
observer is designed to understand each other’s control laws
in these works, and different behavior can be addressed.
The previous works show that a game-theoretic descriptionof the human-robot interaction can provide optimal behavior
in a non-cooperative framework. Despite this, Nash equilibria (i.e. the solutions of the non-cooperative games) arefrequently not Pareto optimal. Thus, cooperation can often
improve payoffs to all players [27]. Hence, in this paper,
Cooperative Game Theory (CGT) is used to describe theinteraction model, and Pareto solutions are sought. Cooperation, indeed, can improve the cost of each agent byagreeing. Hence, in a cooperative framework, each agent
has an incentive to cooperate since its cost will be lowered compared with the non-cooperative solution. Moreover,a cooperative description of the task allows optimal comanipulation of objects. Finally, as described in this work,
Cooperative Game Theory allows for easy role arbitration,
keeping the optimal behavior of the robot. To the best of
the authors’ knowledge, no Cooperative Differential GameTheoretic frameworks have been used yet to describe HRI.
This work aims to show the feasibility of the CGT frameworkwith an application to human-robot arbitration.
II. METHODThis section presents a system modeled as a Cartesian
impedance with two separate external forces provided by thehuman and the robot. The problem is then reformulated as a
Differential Cooperative Game, with solutions lying on the
Pareto frontier. The bargaining problem is addressed with aformulation depending on the interaction force to define thearbitration law. Finally, it is shown that such a model results
in an adaptive impedance controller with variable stiffnessand damping.
A. System modeling
Working in the Cartesian space is more intuitive andnatural for the human operator; hence the desired robot
motion at the end-effector is implemented as an impedancemodel in the Cartesian space:
Mi ẍ(t) + Di ẋ(t) + Ki (x − x0 )(t) = uh (t) + ur (t)(1)

where Mi , Di and Ki ∈ R6×6 are the desired inertia, damping
and stiffness matrices, respectively, ẍ(t), ẋ(t) and x(t) ∈ R6
are the Cartesian accelerations, velocities and positions at theend-effector, x0 the equilibrium position of the virtual spring,and uh (t) ∈ R6 and ur (t) ∈ R6 represent the human and robot
effort applied to the system. The Cartesian coordinates in x
are defined according to [28], with the vector x = [pT φ T ]T
where pT are the position coordinates and φ T the set of
Euler angles that defines the rotation matrix describing theend-effector orientation. With a little abuse of notation, toease the reading, with ẋ = [ ṗT ω T ]T is indicated the vector
containing the linear and angular velocities.
After some reformulation, (1) can be rewritten in a linearized state-space formulation around the working point asż = Az + Bh uh + Br ur

(2)

T
12
where
 6×6z = [x − x0 ẋ] ∈ R is the state space 6×6 vector, A =0Ja0
12 × 6
12
×
6
, Bh
= Br
=
, with 06×6
−Mi−1 Ki −Mi−1 Di
Mi−1denoting a 6 × 6 zero matrix and Ja the analytical Jacobianmatrix, with the dimensions of the considered Cartesiancomponents.
Since robot controllers typically accept reference positions
or velocities in the joint space as a control input, it is worth

converting the reference velocities from the Cartesian space
to the joint space.
Given the reference velocity in the Cartesian space, thefollowing relation allows obtaining reference velocities in
the joint spaceq̇re f (t) = J(q)+ ẋ(t)(3)
where q̇re f (t) ∈ Rn , where n represents the number of joints,
are the reference velocities in the joint space, J(q)+ is
the pseudoinverse of the geometric Jacobian matrix. Simple integration allows commanding joint positions insteadof velocities to the robot. Considering that today’s robots
have excellent tracking performance in the frequency range
excitable by the operator, in this work, hypothesis q̇ ' q̇re f
is assumed to hold.
B. Differential Cooperative Game Theoretic modeling
Rewriting the model (1) as (2), allows to include it in a
Differential Cooperative Game Theory (DCGT) framework,
as described in [29].
The system in (2) can be further rewritten asż = Az + Bu

(4)

with A12 × 12 defined as before, B12 × 12 = [Bh Br ] and u =[uh ur ]T .
The human and the robot can be seen as two agents, each
one with the objective to minimize its quadratic cost function,
defined as
Z
∞

Jh =
and

0

(z Qh z + uh Rh uh ) dt

(5)(z Qr z + ur Rr ur ) dt

(6)

Z ∞

Jr =

0

where Jh and Jr are the cost that the human and the robot
incur, Qh ∈ R2n×2n and Qr ∈ R2n×2n weights on the state andRh ∈ R6×6 and Rr ∈ R6×6 weights on the control input. The
human weights Qh and Rh are unknown and cannot be set
as the robot parameters Qr and Rr . A reasonable estimateof the human parameters, Q̂h and R̂h , can be obtained withinverse model control similarly to [30], [31]. The problem
here presented can be summarized as finding the ur and uhof (5) and (6) subject to the system dynamics defined in(4), and game theory provides a framework for finding such
solutions.
Given the system in 4, the problem reduces to find the control action u as a composition of feedback and feedforward
terms, defined asu = −K f b z + K f f zre f(7)

where the apices f b and f f denote feedback and feedforward
gain matrices, respectively.
While in Non-cooperative Game Theory, the solutions
are represented by the Nash equilibria, in Linear Quadratic
Cooperative Game Theory, each feasible solution lies on the
Pareto frontier (see [29] for definitions).

The cost to be minimized from the two players together
is defined as
Jα = α Jh + (1 − α) Jr =Z ∞0(z Qα z + u Rα u) dt

(8)

where
Qα = α Q̂h + (1 − α) Qr
and
Rα =

α R̂h
06 × 6, and α ∈ (0, 1) represents the weight06 × 6 (1 − α) Rreach player’s cost has in the overall cost.
To find all cooperative solutions for the feedback linearquadratic game, one has to solve a regular Linear Quadratic
optimal control problem that depends on a parameter α. The
solution is given by the matrix P, solution of the infinite
horizon Continuous Algebraic Riccati Equation (CARE),
given byT
AT P + PA − PBR−1
α B P + Qα = 0(9)The feedback gain matrix is defined asT
K f b = R−1
α B P

(10)

and the feedback control actions are given byu f b = −K f b z(t)(11)

From (11) the feedback control terms demanded from thehuman and the robot can be computed as" #ū f bfb(12)
u = hf burwhere ū f b denotes the nominal effort demanded to thehuman. A feedforward term is added to allow trajectory
tracking.
The feedforward gain matrix is defined as−1  A
B
0
K f f = [K f b I]
(13)
C
D
I
with A and B as in (4), and C and D output and feedthroughmatrices of the state-space system description, respectively.
The feedforward term results inu f f = K f f zre f .

(14)

Since the system matrix is typically not square in thisformulation, the pseudoinverse can be used instead of theinverse. This results in a vector of dimension 12, and its
components are" #ū f fff(15)
u = hf furwith the superscripts h and r denoting the human and robot
contributions.
Finally, the total control input results in u = u f f + u f b , andthe control action of the robot can be computed as

C. Bargaining
In a cooperative environment, it is rational to consider
the set of Pareto solutions. Since there are infinity Paretooptimal solutions, we enter the bargaining theory arena to
decide which is the most effective one. The bargaining
problem is how to define the appropriate α. In classicalbargaining solutions (Nash, Kalay-Smorodinsky, egalitarian),
α is defined such that the cost of all players decreases
compared to their non-cooperative cost.
In this work, the two players bargain on who leads andfollows the task; in this sense, one can accept a higher cost if
he is the follower. Hence, α is used as a weighting factor tomove the control authority from the robot to the human andvice-versa. Indeed, for high values of α, the robot cost tends
to disappear from the overall cost computation Jα . Hence itwill be less costly for the robot to put much effort into thesystem, becoming the leader, resulting in a higher cost for thehuman. On the contrary, for low values of α, the robot cost
increases, and its control input will be dramatically reduced,
leading the control authority to the human. Each situation in
between represents a cooperative solution where the control
authority is shared appropriately.
The selection of the weight parameter α depends on theforce applied by the human and is processed by the sigmoid
function:
a
(17)
α =d−
1 + e−b(kuh k−c)where the constant parameters a, b, c, d are used to shape thefunction properly. In particular a defines the height of thefunction, b the width of the transition phase, c represents anoffset in the x direction and d is an offset in the y direction,
moreover the negative sign after d means that α is decreasing
as kuh k increases.
D. The control law as a variable impedance
Looking at (12), can be divided into two components andthe control inputs demanded from the human and the robot
are computed as" #"#ūhf b
Khf b
=−
z(t)
(18)
urf b
Krf b
where Kh ∈ R6×m and Kr ∈ R6×m are the components of thematrix K defined in (10). Looking at the robot control input,
the feedback part can be rewritten asx − x0(19)
urf b = −Krf bẋ
fb
fb
Given Krf b = [Kr,kKr,d], the two components associatedwith the stiffness and damping of the variable impedancecooperative system are defined as
fb
fb
Kimp
= Kr,k

(20)

fb
fb
Dimp
= Kr,d(21)

andur = urf f + urf b(16)

A similar computation can be done for the feedforward
terms, resulting in
urf f = Krf f xre f(22)

where Krf f represents the components relative to the robotof the matrix (13) andff
Kimp= Krf f(23)

The robot control input results infb
fb
ff
ur = −Dimpẋ − Kimp(x − x0 ) + Kimp
xre f

(24)

Substituting (24) into (1) results infb
Mi ẍ(t) + (Di + Dimp)ẋ(t)+
fb
ff+(Ki + Kimp
) (x(t) − x0 (t)) − Kimp
xre f (t) = uh (t)(25)

Because varying α varies matrix P and matrix K accordingly,(25) represents a variable impedance system subject to thehuman force, with the values of Dimp and Kimp updated
according to the human will to lead or follow, detected byforce applied.
III. E XPERIMENTS
An experiment is designed to test the proposed control
method for sharing control authority in human-robot collaboration. The robot has to follow a planar circular trajectory,
while the human has a different path to follow, which
partially overlaps with the robot one. The nominal robot
trajectory is defined as−ρ sin(ω t)
xre f ,r (t) =(26)−ρ cos(ω t)where ρ = 0.2 is the radius of the circumference and ω isthe angular velocity.
The human desired trajectory is piecewise-defined as
x"re f ,r (t), t0 < t <


# t1



−ρ sin(ω t) x + x2 −x1 (t − t ) , t1 < t < t2
0
1
t2 −t1#(27)
xre f ,h (t) = "

−ρ
sin(ω
t)



 x + x3 −x2 (t − t ) , t2 < t < t321
t3 −t2
xre f ,r (t), t3 < twith t0 = 0s, t1 = 3s, t2 = 5s, t3 = 7s.
The nominal robot trajectory, the desired human trajectory,
and the current position are shown in real-time on a screen,
and the human has the goal to make the current trajectory
as close as possible to the human desired trajectory. Figure
1 shows the experimental setup.
In Figure 2 the nominal trajectory of the robot (dashed
red line), the desired trajectory of the human (dashed yellowline), and the actual robot end-effector positions (solid blueline) are shown.
To test and compare the proposed approach, the following
indices are computed:•

Trajectory following error, measured as
Etrack =Z TendTstartxre f ,h (t) − x(t) dt(28)

As low the Etrack is, as close the actual trajectory is tothe nominal one• Geometrical following error, measured as
EXcorr =Z TendTstartxre f ,h (t + δt) − x(t) dt(29)

where δt represents the time delay of the actual trajectory with respect to the nominal one, computed byapplying cross-correlation between the nominal trajectory of the human and the measured one. In this way,
it is possible to compare the capability of the human to
track the geometrical reference. As low the EXcorr is, as
close the actual trajectory is to the geometrical one• Measured interaction force, measured as
F=

Z Tendk f (t)k dt(30)

TstartAs low the F is, as less effort the human has to put inthe cooperative task
• Mechanical work, measured as
W =Z Tend~f (t) · d~S dt(31)

Tstart

The lower the W is, the less energy the human has toput into the cooperative task
The goal of the proposed controller is to allow smoothinteraction and leader-follower transition with the human,
reducing the effort required and allowing good trajectory
tracking. Hence the goal is to minimize the indexes defined
above together.
As a comparison, three different types of impedancecontrollers are used. The CGT controller is compared withan LQR controller, which can be seen as the CGT with
α = 1 always, and two classical impedance controllers withdifferent values of stiffness K. One has high stiffness (HIC),
and the other has a low value of stiffness (LIC). For the

Fig. 1: The experimental setup, showing the application andthe monitor displaying the three trajectories: magenta - robot
desired trajectory, blue - human desired trajectory, green actual trajectory.

two classical impedance controllers (HIC, LIC), the value√ ofthe damping to achieve the critical damping D = 2Dcr KM.
For the LQR and CGT controllers, the values of damping
and stiffness depend on the matrices Q̂h , R̂h , Qr and Rr ,defined as Q̂h = diag(50, 50, 1, 1), R̂h = diag(10, 10, 0, 0),
Qr = diag(20, 20, 1, 1) and Rr = diag(1, 1, 0.01, 0.01) for
the CGT case, Rr = diag(1, 1, 0.1, 0.1) for the LQR case,otherwise the stiffness was too high and too much force was
required to barely move the robot. The Q̂h and R̂h used
are average values. The mass matrix for all the cases isdefined as M = diag(10, 10), the base damping and stiffness
for the CGT and LQR cases are K = diag(0, 0) and D =diag(40, 40). The mass-spring-damper parameters used orcomputed for the experiments are presented in Table I, while
Figure 3 shows the changing parameters during the task.
For the CGT case, the value of α is bounded such that
0.01 ≤ α ≤ 0.99, and the sigmoid parameters are chosen to
be a = 0.98, b = 0.7, c = 7, d = 0.99.
TABLE I: The mass, spring and damping parameters used
for the experiments.
K
D
D/Dcr
M

CGT
55 ÷ 550
53 ÷ 135
1.13 ÷ 0.92
10

HIC
100
57
0.9
10

LIC
20
25.5
0.9
10

LQR
222
84
0.89
10

IV. R ESULTS
In the test campaign, five subjects (age 30 ± 1) are asked
to perform the task five times for each controller. Before
starting the test, the subjects are allowed to practice as longas needed to feel confident with the current controller, thenfive trials in a row with the same controller are recorded. This
procedure is used for each of the four controllers, selected
randomly for each subject.

trajectory comparison

0.2

0.15

0.1

0.05

A UR5 robot is used, controlled by joint velocities. (2) isused to compute the Cartesian reference positions, and (3)provides the robot joint velocity tracking controller with thereference values. A Robotiq FT 300, mounted on the robot
tip, is used to measure the force applied by the human.
A t-test compares the CGT controller in pairs with all theothers for each of the computed indexes. The t-test allows
defining the statistical relevance of the computed values.
The results are in Figure 4, with the mean and standard
deviations, as well as the relative p-values.
The tracking error, computed as in (28), is shown in Figure
4a. The CGT controller shows the best performance, andthe t-test assesses the statistical relevance of the measured
values. This result can be explained by the robot’s highstiffness/low damping ratio while the robot is leading and thelow stiffness/high damping ratio while the human is leading.
On the one hand, the high stiffness/low damping ratio allows
fast and reliable trajectory tracking, as well as fast recovery
after a trajectory modification (i.e. the robot is faster ingetting back to its nominal trajectory after the human leaves
control). On the other hand, a low stiffness/high damping
ratio allows the human to move the robot smoothly andprecisely while leading. Compared to the LIC, constant highstiffness values allow better trajectory tracking in HIC and
LQR.
The geometrical error, computed as in (29) is shown in 4b.
The t-test shows that no significant differences between thefour controllers can be appreciated. This shows that the tasks
are executed correctly, and the comparison of the controllers
for the computed indexes is fair. Indeed, this shows thatsimilar results can be obtained in terms of geometrical pathfollowing with all the controllers.
The force evaluation is presented in 4c. Since the exchanged force mainly depends on the robot stiffness ( measured as f = K(xre f ,r − x)), the lower the stiffness is, thelower the exchanged force. This effect results in the LIC
with the best performance concerning the required force.
Despite this, the CGT controller shows better performancewith respect to LQR and HIC because when interaction
happens, its stiffness lowers. Notably, performances similar
to the LIC can be obtained by a different tuning of the valuesof matrices Qr , Rr .
Finally, similar considerations can be derived for themechanical work shown in Figure 4d. Indeed, dependingon the interaction force, similar behavior is observed withrespect to the force results.
V. D ISCUSSION

0

current poserobot reference
human reference
-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Fig. 2: Comparison between the nominal trajectory of therobot, the nominal trajectory of the human and the measured

The results show that being equal to the capability of
the four controllers in following the nominal geometricalpath under human-robot cooperation, the CGT controller
outperforms the others in trajectory following. Moreover,even if the LIC requires less interaction force to accomplish
the task, it shows the lowest capability in trajectory tracking,and the CGT also requires low interaction forces. As aremark, a similar interaction force can be obtained for the
CGT controller by tuning the parameters differently.

1

140

600

0.9

1.15

130

500

1.1

120

K [Nm]

0.6
0.5

D [Nm/s]

400

0.7

300
200

0.4
0.3

110

1.05

C/C cr

0.8

100
90

70
0.9

100
0.2
0.1

1
0.95

80

60
0

5

10

15

0

0

5

10

15

50

0

5

10

15

0.85

0

5

10

15

t [s]

t [s]

t [s]

t [s]

(a) Arbitration parameter α(b) Variable stiffness(c) Variable damping(d) Variable critical damping

Fig. 3: Variable parameters for the CGT
1.5

0.5

150

6

100

4

0.5

0

CGT HIC

LIC LQR(a) Tracking error
p = 0.0081, <0.0001, 0.1316

W [J]

0.3

F [N]

||e|| [10 -2 m]||e|| [10 -2 m]

0.4

1

0.2

50

2

0.1
0

CGT HIC

LIC LQR(b) X-correlated error
p = 0.2204, 0.0261, 0.8767

0

CGT HIC

LIC LQR(c) Forcep = <0.001, <0.0001, <0.0001

0

CGT HICLIC

LQR(d) Mechanical workp = 0.0093, 0.0093, <0.0001

Fig. 4: Indexes comparison
Interestingly, from Table I and Figure 3d, it can be notedthat the critical damping value varies from lower than criticalto higher than critical, according to the role adaptation. Thiseffect allows the robot to have a more damped behavior
when the human is leading, allowing smooth motion, while alow damping ratio allows better and faster trajectory tracking
when the robot is leading.

application can be for rehabilitation purposes, where therobot behavior is adjusted by tuning the parameter α, to makethe robot contribution high when the patient is barely able
to perform a task and gradually lower the robot contribution
as the patient starts to recover some motion capabilities.

Some limitations of the proposed control can be identified.
First, the human control objective (i.e. the human costfunction parameters) can only be estimated offline with someinverse techniques. Future works will address online estimation techniques for the human control objective estimation toadapt the robot behavior accordingly. Indeed, it is reasonable
to suppose that the human may change his cost function
parameters due to fatigue by repeating a task all day long.
Secondly, many parameters have to be defined arbitrarily.
In particular, it is difficult to predict the variable damping
and stiffness values. Hence, it may be interesting to definesome tuning rules according to the required performances.
Moreover, the arbitration law introduced in this work is a
simple function of the force, but it can be improved andmodified according to other sensing capabilities and different
human intention estimations. Future work will address amore complex and improved arbitration law involving visualfeedback and human intention estimation, along with force
feedback. In such a way, with a more complex sensor-fusion,a more precise estimate of the human will to lead or follow
can be obtained. Some future applications of this framework
will involve the co-manipulation and the co-transportation ofheavy objects, as well as lightweight parts and deformableobjects as composite materials plies. Another interestingThis work proposes a framework for physical HumanRobot interaction based on a Cooperative Game theory formulation. The bargaining problem is addressed with the law
definition for the leader-follower role arbitration. The results
show the capability of the proposed method in managing
the leader-follower transition continuously and show thatwith the proposed controller, high performance in tracking a
trajectory, different from the nominal one of the robot, canbe achieved through role arbitration.
In conclusion, the proposed framework shows good capabilities in describing a human-robot cooperative task. It allows easy integration of different modules (human cost function identification, human intention identification) and easy
implementation of different robot behaviors (leader-follower
adaptation, co-manipulation, co-transportation), which will
be investigated in future works. Future works will consider
different solutions to the bargaining problem to provide
the robot with the proper control input, possibly involving
constraints on the maximum allowed input.

VI. C ONCLUSIONACKNOWLEDGMENT
This project has received funding from the European
Union’s Horizon 2020 research and innovation program
under grant agreement No 101006732 (Drapebot).

R EFERENCES[1] E. Matheson, R. Minto, E. G. G. Zampieri, M. Faccio, and G. Rosati,“Human–robot collaboration in manufacturing applications: A
review,” Robotics, vol. 8, no. 4, 2019. [Online]. Available:
https://www.mdpi.com/2218-6581/8/4/100[2] S. Haddadin and E. Croft, Physical Human–Robot Interaction. Cham:
Springer International Publishing, 2016, pp. 1835–1874.
[3] A. De Santis, B. Siciliano, A. De Luca, and A. Bicchi, “An atlas ofphysical human–robot interaction,” Mechanism and Machine Theory,
vol. 43, no. 3, pp. 253–270, 2008.
[4] N. Hogan, “Impedance Control: An Approach to Manipulation: Part
I—Theory,” Journal of Dynamic Systems, Measurement, and Control,
vol. 107, no. 1, pp. 1–7, 03 1985.
[5] J. Dong, J. Xu, Q. Zhou, and S. Hu, “Physical human–robot interaction
force control method based on adaptive variable impedance,” Journalof the Franklin Institute, vol. 357, no. 12, pp. 7864–7878, 2020.
[6] F. Ficuciello, L. Villani, and B. Siciliano, “Variable impedance controlof redundant manipulators for intuitive human–robot physical interaction,” IEEE Transactions on Robotics, vol. 31, no. 4, pp. 850–863,
2015.
[7] D. P. Losey and M. K. O’Malley, “Trajectory deformations fromphysical human–robot interaction,” IEEE Transactions on Robotics,
vol. 34, no. 1, pp. 126–138, 2018.
[8] S. S. Ge, Y. Li, and H. He, “Neural-network-based human intention
estimation for physical human-robot interaction,” in 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence(URAI), 2011, pp. 390–395.
[9] L. Roveda, S. Haghshenas, M. Caimmi, N. Pedrocchi, and L. Molinari Tosatti, “Assisting operators in heavy industrial tasks: On thedesign of an optimized cooperative impedance fuzzy-controller withembedded safety rules,” Frontiers in Robotics and AI, vol. 6, p. 75,
2019.
[10] L. Roveda, S. Haghshenas, A. Prini, T. Dinon, N. Pedrocchi,
F. Braghin, and L. M. Tosatti, “Fuzzy impedance control for enhancing
capabilities of humans in onerous tasks execution,” in 2018 15th
International Conference on Ubiquitous Robots (UR), 2018, pp. 406–
411.
[11] L. Roveda, J. Maskani, P. Franceschi, A. Abdi, F. Braghin, L. Molinari Tosatti, and N. Pedrocchi, “Model-based reinforcement learning
variable impedance control for human-robot collaboration,” Journalof Intelligent & Robotic Systems, vol. 100, no. 2, pp. 417–433, Nov
2020.
[12] Y. Li and S. S. Ge, “Human–robot collaboration based on motion
intention estimation,” IEEE/ASME Transactions on Mechatronics,
vol. 19, no. 3, pp. 1007–1014, 2014.
[13] D. P. Losey, C. G. McDonald, E. Battaglia, and M. K. O’Malley, “A
Review of Intent Detection, Arbitration, and Communication Aspectsof Shared Control for Physical Human–Robot Interaction,” Applied
Mechanics Reviews, vol. 70, no. 1, 02 2018, 010804.
[14] L. Peternel, N. Tsagarakis, D. Caldwell, and A. Ajoudani,“Robot adaptation to human physical fatigue in human–robot comanipulation,” Autonomous Robots, vol. 42, no. 5, pp. 1011–1021,
Jun 2018.
[15] R. J. Ansari and Y. Karayiannidis, “Task-based role adaptation forhuman-robot cooperative object handling,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3592–3598, 2021.
[16] B. Nemec, N. Likar, A. Gams, and A. Ude, “Human robot cooperation
with compliance adaptation along the motion trajectory,” Autonomous
Robots, vol. 42, no. 5, pp. 1023–1035, Jun 2018.
[17] X. Li, Y. Pan, G. Chen, and H. Yu, “Adaptive human–robot interaction
control for robots driven by series elastic actuators,” IEEE Transactions on Robotics, vol. 33, no. 1, pp. 169–182, 2017.
[18] N. Jarrassé, T. Charalambous, and E. Burdet, “A framework to
describe, analyze and generate interactive motor behaviors,” PLOS
ONE, vol. 7, no. 11, pp. 1–13, 11 2012.
[19] Y. Li, K. P. Tee, W. L. Chan, R. Yan, Y. Chua, and D. K. Limbu,“Continuous role adaptation for human–robot shared control,” IEEE
Transactions on Robotics, vol. 31, no. 3, pp. 672–681, 2015.
[20] Y. Li, K. P. Tee, W. L. Chan, R. Yan, Y. Chua, and D. Limbu, “Roleadaptation of human and robot in collaborative tasks,” in 2015 IEEE

International Conference on Robotics and Automation (ICRA), 2015,
pp. 5602–5607.
[21] W. Bi, X. Wu, Y. Liu, and Z. Li, “Role adaptation and force, impedance
learning for physical human-robot interaction,” in 2019 IEEE 4thInternational Conference on Advanced Robotics and Mechatronics(ICARM), 2019, pp. 111–117.
[22] Y. Li, C. Yang, and W. He, “Towards coordination in human-robot interaction by adaptation of robot’s cost function,” in 2016 International
Conference on Advanced Robotics and Mechatronics (ICARM), 2016,
pp. 254–259.
[23] Y. Li, K. P. Tee, R. Yan, W. L. Chan, Y. Wu, and D. K. Limbu,“Adaptive optimal control for coordination in physical human-robot
interaction,” in 2015 IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), 2015, pp. 20–25.
[24] Y. Li, K. P. Tee, R. Yan, W. L. Chan, and Y. Wu, “A framework ofhuman–robot coordination based on game theory and policy iteration,”
IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1408–1418, 2016.
[25] Y. Li, G. Carboni, F. Gonzalez, D. Campolo, and E. Burdet, “Differential game theory for versatile physical human–robot interaction,”
Nature Machine Intelligence, vol. 1, no. 1, pp. 36–43, Jan 2019.
[26] R. Zou, Y. Liu, J. Zhao, and H. Cai, “A framework forhuman-robot-human physical interaction based on n-player gametheory,” Sensors, vol. 20, no. 17, 2020. [Online]. Available:
https://www.mdpi.com/1424-8220/20/17/5005[27] A. Seierstad, “Pareto improvements of nash equilibria in differential
games,” Dynamic Games and Applications, vol. 4, 01 2011.
[28] B. Siciliano and L. Villani, “Robot force control,” 2000.
[29] J. Engwerda, LQ dynamic optimization and differential games. John
Wiley & Sons, 2005.
[30] S. Rothfuß, J. Inga, F. Köpf, M. Flad, and S. Hohmann,“Inverse optimal control for identification in non-cooperative
differential games,” IFAC-PapersOnLine, vol. 50, no. 1, pp.
14 909–14 915, 2017, 20th IFAC World Congress. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S2405896317334602[31] F. Köpf, J. Inga, S. Rothfuß, M. Flad, and S. Hohmann, “Inverse
reinforcement learning for identification in linear-quadratic dynamic
games,” IFAC-PapersOnLine, vol. 50, no. 1, pp. 14 902–14 908, 2017,
20th IFAC World Congress.