Topological Synergies for Grasp Transfer
Dmitry Zarubin1 , Florian T. Pokorny2 , Dan Song2 , Marc Toussaint1 and Danica Kragic2

Abstract— In this contribution, we propose a novel approach
towards representing physically stable grasps which enables
us to transfer grasps between different hand kinematics. We
use a low dimensional topologically inspired coordinate representation which we call topological synergies, and which is
motivated by the topological notion of winding numbers. We
address the transfer problem as a stochastic optimization task
and carry out motion planning in our topologically inspired
coordinates using the Approximate Inference Control (AICO)
framework. This perspective allows us to compute not only the
final grasp itself, but also a trajectory in configuration spaceleading to it. We evaluate our approach using the simulation
framework PhysX. The presented experiments, which develop
further recent attempts to use topologically inspired coordinatesin robotics, demonstrate that our approach makes it possible
to transfer a large percentage of grasps between a simulated
human hand and a 3-finger Schunk hand.

Fig. 1. Demonstrated stable human grasp (left) and transfered grasp (right)
applying our topological approach

I. INTRODUCTION
Humans are able to easily imitate another human grasping
and manipulating an object. Furthermore, even if one or twoof our fingers are incapacitated due to a bandage or injury, weare still able to carry out most tasks as before. While muchresearch in the last few decades has concentrated on robot
grasping, the question of how to efficiently represent a stablegrasp and how to transfer such grasps between differing
hand kinematics remains an open area of research [3]. The
work [1] suggested that humans use the concept of virtual
fingers in order to control hand movements during grasp
execution which supports the hypothesis that there might
be a non-trivial low-dimensional representation that is usedby humans to control, observe and reproduce grasps. In asimilar vein, the framework of postural synergies [16], [15]and force synergies [17], where a linear subspace is used for
representing grasp postures and forces respectively, tries tofind such a low-dimensional representation.
A classical and popular approach towards finding good
grasps is to proceed via a force closure analysis which takes
into account the contact positions and normals between thehand’s surface and the object being grasped. While thisapproach is theoretically sound, it can be problematic inpractice since the robot’s sensor input might be noisy, making
object pose and normal estimation difficult. Furthermore,even if perfect knowledge of the object is assumed, computing stable grasps often still relies on a computationallyexpensive brute force search. Simulation software packages
1 Machine Learning and Robotics Lab, Universität Stuttgart, Stuttgart,
Germany dmitry.zarubin@bccn-berlin.de
2 Computer Vision and Active Perception Lab, Centre for Autonomous
Systems, School of Computer Science and Communication, KTH RoyalInstitute of Technology, Stockholm, Sweden
This work was funded by the EU FP7 project TOMSY (IST-FP7-270436)

such as GraspIT [11], for example, can attempt to findforce-closed grasps by sampling hand positions on a spheresurrounding the object, approaching the object until contact
occurs and then closing the hand as much as possible, butusing such a ‘blind’ method requires a lot of tests that mighttake too long to carry out on a real robot.
In this workshop paper, we argue that some of the problems surrounding this type of analysis are caused by the
representation of the state space. If one works, for example,
with the joint space coordinates directly, there exists noobvious way of transferring a grasp to a new kinematic hand
structure with different joint lenghts or a different number ofjoints. In this work, we hence begin to explore an alternative– topologically motivated – coordinate representation which
can be transferred between different hand kinematics andwhich describes not just the hand’s state space but also theinteraction between the hand and object. Our work falls intoa class of recent methods in robotics and computer graphics
inspired by concepts from topology such as [8], [19], [12].
In summary, this work makes the following contributions:a) We evaluate a novel low-dimensional topologically motivated grasp representation for the purpose of grasp
transfer which describes how much a hand is wrappedaround an object.
b) We use this representation with a planning framework
based on Approximate Inference Control (AICO) [18]in order to transfer grasps between different hands.
c) We demonstrate the success of our motion synthesis
method in simulation by transferring a set of grasps
from a human hand to the Schunk robot hand.
d) We evaluate the transferred grasps’ stability using arealistic physics simulation carried out using the PhysX
simulation software.

II. RELATED WORK
A posture Ph of a human hand has a state space with morethan 20 degrees of freedom [4], [5], [16], while a Schunkrobot hand has just 7 degrees of freedom. The questionof how to transfer a human hand configuration to a robot
hand configuration is hence highly non-trivial and somewhat
ill-defined since there exists no natural comparison metric.
In a programming by demonstration context [7], [10] one
would however like to transfer a demonstration from a humansubject to the robot in order to teach a robot how to carryout various tasks. Similarly, in the context of teleoperation[13] it is highly desirable to be able to transfer grasps from ahuman to a robotic hand. Several approaches to the transfer
problem have been explored. The work [14] discusses threebroad methods for transfer given by a) linear joint mapping- which is applicable if the robot’s hand kinematics are very
similar to those of the human hand, b) pose mapping - using
least squares fitting and c) synergistic fingertip mapping. In
more recent work, approaches related to the notion of virtual
fingers have been explored in [6], [10], where a subset of thefingers of the human hand are manually mapped to one ormore fingers of a robot hand.
One of the key features that the virtual finger, fingertip
and synergy approaches share is that they attempt to first
reduce the number of dimensions needed to describe ahand pose to a common minimum. Similarly, in the caseof postural synergies [17], [2], a lower dimensional linear
subspace of the full joint space is extracted using principal
component analysis. Recent work [15] investigates cases
where a linear dimensionality reduction might be subobtimalfor this purpose and explores the use of the nonlinear GPLVM dimensionality reduction framework. The representation which we develop here falls into this non-linear classof state space representations, but while [15] attempt tofind such a representation by data-analysis, in this work,
we consider designing such a representation by drawing
inspiration from the tools that topology offers us.
Topology-based representations
Topology, in the rigorous sense, studies topological spacesuch as, but not restricted to, smooth manifolds. Of particularinterest to us is the fact that a large class of topologicalinvariants – that is, quantities defined on a topological space
which stay invariant under certain continuous deformationsof the space – have been developed in this field.
In the context of character animation, [8] have investigateda state space representation utilizing ideas from topology.
There, the topological Gauss linking number, which describes the linking between two closed curved in R3 is used
as an inspiration to construct a writhe matrix to describe
how wrapped the limbs of two avatars are. While the writhe
matrix itself is not a topological invariant, [8] demonstratethat drawing inspiration from topology can be a very fruitfulidea. In the work [19], the writhe matrix is applied in arobotics context for the first time. There, the authors showthat the writhe matrix can successfully be used in conjunction
with the AICO framework to plan motions. As an example,

the authors demonstrate unwrapping a multi-joint robot arm
which is initially wrapped around a line segment - a task
which is difficult to solve in the high-dimensional joint space
directly.
In this paper, we begin to develop a low-dimensionaldescription of the grasp state space which is inspired byanother classical topological invariant: The winding numberof an oriented closed curve γ : [0, 1] → R2 not containing
a point p ∈ R2 . The winding number for a closed curveis hence integer valued and measures how many times
the curve wraps around p. If two plane curves γ1 , γ2 notcontaining a point p have a non-zero winding numbers
w(γ1 ), w(γ2 ), they cannot be continuously deformed into
each other without crossing the point p which explains why
w is of interest in topology. Let us remark here that, if one ispresented with a piecewise linear curve γ : [0, 1] → R2 notcontaining the origin, one can easily evaluate w by simply
adding up the local angles which the various line segments
make with the origin. In the work [12] homology groups
and a version of winding numbers of plane-curves were
investigated to generate caging grasps on objects with holes.
While we are also actively exploring the use of this notionof winding numbers for plane-curves, we will work with athree-dimensional analogue of the winding number that canbe calculated for any curve γ : [0, 1] → R3 which does nottraverse some fixed reference point p ∈ R3 .

pγ

Fig. 2.

A closed oriented plane curve γ : [0, 1] → R2 with w(γ) = 2.

III. OUR METHODOLOGY
A. Invariant representation
Given a robotic or human hand with n > 2 fingers
f1 , . . . , fn , we assume that one of the fingers, say fn , canbe labelled as a thumb. In practice, the thumb for a robotic
hand such as the Barrett, Schunk, and Shadow hand, can beeasily identified.
We then consider piecewise linear curves γ1 , . . . γn−1 ,such that γi (0) starts at the tip of the thumb fn and ends
at the tip of fi by traversing the joints of fn , then going
through the center of the base of the hand and continuing
through the joints of fi (see Figure I, where γ1 (0) is depicted
in white). Such a curve is also used in [12].
Observing that the traditional winding number w(γ) ∈ Z,
for a closed curve γ in R2 measures the winding arounda point p ∈ R2 by calculating the total change in angularcoordinates, we now define a similar quantity for piecewise
linear curves in R3 . Suppose that γ : [0, 1] → R3 is the piecewise linear curve connecting the points X0 , . . . , Xn ∈ R3 bylinear line segments from Xi to Xi+1 , for i = 0, . . . , n − 1such that a fixed point p ∈ R3 is not contained in the imageof γ. We define

y0

y1
z0

y2
z1

yT
z2

topology−based space

zT

tasks

x0

x1

x2

xT

configuration space

u0

u1

u2

uT

controlnŵ(γ) =

1 X
anglep (Xi , Xi+1 ),2π i=0where anglep (Xi , Xi+1 ) denotes the angle between the vectors Xi − p and Xi+1 − p. If γ is a closed curve that lies
completely in a plane containing p, the above quantity isjust the usual winding number. We will show in this workthat ŵ(γ) can be reliably used in a grasping application toquantify how much a curve is ‘wrapped’ around a point.

Xi

Xi+1

γ
αi
p
Fig. 3. A non-closed curve γ : [0, 1] → R3 (solid lines) with reference
point p and αi = anglep (Xi , Xi+1 ).

Motion planning
The introduction of any new representation of a state space
directly impacts motion synthesis. One of the motivations forthe search for alternate representations is that these mightallow us to express “wrapping-type motions”, which are
typical for grasping, as a simple Gaussian process prior. In
this work we shall use the quantities ŵ(γi ), as a compact
reduced representation of the “wrappedness” of a grasp. In
particular, for a human grasp posture p, we shall consider thequantitiy y = (ŵ(γi ), ŵ(γj )) ∈ R2 , where γi , γj are the two
curves with the highest winding ŵ around the centre of massof the grasped object as a “topological state descriptor” ofa grasp. We shall record y for a human demonstration grasp
and we will then use this topological coordinate to control
the Schunk robotic hand to attain the same winding values
with respect to the two curves γ10 , γ20 running through the
Schunk robotic hand.
Approximate Inference Control (AICO) frames the problem of optimal control as a problem of inference in a
dynamic Bayesian network. Let xt be the state of thesystem—we will always consider the dynamic case wherext = (qt , q̇t ). The robot dynamics are described by the transition probabilities P (xt+1 |ut , xt ). We introduce an auxiliary
random variable zt with P (zt = 1|xt ) ∝ exp{−cx (xt )}, thatis, z = 1 if the task costs cx (xt ) are low in time slice t.
AICO in particular tries to estimate the posterior trajectoryand controls. In [18], this is done using Gaussian message
passing (comparable to Kalman smoothing) based on local
Gaussian approximations around the current belief model.
Expressing motion priors in topology-based spaces andcoupling spaces: Employing AICO with a linear GaussianFig. 4. AICO in configuration and topology-based space. The new topology
motivated space represented as an additional task variable.

motion prior in a topologically motivated space is not sufficient to solve general motion synthesis problems, we needmechanisms to couple inference in topology-based spaceand state space. Figure 4 displays a corresponding graphicalmodel. The top layer represents a process in alternate space
and is coupled with the state-space layer by introducing
additional factors
1(1)
f (xt , yt ) = exp{− ρkϕ(qt ) − yt k2 },2which essentially attempt to minimize the squared distance
between the topology-based state yt and the one computedfrom the joint configuration ϕ(qt ), weighted by a precision
constant ρ. Note that for Gaussian message passing between
levels using a local linearisation of ϕ (having the Jacobianof the topology-based space) is sufficient. These factors
essentially treat the topology-based state yt , namely thewinding values (ŵ(γ1 ), ŵ(γ2 )), as an additional task variable
for the lower level inference, analogous to other potential
task variables like end-effector position or orientation.
IV. EXPERIMENTS
For our experiments we will use libORS, a freely available
robot simulator incorporating routines for planning trajectories in the AICO framework. We use two kinematic models.
Firstly, a model based on a 3d scan of a human hand with
20 DOF (Figure I, left) which serves as our “demonstrator”
and secondly a model of the Schunk robot hand (FigureI, right) which is the target of our transfer method. The
Schunk hand is connected to the Schunk arm and motion
are generated for the full 14 DOF robot.
Instead of generating training motions from human subjects, we have opted to automatically generate a large setof grasps in simulation in order to obtain a statisticallysignificant amount of test data.
Test data generation
The test data consists of a set of stable grasps with ahuman hand model and with two object models: a bottle
and a hammer as shown in the left column of Figure 6.
The grasps are generated using a part-based grasp planningsystem – BADGr [9].
For each object, its 3D shape and dimension is first
approximated through a constellation of oriented boundingboxes. These boxes decompose the object according to aconvexity index, so that each box encapsulates a compact
part of the object that is the candidate for grasp generation.
For each reachable facet of the boxes, 4 grasp hypotheses areFig. 5. Left side: examples of bottle grasps generated with human-like
hand. Right side: Results of motion planning with a 3-finger Schunk hand.

Fig. 7. A complex grasping trajectory in configuration space is represented
by a pair of simple monotonically increasing curves in our topologicallymotivated state space consisting of yt = (ŵ(γ1 (t)), ŵ(γ2 (t))). The graphdisplays steps t in the simulation (horizontal axis) against the values ofŵ(γ1 (t)) and ŵ(γ2 (t)).
TABLE IC OMPARITIVE STABILITY ANALYSIS
Objects
Total number of grasps
Stable grasps
Autoclose based on initial human
hand configuration
Grasp transfer using topologicalsynergies

Fig. 6. Left side: examples of hammer grasps generated with human-like
hand. Right side: Results of motion planning with a 3-finger Schunk hand.

generated by aligning the approach vector to its normal andthe 4 orientations to its 4 edge vectors. Details of the grasp
planning process can be found in [9]. The hypotheses are
executed on the objects in the simulator which gives a grasp
quality measure for stability assessment. We tested those
grasps for physical stability using the simulation environment
PhysX by moving the hand with the object by random
rotational motions. A grasp was accepted as stable if theobject did not fall out of the hand during this process. Using
this procedure, we first generated 251 grasps for the hammer
and bottle object. After testing for stability in PhysX, thisresulted in 67 stable grasps for the Hammer and 32 suchgrasps for the bottle (see Figure I). We shall use this set ofstable human grasps as our candidate data set for transfer tothe Schunk robot hand.
Transfer
The transfer itself consists of two phases: motion planning
using AICO and the closing of the fingers. This distinguishes

Hammer101
67
13%

Bottle
151
32
25%

53%

63%

our approach from similar methods, which focus mainly on
the last phase or which try to find good contact points. We
instead let the planner decide what an optimal configurationof the hand should be. For the optimization problem we use
the winding numbers extracted from human hand example
grasps as goals - in particular, we choose the winding withrespect to the two curves γ1 , γ2 with the largest winding
numbers around the centre of mass of the object (out of thefour curves γ1 , . . . , γ4 , running along the human hand). We
have also included a collision task variable, which defines apotential in the neighbourhood of the object and which hasvery high costs for states penetrating the object. Since not allorientations are reachable by the robot arm, we imitate theapproach direction of the simulated human grasp by rotating
the object so that the grasp approach direction is feasible forthe robot arm. In the final phase, the fingers are closed untilcontact with the surface occurs.
Evaluation
In order to benchmark the transfer of grasps, we applied a realistic simulation within PhysX. The object andthe hand (without the arm) were copied to the physicalenvironment with gravity and friction forces. We simulated
random rotational motion of the hand in 100 simulation steps
and measured if the object was still inside the hand. This
procedure then results in a success classification of the grasp.
We have mentioned in the previous section that ourapproach consists of two phases: Control via AICO and a
final automatic closing of the fingers. One could argue thattransplanting the Schunk hand to the same position as thehuman hand and an automatic closing of fingers might beenough for reliable transfer. For comparison, we first tested
such a direct ’transplantation’. For this, we put the Schunk
hand into the same position and orientation as the human
hand and performed automatic finger closing. As can be seenfrom table I, this, in a sense, naı̈ve method, has a very low
success rate w.r.t the total number of stable grasps.
Topological synergies, on the other hand, yield a muchhigher percentage of successful grasps. At least half of thetransferred grasps are stable under realistic physical conditions. It is worth mentioning that the amount of informationneeded for our method is significantly less than requiredby other approaches - only two winding numbers and theorientation of the object is sufficient for simple objects. The
trajectory in topological space (see Figure 7) is also very
simple, but as a result of the non-linear properties of the
mapping between layers of our Bayesian network, we obtaina relatively complex behaviour in the configuration space.
V. CONCLUSIONSIn this workshop contribution, we have begun to investigate a novel grasp representation based on the notion ofwinding numbers which measures how much a hand wraps
around a target object. We have used this representation
to transfer a large percentage of stable human grasps tostable grasps of a 7 degree of freedom Schunk hand. Our
method incorporates planning using the AICO framework,
which allows us to combine our topological task goals
with traditional constraints such as collision avoidance. An
initial evaluation of our approach using the PhysX simulation
framework suggests that such “topological synergies” can beused to successfully transfer grasps between a human handmodel and a Schunk hand. The strength of our approach liesin the fact that we are able to use topological task goals
in conjunction with more traditional task variables in orderto synthesize complex motions. In future, we would like toinvestigate how to incorporate additional information about
object structure into our method and we are interested in
exploring further how ideas from topology, such as winding
numbers, linking numbers and writhe can be used for the
purpose of grasping in robotics.
R EFERENCES[1] M. A. Arbib, T. Iberall, and D. Lyons. Coordinated control programsfor movements of the hand. Hand Function and the Neocortex, A. W.
Goodwin and I. Darian-Smith (Eds), pages 111–129, 1985.
[2] A. Bicchi, Marco Gabiccini, and Marco Santello. Modelling naturaland artificial hands with synergies. Phil. Trans. R. Soc., 2011.
[3] A. Bicchi and V. Kumar. Robotic grasping and contact: A review. In
ICRA, 2000.
[4] B. Buchholz and T. J. Armstrong. A kinematic model of the human
hand to evaluate its prehensile capabilities. Journal of Biomechanics,
25(2):149 – 162, 1992.
[5] D. Dragulescu, V. Perdereau, M. Drouin, L. Ungureanu, and K. Menyhardt. 3D active workspace of human hand anatomical model.
BioMedical Engineering OnLine, 6(1):15, 2007.

[6] S. Ekvall and D. Kragic. Interactive grasp learning based on humandemonstration. Robotics and Automation, 2004. Proceedings. ICRA
’04. 2004 IEEE International Conference on, 4:3519–3524 Vol.4, July
2004.
[7] Holger Friedrich, Rdiger Dillmann, and Oliver Rogalla. Interactive
robot programming based on human demonstration and advice. In
Sensor Based Intelligent Robots’98, pages 96–119, 1998.
[8] Edmond S. L. Ho and Taku Komura. Character motion synthesis by
topology coordinates. Comput. Graph. Forum, 28(2):299–308, 2009.
[9] Kai Huebner. A toolbox for box-based approximation, decomposition
and GRasping. Robotics and Autonomous Systems, 60(3):367–376,
2012.
[10] Sing Bing Kang, Katsushi Ikeuchi, and Senior Member. Toward automatic robot instruction from perception - mapping human grasps tomanipulator grasps. IEEE Transactions on Robotics and Automation,
11:432–443, 1997.
[11] A. T. Miller and P. K. Allen. Graspit! a versatile simulator for robotic
grasping. IEEE Robotics and Automation Magazine, 2004.
[12] Florian T. Pokorny, Johannes A. Stork, and Danica Kragic. Grasping
objects with holes: A topological approach. In IEEE ICRA, 2013.
[13] R. N. Rohling and John M. Hollerbach. Optimized fingertip mapping for teleoperation of dextrous robot hands. IEEE International
Conference on Robotics and Automation, pages 769–775, 1993.
[14] R. N. Rohling, John M. Hollerbach, and S. C. Jacobsen. Optimized
fingertip mapping: a general algorithm for robotic hand teleoperation.
Presence - Teleoperators and Virtual Environments, 2:203–230, 1993.
[15] J. Romero, T. Feix, C. H. Ek, H. Kjellström, and D. Kragic. Extracting
postural synergies for grasping. RSS, 2012.
[16] M. Santello, M. Fl, and J. F. Soechting. Postural hand synergies fortool use. The Journal of Neuroscience, 1998.
[17] Marco Santello and John F. Soechting. Force synergies for multifingered grasping. Exp Brain Res, 2000.
[18] Marc Toussaint. Robot trajectory optimization using approximate
inference. In Proceedings of the 26th Annual International Conference
on Machine Learning, ICML ’09, pages 1049–1056, New York, NY,
USA, 2009. ACM.
[19] Dmitry Zarubin, Vladimir Ivan, Marc Toussaint, Taku Komura, andSethu Vijayakumar. Hierarchical motion planning in topologicalrepresentations. In Proceedings of Robotics: Science and Systems,
Sydney, Australia, July 2012.