To appear in the IEEE International Conference on Robotics and Automation, 2014.
This copy is for personal use only. © 2014 IEEE
Prints will be available at http://ieeexplore.ieee.org shortly.

Grasp Moduli Spaces and Spherical Harmonics
Florian T. Pokorny

Yasemin Bekiroglu

Abstract— In this work, we present a novel representation
which enables a robot to reason about, transfer and optimize
grasps on various objects by representing objects and grasps
on them jointly in a common space. In our approach, objects
are parametrized using smooth differentiable functions which
are obtained from point cloud data via a spectral analysis. We
show how, starting with point cloud data of various objects,
one can utilize this space consisting of grasps and smoothsurfaces in order to continuously deform various surface/grasp
configurations with the goal of synthesizing force closed grasps
on novel objects. We illustrate the resulting shape space for acollection of real world objects using multidimensional scaling
and show that our formulation naturally enables us to use
gradient ascent approaches to optimize and simultaneously
deform a grasp from a known object towards a novel object.

I. I NTRODUCTION
The problem of determining a suitable representation
which enables a robot to reason about the manipulationof objects in its environment is fundamental to robotics.
While most current grasp synthesis approaches are based on
non-smooth object representations such as triangle meshes,
this work considers smooth representations of objects thatcan be continuously varied. Furthermore, while previously
objects such as boxes, cylinders, spheres and quadric surfaces have been considered as primitives for the purpose ofgrasping and manipulation (see [1] and references therein),
we shall argue that shape primitives defined in terms of aspectral analysis of the underlying objects provides a usefulalternative. Our work extends the approach of [2], where
an infinite dimensional space – the Grasp Moduli Space– for contact configurations on surfaces parametrized by acylindrical coordinate chart was introduced. It was shownthat shape deformation based approaches in this space can
The authors are with the Computer Vision and Active Perception
Lab, Centre for Autonomous Systems, School of Computer Science andCommunication, KTH Royal Institute of Technology, Stockholm, Sweden,{fpokorny, yaseminb, dani}@kth.se. This work was supported by the EU grants FLEXBOT (FP7-ERC-279933), eSMCs (FP7-ICT270212) and RoboHow.Cog (FP7-ICT-288533).

Danica Kragic

be used to synthesize and transfer stable grasps if a smoothsurface representation of the object is known. In this work,
we utilize a spectral object representation, which has also
recently been used in neuroanatomy [3], and show how
such a representation can form a basis for a deformation
based grasp synthesis framework. Our present work extends
the approach of [2] since it allows us to use point clouddata directly. Furthermore, grasps on any genus zero (i.e.
without holes) surface can in principle be modeled withthe techniques we consider here, vastly extending the set ofobjects usable within this approach. The main contributionsof this paper can be summarized as follows:a) We discuss the infinite-dimensional shape spaces Msph
and Mrad for representing shapes for the purpose ofrobotic grasping, and we investigate convexity properties of Mrad in particular.
b) For a set of example surfaces, we show how a point
cloud representation P leads to a smooth surface Sf ∈Msph using a spectral analysis of P .
c) We define the Grasp Moduli Spaces G sph and G rad form contacts on parametrized shapes in Msph and Mradrespectively. We show how distances of spectral coefficients can be used to reason about nearby grasp/object
configurations.
d) We evaluate the success of a simple gradient ascentapproach in G rad in order to locally optimize grasp
quality on a fixed surface.
e) We show how to efficiently transfer grasps in G rad
between objects using a gradient based optimization andevaluate our approach in simulation.
The paper is structured as follows: In Sec. II, we discuss
our motivation for this work and review common shape
representations in robotic manipulation and the basics ofspherical harmonics. In Sec. III, we discuss the theoreticalaspects of our work, and in Sec. IV we present experimental
results. We conclude in Sec. V and discuss future work.

II. BACKGROUND AND M OTIVATION
A. Grasp Synthesis
For a robot to grasp and manipulate objects, it is necessary
to first determine an appropriate mathematical abstraction ofthis process. Grasp synthesis classically relies on a notion ofgrasp quality [4] and force-closure in particular (see [5] fora review). In order to determine if a grasp is force closed,
one then typically assumes that all contacts between the
robot hand and the object are point contacts, and that thegeometry of the object is explicitly described – in practiceusually by a triangle mesh. We hence think of a grasp g ona surface S with centre of mass z, with m contact points
ci and associated inward-pointing normal vectors ni as aconfiguration

between similar objects and to enable a statistical analysis of‘similar’ grasps on ‘similar’ surfaces on a unified space of
shapes and grasps. The work of [10] has taken a step in thisdirection, using point cloud matching and a combinatorial
search to transfer grasps between objects. The approach of[11] instead decomposes an object into parts that can bestudied separately. There, graspable parts are then matchedbetween objects for the purpose of grasp transfer.
Note that, in both approaches, the surface geometry isrepresented using discrete surface descriptors. Our approachis instead based on a smooth object parametrization andextends the work of [2], where the space of surfaces withcylindrical coordinates, Mcyl , was considered. The space
Mcyl is convex, allowing us to deform between any two
surfaces S1 , S2 using a curve γ : [0, 1] → Mcyl ,g = (c1 , . . . , cm , n1 , . . . , nm , z) ∈ R3m × (S2 )m × R3

γ(t) = (1 − t)S1 + tS2 .

and will use the L1 grasp quality measure Q defined in [4]to determine if such a configuration is force closed given
a surface friction coefficient µ > 0. Q is defined to be theradius of the largest ball around the origin and entirely inside
the grasp wrench space if such a sphere exists and zerootherwise. Q takes positive values if wrenches in arbitrary
direction can be withstood by the grasp g and the grasp
is considered more stable the larger Q is. State of the art
approaches such as [6] then typically incorporate a grasp
quality ranking with sampling and a decomposition of theobject in order to return a list of viable grasp hypotheses.

A further benefit of the convexity is the fact that any finite setof surfaces X = {S1 , . . . , Sn } induces a finite dimensional
shape subset Conv(X) consisting of convex combinations ofelements of X. In [2], this shape space is combined with aspace of configurations of contact points to define a Grasp
Moduli Space G cyl parametrizing both a shape and a setof contact points on the shape. A metric on G cyl can thenbe defined and one can continuously move between any two
points in G cyl . A particular advantage of this approach is that
there exist subsets of shapes on which grasp synthesis can be
carried out very efficiently, such as the surfaces of revolution
Mrev ⊂ Mcyl considered in [2]. Using a deformation in
G cyl , these grasps can then be generalised to nearby surfaces
which are not themselves rotationally symmetric.
Noise can also naturally be described in G cyl since we areable to describe ‘shape-noise’ and ‘contact point configuration noise’. Three important cases are 1) when the shapeis precisely perceived, but the hand cannot be controlled
sufficiently precisely, 2) when a slightly deformed shapeis encountered but the grasp was planned on the original
internal model of the shape, and 3) when both the shapeand the final hand position are not precisely the same as aninternal representation had predicted.
We will utilize spherical harmonics to obtain a smoothsurface representation which have surfaced for manipulationin the work of [12], where the resulting smooth parametrization was used to control a rolling interaction between asimple gripper and an object. Based on the resulting surface
parametrizations, we discuss a shape space which generalises
Mcyl since, in principle, any closed surface of genus zerocan be modelled given dense samples of points from theobject’s surface.
C. Spherical Harmonics
In order to extract smooth surfaces from point cloud data,
we shall make use of a spectral analysis on the spherewhich provides an analogue of classical Fourier analysis.
Consider the space L2 (S2 ) of integrable functions on thesphere S2 ⊂ R3 . We choose a coordinate chart p(θ, ϕ) =(sin θ cos ϕ, sin θ sin ϕ, cos θ) on the sphere, with (θ, ϕ) ∈[0, π] × [0, 2π). For integers l, m, the real valued spherical

B. Shape Representations for Grasping
Triangle meshes provide a commonly used object representation for robotic grasping. To determine contact configurations on a mesh, a common approach used e.g. by
the popular simulation environment GraspIT [7] is based on
sampling hand poses that establish contact with the object
and to then rank the result by a grasp quality measure suchas the previously mentioned Q function. The work of [8] alsouses a sampling based approach to determine stable contact
configurations. A reduced object representation based on the
medial axes of a surface has been investigated in [9] for the
purpose of grasp synthesis. An alternative approach is basedon a hierarchical decomposition of objects into primitives
such as spheres, boxes or quadrics. The work of [1] focusses
on a box based decomposition in particular.
A problem with these representations is that it is not clear
how a robot can utilize learned grasps in novel situations,e.g. when an object which is slightly different from previously investigated cases is encountered. Unlike problems in
Computer Vision, where it is possible to obtain millions of
training examples for a given learning problem, a robot will
typically not grasp a given object more than a few dozen
times and even simulated grasps are difficult to obtain in
sufficient numbers so that the high-dimensional state-spaceconsisting of robot hand configurations is densely sampled
for each object instance.
A guiding motivation for our work is hence the desire
to determine a representation allowing us to transfer grasps

(a) Y4,2(b) Y5,3(c) Y9,4

Fig. 1: Illustration of some eigen-functions of ∆ on S2 .
harmonic function of degree l and order m, |m| 6 l, isdefined by|m|
cl,m Pl (cos θ) sin(|m|ϕ) −l 6 m 6 −1
cl,m 0√ P (cos θ)
Yl,m (θ, ϕ) =
m=0
l

 2 m
cl,m Pl (cos θ) cos(mϕ)
1 6 m 6 l,q2l+1 (l−|m|)!
m
where cl,m =denotes the associ2π (l+|m|)! and Pl
ated Legendre polynomial of order m and degree l. The
functions Yl,m arise as eigen-functions of the Laplacian2∂∂∆ = sin1 θ ∂θ(sin θ ∂θ
) + sin12 θ ∂∂2 ϕ on S2 [3]. For fixed l,there are 2l + 1 eigenfunctions {Yl,m : m ∈ {−l, . . . , l}}satisfying∆Yl,m = λl Yl,m ,where λl = l(l + 1) denotes the corresponding eigenvalue. These eigen-functions, for all l, m, form an infinite
orthonormal basis for LR2 (S2 ) with respect to the standard
inner product hf, gi = S2 f g dVol, for f, g ∈ L2 (S2 ) andwhere dVol = sin θdθdϕ. Fig. 1 displays a few of these
eigen-functions. Any f ∈ L2 (S2 ) can hence be expanded asf (θ, ϕ) =

∞ X
l
X

Furthermore, define Y = [Y0 , . . . , YL ] and β =(β t0 , . . . , β tL )t . Then, as explained in more detail in [13],a choice of coefficients given by β = (Yt Y)−1 Yt f yields
a solution h ∈ HL minimizing the error term. This approachis part of the theory of spline based regression which can beextended by a regularization term [14]. The above procedure
hence yields a regression methodology for functions definedon spheres. As can be seen in examples in [3], the firstL = 70 eigen-spaces are usually sufficient to capture evenminute details of f . The work of [15] uses the above
approach to find a spherical parametrization for surface
meshes. As a first step, the mesh-vertex x, y, z coordinates
are parametrized as functions on S2 using the methods
outlined in [16]. The resulting functions x, y, z : S2 → R
are then approximated using an expansion in terms of Yl,mas above. Given a sufficiently densely sampled general point
cloud of a closed surface with genus zero (i.e. withoutholes), one can also determine an embedding mapping thepoint cloud onto a sphere. One approach for this is basedon energy minimization [17], while a recent method uses
an approximated Laplace-Beltrami operator for this purpose[18], see [19] for an overview. Once the x, y, z coordinates ofthe sampled points (xi , yi , zi ) are mapped to correspondingθi , ϕi values, the above regression method can be appliedand yields a smooth parametrization (x, y, z) : S2 → R3of the point cloud. In neuroanatomy, [13] recently useda spherical harmonics expansion of highly curved brain
surfaces to characterize cortical thickness. There, a subject’s
brain surface is first mapped onto a sphere and regression isthen performed on this transformed data based on the leastsquares approach outlined above.
III. T HEORETICAL F RAMEWORK

al,m Yl,m (θ, ϕ),l=0 m=−lfor some al,m ∈ R. Suppose now that we have samples
pi = p(θi , ϕi ) ∈ S2 on the sphere and associated noisy
function values fi ∈ R, for 1 6 i 6 n, of some underlying
function f : S2 → R. We can approximately reconstruct f
using a function h ∈ L2 (S2 ) such that h lies in the direct
sum HL of the eigen-spaces of ∆ up to degree L. The vector
space HL has dimension (L + 1)2 and any such h is of the
form
L X
l
X
h(θ, ϕ) =βl,m Yl,m (θ, ϕ),l=0 m=−lfor some βl,m ∈ R. A least-squares fit, minimizing the error
term
n
X
2
|h(θi , ϕi ) − fi |i=1

is given as follows (see [13] for details): let f = (f1 , . . . , fn ),β l = (βl,−l , . . . , βl,l ),Yl,−l (θ1 , ϕ1 ) . . . Yl,l (θ1 , ϕ1 )


..
..
..
Yl = 
.
.
.
.
Yl,−l (θn , ϕn ) . . .

Yl,l (θn , ϕn )We now describe how to extract a suitable object representation from point cloud data. We then formalize and study
the shape space which we will use in this paper.
A. Spaces of Shapes and Spherical Coordinates
In this work, we are interested in smooth closed embedded
surfaces in R3 of genus zero. We make the following
definition:
Definition 3.1: Given f : S2 → R3 , we denote the image
by Sf = {f (p) : p ∈ S2 } ⊂ R3 . We define
Msph = {f ∈ C ∞ (S2 , R3 ) : f is an smooth embedding. }We furthermore denote by Mrad those f ∈ Msph for which
there exists a radial function r : S2 → R>0 such thatf (p(θ, ϕ)) = r(θ, ϕ)(sin θ cos ϕ, sin θ sin ϕ, cos θ),
for all (θ, ϕ) ∈ [0, π] × [0, 2π).
The set Msph describes parametrizations of closed embedded smooth surfaces of genus zero, while Mrad ⊂ Msph
contains parametrizations of simpler such surfaces which
can be described by a single radial coordinate functionr : S2 → R>0 . The surfaces defined by such radial functions
yield an interesting subset Mrad ⊂ Msph which we shall
explore in this work in particular. Recall that the image Sf

of a smooth embedding does not have self-intersections andyields a smooth closed surface suitable for representing avariety of real world objects. We can mirror the observations
made about the space of surfaces with cylindrical coordinatesin [2] also in the case of Mrad :Lemma 3.2: For any f1 , f2 ∈ Mrad and α1 , α2 > 0, we
have α1 f1 + α2 f2 ∈ Mrad . In particular, Mrad is convex.
Proof: Let f1 , f2 ∈ Mrad with corresponding radialfunctions r1 , r2 : S2 → R>0 . Then, for α1 , α2 > 0, f =α1 f1 +α2 f2 has radial function α1 r1 +α2 r2 which is smoothand positive. By a simple explicit computation of f and Jf
in smooth coordinate charts covering S2 , one can verify thatf is injective and that Jf has full rank everywhere. Hencef is an injective immersion. Since S2 is compact, it follows
that f is in fact a smooth embedding. Hence f ∈ Mrad . The
statement about convexity follows at once.
Note that the above does not state that the shapes in Mradthemselves have to be convex, but that the shape space Mrad
is. The above simple observation implies in particular that,given any finite set of smooth functions X = {f1 , . . . , fn } ⊂Mrad , Conv(X) ⊂ Mrad and any element of Conv(X)yields a valid smooth embedded surface. Conv(X) hence
yields a natural finite dimensional subspace of surfaces
induced by a finite set of example surfaces. Similarly, we
can consider Cone(X) = {αf : α > 0, f ∈ Conv(X)}.
We can deform between any two surfaces f1 , f2 ∈ Mradusing f (t, p) = (1 − t)f1 (p) + tf2 (p), t ∈ [0, 1], p ∈ S2 and,more generally, interpolate between any finite set of example
surfaces. For the larger set Msph , one can still consider such
convex combinations, but care has to be taken to verify thatthe resulting surfaces are still elements of Msph to makesure no degeneracies or self-intersections occur.
Distances between Shape Parametrizations: Note that,for f1 , f2 ∈ Msph
R , we can 2 define 1 a distance function d(f1 , f2 ) = ( S2 |f1 − f2 | dVol) 2 , where dVol =sin θdθdϕ. If f1 , f2 ∈ Mrad with corresponding radialfunctions r1 , r1 , then in fact
Z
Z
2
2
2
d(f1 , f2 ) =|f1 − f2 | dVol =|r1 − r2 | dVol .
S2

Sf with f ∈ Mrad , corresponding θi , ϕi coordinates can be
determined by computing the standard spherical coordinatesof each p ∈ P directly. Furthermore, instead of finding a
regression solution for the x, y, z coordinates separately, one
can then directly compute the standard radial coordinate ri
for each pi ∈ P and determine a radial function r : S2 → R
approximating the data (ri , θi , ϕi ) using eigen-spaces up todegree L > 0. Since the objects considered for grasping inthis paper can be modelled as elements of Mrad , we will
employ this simplified approach here and defer a full study ofmore general objects to later work. The function r : S2 → R
is then given byr(θ, ϕ) =L XlX

βl,m Yl,m (θ, ϕ),l=0 m=−land we denote by β(r) = (β0,0 , β1,−1 , . . . , βL,L ) the vector
of coefficients characterizing r. Let r1 , r2 be two suchfunctions with coefficients β(r1 ), β(r2 ). Observe that, sincehYl,m , Yl0 ,m0 i = δl,l0 δm,m0 , where δ denotes the Kronecker
delta function, we can quickly compute the distance d
between the resulting surfaces using the vector norm:Z2
2d(r1 , r2 ) =|r1 − r2 | dVol = kβ(r1 ) − β(r2 )k2 .
S2

Fig. 2 displays a point cloud P and a resulting reconstructedsmooth surface parametrization f ∈ Mrad which is obtainedby using all eigen-spaces of ∆ up to degree L. As L is
increased, more and more detail about the surface is captured.
Note that – unlike common mesh representations for objects– the reconstructed surface parametrization f is smooth andderivatives of arbitrary order can be computed.

S2

(a) P

(b) L = 10(c) L = 30(d) L = 50

The above formula allows us to quantify the similarity between the corresponding parametrized surfaces Sf1 , Sf2 andwe shall use the notation d(f1 , f2 ), d(r1 , r2 ) interchangeably.

Fig. 2: Point cloud P with 25000 points and reconstructedsurfaces, where all eigen-spaces up to degree L are used.

B. Spherical Harmonics and Point Clouds

C. Grasp Moduli Spaces and Spherical Coordinates

Let us here describe our methodology for obtaining asurface parametrization f ∈ Msph or f ∈ Mrad for a given
input point cloud P = {p1 , . . . , pn } ⊂ R3 sampled fromsome genus zero surface S ⊂ R3 . Such point clouds arise forexample from laser range scanners or from a structured lightsensor such as the kinect sensor. As mentioned previously,
there are several methods available to assign spherical coordinates for each point in P [17], [18], [19] and these methods
apply even for complicated non-convex objects. Once corresponding (θi , ϕi ) coordinates are found, each of the functions
x, y, z : S2 → R can then be recovered using the presented
methods. In the case where P was sampled from a surface

A long term goal of our work is the construction of ashape and grasp configuration space which allows us tocontinuously move between shape/grasp configurations andwhere we can employ a) natural optimization methods forsmooth objective functions and b) probabilistic methods
which incorporate shape and grasp uncertainties. We now
take a step in this direction by formalizing a space of grasps
on Msph and a natural grasp/shape distance function.
Definition 3.3: The Point-Contact Grasp Moduli Spacefor m point-contacts on Msph is given by G sph = Msph ×(S2 )m . The corresponding space for Mrad is given by
G rad = Mrad × (S2 )m .

A point (f, (p1 , . . . , pm )) ∈ G sph now parametrizes botha surface S = Sf and a grasp g(f, (p1 , . . . , pm )) =(c1 , . . . , cm , n1 , . . . , nm , z), where ci = f (pi ) denotes theith contact point, ni the surface normal of Sf at ci and z
the corresponding centre of mass of Sf . As in [2], we candefine a metric on grasp contact configurations as follows:
consider g, g 0 ∈ R3m × (S2 )m × R3 ,g = (c1 , . . . , cm , n1 , . . . , nm , z)0g = (c01 , . . . , c0m , n01 , . . . , n0m , z 0 )and defineFig. 3: A picture of the objects used in our experiments.

0d(g, g ) = max(dR3 (ci , c0i ), dR3 (ni , n0i ), dR3 (z, z 0 )).
i
We can in turn combine this with our metricon Msph to define a metric on G sph byd((f, (p1 , . . . , pm ))), (f 0 , (p01 , . . . , p0m )) = wd(f, f 0 ) +(1 − w)d(g, g 0 ), for some fixed w ∈ [0, 1]. Here,g = g(f, (p1 , . . . , pm )) and g 0 = g(f 0 , (p01 , . . . , p0m )). Using
this metric, we can quantify how far a given grasp/surface
configuration is from a different such configuration. The w
parameter can be chosen to weigh the relative importance of
these distances. Note also that we can continuously deform(f, (p1 , . . . , pm )) ∈ G rad to (f 0 , (p01 , . . . , p0m )) ∈ G rad ,by a curve γ = (u, v) : [0, 1] → G rad , where e.g.
u(t) = (1 − t)f + tf 0 and v deforms each pi ∈ S2
along a spherical geodesic towards p0i . The same approach
can be attempted in G sph , but one has to take care that
Su(t) ∈ Msph for all t ∈ [0, 1].
IV. E XPERIMENTS
We now describe experiments which utilize the previously discussed framework in various ways in orderto generate force closed grasp hypotheses and to explore the resulting moduli space. Details about our approach and our implementation of spherical harmonics
based regression can be found on the first author’s website
http://www.csc.kth.se/~fpokorny.
A. Object Data and Shape SpaceWe work with the collection of real world objects depicted
in Fig. 3. For each object, we generate a point cloud Pj
of 25000 uniformly sampled points on the object’s surface
which forms the basis of our approach. For this purpose, weused the laser scan data of [20] for all surfaces but S2 , S7 ,S11 , S12 in Fig. 4, for which we generated uniform samples
from our own mesh models.
This data is then centred at the mean of the point cloud
which we assume to be the centre of mass of the object, andthe spherical coordinates for each data-point are computed.
We utilize the spherical regression technique described inSec. III-B and all eigen-spaces up to degree L = 50 toobtain a respective smooth radial function r : S2 → R>0
for each point cloud which then results in a surface Sf ,f (θ, ϕ) = r(θ, ϕ)(sin θ cos ϕ, sin θ sin ϕ, cos θ). Fig. 4 displays the resulting surfaces S1 up to S12 . A more detailed
view of the point cloud for S8 can also be seen in Fig. 2.

50

S11
0
−50
−100

S12

S4
S6
S3
S9
−50

S2
S1

S7
S8
S10
0

S5
50

100

Fig. 4: The first 3 rows display smooth parametrized surfaces
determined from point clouds (25000 points each) of our objects
from Fig. 3. The last row displays the MDS projection of β vectors
parametrizing these objects in Mrad (green dots), a sequence of
spheres (black dots) and example deformations (solid lines).

Each surface corresponds to a β coefficient vector in
R2601 . In the bottom part of Fig. 4, we visualize the distribution of the β vectors for our 12 surfaces, indicated by green
dots. To enrich the set of available shapes, we also created
25000 uniformly distributed random points and respective
smooth reconstructions from 30 spheres with radii between
3cm and 6cm and we added examples of shape deformations
from shape S5 to S1 , S2 , S10 , S11 , where a deformation
between Si and Sj is obtained by a convex combination of
the respective vectors β i , β j : β(t) = (1 − t)β i + tβ j . On
the resulting set of β vectors, we then performed manifold
multidimensional scaling (MDS) to obtain Fig. 4. Note that,

(a) S5(b) S8(d) S5(e) S8Fig. 5: Random samples of points in G(c) S10(f) S10rad, where Gaussian noiseis added to β 0 , . . . , β 9 in the top row and to β 499 , . . . , β 599 in thebottom row.

as S5 is deformed, it passes the neighbourhood of the shapes
S7 and S8 . The deformations correspond to straight lines in
R2601 . The sequence of spheres is furthermore also clearlyvisible as a sequence of nearby points passing S11 and S12 .
B. Compatibility with probabilistic approaches
Note that the shape-space component of G sph as approximated by the first L eigen-spaces of ∆ is simply
corresponding to a (L + 1)2 dimensional vector space containing β coefficients. It is hence clear that the full vectorspace based machinery of machine learning and statisticalanalysis is available on this space. Additionally, appropriate
distributions on spheres exist, endowing the coordinate partof G sph with natural probability distributions. One can hence
consider building e.g. mixture models on our moduli space.
In future work, we intend to explore this direction. One could
for example consider modelling task constraints in graspingprobabilistically using such distributions. In order to illustratesampling in G sph , we created the deformed object/grasp
configurations in Fig. 5 by sampling three random (θ, ϕ)grasp contact coordinates and by adding Gaussian noise to
the first 10 dimensions of β in the top row and to dimensions
499 to 599 in the bottom row. As we can observe, ina situation mirroring the familiar case of Fourier analysis
on the circle, modifications of the first few β coefficients
correspond to large scale object properties, while the higher
dimensions correspond to higher ‘frequencies’.
C. Initial grasp synthesis
We work with a Schunk dexterous hand (SDH, 7 DOF)
displayed in Fig. 6 for which we use a simplified collisionmodel displayed in the top left part of that figure. To detectcollisions with our reconstructed smooth surfaces, we used a

Fig. 6: Top row: SDH hand collision model and two pre-shapes.
Remaining rows: stable grasps generated in simulation and their
execution in reality (background whitened).

regular mesh on (θ, ϕ) ∈ [0, π] × [0, 2π) with 10000 vertices
and mapped it onto the surface using f (θ, ϕ). These meshes
are also used in Fig.4 for visualization. We sampled SDH
hand positions oriented towards the origin and on the upper
hemisphere of varying radii in an approach similar to the oneused by GraspIT [7]. The hand was put either in a parallel
pre-shape with open fingers or in a pre-shape where thefingers are equally spread out (see the two figures to the rightin the first row of Fig. 6). We then closed the fingers until
the finger-tips were less than 1mm away from the object.
Configurations where the fingers did not reach the surface
vicinity, or where other hand-parts were less than 5mm awayfrom the object or closer than 5mm to each other werediscarded. The closest points p1 , p2 , p3 on the object andfrom the finger-tips were calculated and corresponding unit
normals n1 , n2 , n3 of the surface Sf were computed baseddf
on our smooth representation by normalizing dfdθ × dϕ . Froman initial set of 1400 random hand-positions per surface, we
obtained a set of more than 200 stable grasps per object
which we ranked by grasp quality. Let us denote by Gi the
50 best such grasps and by Ĝi the best 10 grasps on surface
Si . To verify that grasps which were stable in simulation also

1
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12

0.8

Qf inal

0.6

0.4

0.2

0

0

0.2

0.4

0.6

0.8

1

QstartFig. 7: Initial vs. final grasp quality with gradient ascentcorresponded to stable grasps in reality, we performed a shortevaluation where two simulated stable grasps on each surface
were executed on a real Schunk (SDH) hand attached to a
Kuka arm. 21 of the resulting grasps were stable, while 1
failure occurred on each of the objects S1 , S3 , S6 – most
likely due to errors in the positioning of the hand. The
supplementary video contains more details on this evaluationand examples from these experiments are also displayed inFig. 6 alongside the corresponding simulated grasps.
D. Point-contact based grasp synthesis and transfer on G rad
The synthesis and optimization of stable point-contact
configurations are important sub-problems in grasp synthesis
since stable configurations can serve as goals for a planningalgorithm positioning the robot hand.
We now study the problem of grasp optimization given astable starting grasp. For this purpose, we consider a grasp
g on a surface Sf with three contact points. Each such grasp
is determined by f and (θ, ϕ) ∈ [0, π]3 × [0, 2π)3 ⊂ R6 . The
grasp quality Q can hence be thought of as a function ofθ, ϕ which we attempt to maximize. We propose a straightforward gradient ascent approach as follows: we approximate(∇Q(x))i ≈ 1δ (Q(x + δei ) − Q(x)), where ei ∈ R6 denotes
the i-th basis vector, i ∈ {1, . . . , 6}, with δ = 10−5 as in[2]. Starting from an initial grasp x0 ∈ [0, π]3 × [0, 2π)3 ,we update xn+1 = xn + 0.01 clamp(∇Q(xn )), wherexclamp(x) = x if kxk 6 1 and kxk
otherwise. We applied
1000 steps of this approach to each grasp in Gi for eachsurface Si . The resulting initial and final grasp qualities
are summarized in Fig. 7 with grasps on different surfaces
indicated by differing symbols and colours. Note that grasp
quality is typically improved significantly, but that, in somecases, our simple fixed step-length gradient ascend does notconverge and can in fact result in final grasps which have

Fig. 8: Initial grasp (long black, grey and red normals) and changes
during the gradient ascent (smaller lines) are displayed. The finalcontact configurations are depicted as long blue lines. The grasp
quality improved in all but the bottom right example.

worse quality than the initial grasp. This behaviour could
be partially avoided by using an adaptive step size gradientascent approach instead. Fig. 8 displays successful gradientascent results with an improvement in Q from 0.51, 0.54,
0.62 and 0.62 to 0.72 (top left), 0.64 (top right), 0.70 (bottom
left) and 0.67 (bottom right) respectively. Note how a large
adjustment to the grasp configuration is made in the top
left plot while in the bottom right example only a smallchange occurs during our gradient ascent. We can think ofthis gradient ascent as forming a sequence of points in G rad ,where the shape part is held fixed. Let us remark furthermore
what we could use our approach with any grasp grasp quality
function for which we can approximate gradients, including
quality functions taking into account additional constraints.
E. Optimization-based grasp transfer in G rad
Next, we investigate the ability of our approach to be
used to transfer stable grasps from one object to anotherone. We consider, for i, j ∈ {1, . . . , 12}, i 6= j the curveγ : [0, 1] → Mrad obtained by scaling the parametrizations
fi , fj corresponding to the surfaces Si and Sj by γ(t) =(1 − t)fi + tfj . This deformation in turn corresponds tolinear interpolation between the respective β-coefficients of
Si , Sj . The naı̈ve approach to transfer a grasp to a new
surface is obtained by simply fixing the θ, ϕ coordinates of
the contact points as the underlying surface deforms. We
study an alternative approach, as introduced in [2] for G cyl ,and start with a grasp g ∈ Ĝi on Si , apply 100 steps ofour gradient ascent on Sfi = Sγ(0) , then we move the grasp
to the surface Sγ(0.1) and transfer the grasp to the slightly
deformed surface by keeping the same θ, ϕ coordinates percontact. We then apply again 100 steps of our gradientascent and move to Sγ(0.2) etc. until we reach a grasp on
Sγ(1) = Sfj which is again optimized with 100 steps of our

1remain. It is clear that our smooth parametrization will
deteriorate with partial point cloud data. An investigationof the connection of our approach to Gaussian Process
Regression [14] and a suitable probability distribution on
our shape space to accommodate missing data while avoiding
unrealistic shape reconstructions are potential future research
directions. Similarly, our method currently requires an initialmatching of the object’s rotation, e.g. using an iterative
closest point method. It would be interesting to further study
rotationally invariant methods with are compatible with ourdeformation based framework. Finally, it is clear that our
gradient ascent approach is not guaranteed to converge to a
global optimum. We hence intend to also explore alternative
numerical approaches for optimizing grasp quality within our
deformation based Grasp Moduli Space framework.

S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12

0.8

Qf inal

0.6

0.4

0.2

0

R EFERENCES
0

0.2

0.4

0.6

0.8

1

QstartFig. 9: We display initial vs. final grasp quality whentransferring grasps from one surface to another.
gradient ascent. We applied this procedure for every one of
the grasps in Ĝi and every starting surface Si and target
surface Sj 6= Si , resulting in 110 grasps per starting surface
and 1320 grasp transfers in total. Note that we concentrateon the problem of grasp transfer from a point-contact pointof view here. This approach can then be integrated at a laterstage with a motion planning approach to adapt the handas the point contact configuration is deformed. The resultsof our deformation based transfer are summarized in Fig. 9.
As one might expect, the grasp quality on the final target
surface is not always positive since our simple optimization
method can get stuck in local maxima for this non-convex
optimization problem. However, the vast majority of the
final grasps is indeed stable and of good grasp quality
after the grasp is transferred. Among the 1320 transferredgrasps only 9 had a grasp quality of less than 0.01 which
we consider unstable. Of 110 transfers per starting surface
only 2, 1, 1, 4 and 1 such unstable grasps occurred for thestarting surface S2 , S4 , S5 , S9 and S10 respectively, while
all 110 grasp transfers succeeded for each of the remaining
starting surfaces. The figure on the first page illustrates two
deformations in G rad during this optimization. The contact
normals are indicated by red, grey and green lines.
V. C ONCLUSION AND O PEN P ROBLEMSIn this work, we have further developed our long term aim
of formulating an object/grasp representation which allows
for a deformation based grasp synthesis in a combined spaceof grasps and shapes. We have shown that commonly available point cloud data can be used with our approach and that
it is naturally compatible with gradient based optimization
methods for point contact synthesis. Additionally, our approach is compatible with probabilistic modelling techniques
since probability distributions can naturally be defined on our
Grasp Moduli Space G sph . Fundamental challenges however[1] K. Huebner, S. Ruthotto, and D. Kragic, “Minimum volume boundingbox decomposition for shape approximation in robot grasping,” inIEEE ICRA, 2008, pp. 1628–1633.
[2] F. T. Pokorny, K. Hang, and D. Kragic, “Grasp moduli spaces,” inProceedings of Robotics: Science and Systems, 2013.
[3] M. K. Chung, Computational Neuroanatomy: The Methods. World
Scientific, 2013.
[4] C. Ferrari and J. Canny, “Planning optimal grasps,” IEEE ICRA, pp.
2290–2295, 1992.
[5] A. Bicchi and V. Kumar, “Robotic grasping and contact: A review,”
in IEEE ICRA, 2000, pp. 348–353 vol.1.
[6] J.-P. Saut and D. Sidobre, “Efficient models for grasp planning witha multi-fingered hand.” Robotics and Autonomous Systems, vol. 60,
no. 3, pp. 347–357, 2012.
[7] A. T. Miller and P. K. Allen, “Graspit!: A versatile simulator for
robotic grasping,” IEEE Robotics and Automation Magazine, vol. 11,
pp. 110–122, 2004.
[8] C. Borst, M. Fischer, and G. Hirzinger, “Grasping the dice by dicing
the grasp,” in IEEE/RSJ IROS, 2003, pp. 3692–3697 vol.4.
[9] M. Przybylski, T. Asfour, and R. Dillmann, “Planning grasps for
robotic hands using a novel object representation based on the medial
axis transform,” in IEEE/RSJ IROS, 2011, pp. 1781–1788.
[10] U. Hillenbrand and M. A. Roa, “Transferring functional grasps throughcontact warping and local replanning,” in Intelligent Robots andSystems (IROS), 2012 IEEE/RSJ International Conference on. IEEE,
2012, pp. 2963–2970.
[11] R. Detry, C. H. Ek, M. Madry, and D. Kragic, “Learning a dictionaryof prototypical grasp-predicting parts from grasping experience,” inIEEE International Conference on Robotics and Automation, 2013.
[12] A. Bicchi and A. Marigo, “Dexterous grippers: Putting nonholonomy
to work for fine manipulation,” The International Journal of Robotics
Research, vol. 21, no. 5-6, pp. 427–442, 2002.
[13] M. K. Chung, K. M. Dalton, L. Shen, A. C. Evans, and R. J.
Davidson, “Weighted fourier series representation and its application
to quantifying the amount of gray matter,” Medical Imaging, IEEE
Transactions on, vol. 26, no. 4, pp. 566–581, 2007.
[14] G. Wahba, Spline models for observational data. Siam, 1990, vol. 59.
[15] L. Shen and M. K. Chung, “Large-scale modeling of parametricsurfaces using spherical harmonics,” in Third Int. Symp. on 3D Data
Processing, Visualization, and Transmission. IEEE, 2006.
[16] M. S. Floater and K. Hormann, “Surface parameterization: a tutorialand survey,” in Advances in multiresolution for geometric modelling.
Springer, 2005, pp. 157–186.
[17] M. Alexa, “Recent advances in mesh morphing,” in Computer graphics
forum, vol. 21, no. 2. Wiley Online Library, 2002, pp. 173–198.
[18] M. Zwicker and C. Gotsman, “Meshing point clouds using sphericalparameterization,” in Proc. of the First Eurographics conf. on PointBased Graphics. Eurographics Association, 2004.
[19] C. Gotsman, X. Gu, and A. Sheffer, “Fundamentals of sphericalparameterization for 3d meshes,” in ACM Transactions on Graphics(TOG), vol. 22, no. 3. ACM, 2003, pp. 358–363.
[20] R. Dillmann, “Kit objectmodels web database,” 2013. [Online].
Available: http://i61p109.ira.uka.de/ObjectModelsWebUI/