Grasp Transfer for Deformable Objects
by Functional Map Correspondence
Cristiana de Farias†∗ , Brahim Tamadazte‡ , Rustam Stolkin† , Naresh Marturi†
Abstract— Handling object deformations for robotic graspingis still a major problem to solve. In this paper, we propose anefficient learning-free solution for adapting grasp hypothesis todeformed versions of an object. To this end, we investigate theapplicability of functional map (FM) correspondence, where
the shape matching problem is treated as searching for correspondences between geometric functions in a reduced basis.
For a user selected region of an object, we apply the local
contact moment (LoCoMo) grasp planner to generate grasp
candidates. Next, these candidates are transferred to an instanceof the object that has suffered an arbitrary level of deformation.
Finally, the best feasible grasp, is executed on the object while
respecting the original finger configuration as much as possible.
Experimental validation in simulation shows the efficiency in
our approach.

I. I NTRODUCTION
Reliable robotic grasping of non-rigid deformable objects
is a challenging research problem. The grasp planning literature predominantly focuses on searching for stable grasps
over the surfaces of rigid objects [1]–[4]. On the other hand,
not many works focus on grasping articulated or deformableobjects [5]. In this work, we present a method for transferring
grasps between objects that have suffered deformation.
The problem of adapting a grasp between two objects, canbe formulated as transferring information between similar
surface regions. One approach to solve this problem is toexploit similarities between familiar objects of the same
category [6]–[12]. Authors in [10], [11] have proposed amethod to transfer grasps between same category objects byrigid alignment of similar shapes, contact wrapping, and local
re-planning. Similarly, [12] solved this problem by meansof bijective contact mapping and grasp re-planning. In thiscase, rigid alignment was found by sampling the surface oftwo objects and minimising the deviation between points. A
non-rigid registration method based on Coherent Point Drift(CPD) is used in [6] to transfer manipulation skills betweenobjects. In [9], the authors have used CPD to transfer andrefine grasps between the same class of objects. A learning
method using CPD to account for shape deviations whentransferring manipulation skills between objects is proposedin [7], [8]. We note that, variations of CPD are the main
choice for non-rigid shape matching in previous works.
This work was supported by the UK National Centre for Nuclear Robotics(NCNR). Part funded by CHIST-ERA under Project EP/S032428/1 PeGRoGAM and in part supported by Faraday Institution sponsored Recyclingof Lithium Ion Batteries (ReLiB) project (grant: FIRG005).
† Extreme Robotics Laboratory, School of Metallurgy and Materials,
University of Birmingham, Birmingham, United Kingdom. ‡ Sorbonne Université, CNRS UMR 7222, INSERM U1150, ISIR, F-75005, Paris, France.
∗ Corresponding Author: CXM1029@student.bham.ac.uk

Fig. 1.
Deformable object grasping by the proposed grasp transfer
method. (left) Source and target meshes, and their functional maps. (middle)
Transferred possible grasp configurations. (right) Executed best grasp.

Although an efficient choice, CPD requires a good initialpose to work well and is prone to get stuck in local minima;
thus, leading to subpar results in the presence of larger
deformations.
We believe that functional map (FM) correspondence,
which was first formulated in [13], can be an efficient alternative to CPD for handling object deformations. In [13], theauthors propose that the non-rigid shape matching problem
can be treated as searching for correspondences between
geometric functions in a reduced basis. This choice to treat
point correspondences between objects as functions leads
to simpler convex least-square optimisation problems andprovide greater flexibility to incorporate linear constraints
to the problem. Particularly, FM has been shown to perform
well in challenging settings with large deformation [14].
Our main contribution in this work is to exploit the FM
framework to propose a robust learning-free solution to theproblem of transferring grasps between objects that havesuffered non-rigid deformations. Our solution leverages user
input and grasps generated by our Local Contact Moments(LoCoMo) based grasp planner [1] to create a ranked listof grasps focused on a region of interest of an object. We
then propose a FM-based method to map these grasps toan instance of the object that has suffered arbitrary level ofdeformation (see Fig. 1). Finally, we transform and adapt
the grasps to the deformed object and execute the best
(kinematically feasible) grasp. We validate our method byperforming a number of experiments, using a simulated 7axis robot fitted with a 2-finger gripper, on different objects
with multiple deformations. For more details, we refer thereader to [15].
II. G RASP T RANSFER T HROUGH F UNCTIONAL M APS
The pipeline of our method is shown in Fig. 2. It consistsof three main steps: grasp model generation, shape correspondence, and grasp transfer and evaluation.

Fig. 2. Proposed pipeline of our methodology to transfer non-rigid grasps using functional maps. It consists of three steps: grasp model generation (top– green); finding shape correspondences (middle – blue); and, grasp transfer (last – red).

During the model generation step we present the user
with an interface where a source object can be selected
as well as the grasp region for the object. We apply asegmentation technique on the object and select a grasping
region, in which we use the LoCoMo grasp planner [1] togenerate a ranked list of grasps. The second step is shapecorrespondence, which starts with the robot observing a newscene with a deformed version of the source object (referredas target object). The robot captures the point cloud of thescene and generates a mesh of the target object. We thenfind the relation between source and target object shapes.
The FM framework helps us in accomplishing this shapecorrespondence and we transfer the segmented region to ourtarget mesh. Finally, we compute the grasp transformation,transfer and adjust the grasp to the target so that it is bothstable and feasible. We evaluate the grasp stability as in [16]As mentioned, our approach exploits FM to compute thecorrespondences between features of two shapes and then use
this mapping to transfer the grasp configuration. We denotea shape by the Riemannian manifold embedded in R3 . Let
X , Y be respectively the source and target shapes, with nX
and nY vertices stored in triangular meshes. Tp : Y → Xrepresents the point-wise correspondence map. This problem,given that nX = nY , can also be formulated as finding thepermutation matrix Π of size nY ×nX in which all lines will
sum up to 1. Although intuitive, inferring Π directly can be acomplex problem which lacks flexibility and is badly-suitedfor more general rigid deformations. FM is an alternativetechnique that has been widely adopted for solving this overthe last few years.
The underlying idea behind FM is that it is often easier
to optimise between real-valued functions rather than pointsin a shape. Indeed, finding matches in 3D space can leadto complex non-convex optimisation problems, whereas FM
offers an elegant formalism that allows for a compact matrix
representation in a low rank basis. Furthermore, this method

Fig. 3.

Illustration of grasp transfer with our proposed method.

allows for the easy incorporation of linear constrains toregularise the map, which yields simple convex least squares
optimisation problems that are much more tractable [13]. Fora more detailed understanding, we refer the reader to [17].
III. R ESULTS
In order to evaluate our pipeline, we have performedexperiments in simulation using the open-source PyBullet
python module [18]. Our robotic setup consists of a 7-axes
robot arm fitted with a parallel jaw gripper with a camera
attached to the end-effector. To run the experiments, we have
prepare a data set with four different objects: teddy bear,cable, pliers, and pipe. Each in five different configuration1 .
We note that to run our pipeline we generated meshes fromvisual data obtained in simulation.
For each object, one of the five meshes is selected as
the model mesh and the remaining four are used with robot
experiments. The model mesh is segmented using k-meansclustering and is presented to the user on a initial screen.
1 Available at https://gitlab.com/cristianafar/deformable-object-dataset

For experiments, we have used NC = 7 clusters for eachobject. The user then selects a region of interest (one ofthe clusters). Then a set of grasps that fall into the selected
region is generated by our LoCoMo grasp planner and stored.
After transferring, grasps are executed and grasping resultsare reported following the evaluation protocol presented in[16]. That is, the object is lifted and shaking and rotation
tests are performed to verify grasp stability.
In Fig. 3 we show some of the results for the pipe, cableand pliers objects. The first column shows the shape matching results, in which similar colours represent similar regions.
The transformation column shows the selected region of thesource object (in blue) being transformed to the target shapein gray. Then, we show the generated grasp candidates andfinally the executed grasp.
Additionally, to compare our method’s performance, wehave conducted baseline experiments by replacing the FM
module in our pipeline to both CPD [19] and ICP [19].
CPD is implemented using PyPCD library and the ICP using
Open3D Python library.
Our results have shown that our method can effectively
grasp objects with a high level of deformation. Out ofall object configurations, the proposed FM-based pipeline
showed 93.75% success rate for accuracy tests, as well as an
overall success rate of 100% in lifting objects and 81.25%
of success after rotation and shaking tests are performed. For
this set of experiments CPD had a performance of 56% ingrasping the correct region, 68.75% in lifting and 43.74%
after shaking and rotation tests, whereas ICP grasped thecorrect region in 31.24% attempts and had 31.25% successafter lifting and 12.5% after the tests.
R EFERENCES[1] M. Adjigble, N. Marturi, V. Ortenzi, V. Rajasekaran, P. Corke, andR. Stolkin, “Model-free and learning-free grasping by local contact
moment matching,” in 2018 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), 2018, pp. 2933–2940.
[2] A. Sahbani, S. El-Khoury, and P. Bidaud, “An overview of 3d object
grasp synthesis algorithms,” Rob. Auton. Syst., vol. 60, no. 3, pp. 326–336, 2012.
[3] Z. Zhao, W. Shang, H. He, and Z. Li, “Grasp prediction and evaluationof multi-fingered dexterous hands using deep learning,” Rob. Auton.
Syst., p. 103550, 2020.
[4] C. de Farias, N. Marturi, R. Stolkin, and Y. Bekiroglu, “Simultaneoustactile exploration and grasp refinement for unknown objects,” IEEE
Robotics and Automation Letters, vol. 6, no. 2, pp. 3349–3356, 2021.
[5] J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, and Y. Mezouar,“Robotic manipulation and sensing of deformable objects in domestic
and industrial applications: a survey,” The International Journal ofRobotics Research, vol. 37, no. 7, pp. 688–716, 2018.
[6] J. Stückler and S. Behnke, “Efficient deformable registration of multiresolution surfel maps for object manipulation skill transfer,” in 2014
IEEE International Conference on Robotics and Automation (ICRA),
2014, pp. 994–1001.
[7] D. Rodriguez and S. Behnke, “Transferring category-based functional
grasping skills by latent space non-rigid registration,” IEEE Roboticsand Automation Letters, vol. 3, no. 3, pp. 2662–2669, 2018.
[8] D. Rodriguez, C. Cogswell, S. Koo, and S. Behnke, “Transferring
grasping skills to novel instances by latent space non-rigid registration,” in IEEE Int. Conf. on Robotics and Automation (ICRA), 2018.
[9] H.-C. Lin, T. Tang, Y. Fan, and M. Tomizuka, “A framework for robot
grasp transferring with non-rigid transformation,” in 2018 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS),
2018, pp. 2941–2948.

[10] T. Stouraitis, U. Hillenbrand, and M. A. Roa, “Functional powergrasps transferred through warping and replanning,” in 2015 IEEE
International Conference on Robotics and Automation (ICRA), 2015,
pp. 4933–4940.
[11] U. Hillenbrand and M. A. Roa, “Transferring functional grasps throughcontact warping and local replanning,” in 2012 IEEE/RSJ International
Conference on Intelligent Robots and Systems, 2012, pp. 2963–2970.
[12] H. Tian, C. Wang, D. Manocha, and X. Zhang, “Transferring grasp
configurations using active learning and local replanning,” in 2019
International Conference on Robotics and Automation (ICRA), 2019,pp. 1622–1628.
[13] M. Ovsjanikov, M. Ben-Chen, J. Solomon, A. Butscher, andL. Guibas, “Functional maps: A flexible representation of maps
between shapes,” ACM Trans. Graph., vol. 31, no. 4, Jul. 2012.
[Online]. Available: https://doi.org/10.1145/2185520.2185526[14] R. M. Dyke, F. Zhou, Y.-K. Lai, P. L. Rosin, D. Guo, K. Li, R. Marin,and J. Yang, “SHREC 2020 Track: Non-rigid shape correspondence ofphysically-based deformations,” in Eurographics Workshop on 3D Object Retrieval, T. Schreck, T. Theoharis, I. Pratikakis, M. Spagnuolo,and R. C. Veltkamp, Eds. The Eurographics Association, 2020.
[15] C. de Farias, B. Tamadazte, R. Stolkin, and N. Marturi, “Grasp transferfor deformable objects by functional map correspondence,” IEEE Int.
Conf. on Robotics and Automation (ICRA), 2022.
[16] Y. Bekiroglu, N. Marturi, M. A. Roa, K. J. M. Adjigble, T. Pardi,C. Grimm, R. Balasubramanian, K. Hang, and R. Stolkin, “Benchmarking protocol for grasp planning algorithms,” IEEE Robotics andAutomation Letters, vol. 5, no. 2, pp. 315–322, 2020.
[17] M. Ovsjanikov, E. Corman, M. Bronstein, E. Rodolà, M. BenChen, L. Guibas, F. Chazal, and A. Bronstein, “Computingand processing correspondences with functional maps,” in ACM
SIGGRAPH 2017 Courses, ser. SIGGRAPH ’17. New York,NY, USA: Association for Computing Machinery, 2017. [Online].
Available: https://doi.org/10.1145/3084873.3084877[18] E. Coumans and Y. Bai, “Pybullet, a python module for physicssimulation for games, robotics and machine learning,” 2016.
[19] A. Myronenko and X. Song, “Point set registration: Coherent pointdrift,” IEEE transactions on pattern analysis and machine intelligence,
vol. 32, no. 12, pp. 2262–2275, 2010.