# AGOD-Grasp: an Automatically Generated Object Dataset for benchmarking and training robotic grasping algorithms

Mihai Andries, Yoann Fleytoux, Serena Ivaldi, J.-B. Mouret

► To cite this version:

Mihai Andries, Yoann Fleytoux, Serena Ivaldi, J.-B. Mouret. AGOD-Grasp: an Automatically Generated Object Dataset for benchmarking and training robotic grasping algorithms. 2023. hal-03983079v2

HAL Id: hal-03983079

https://inria.hal.science/hal-03983079v2

Preprint submitted on 3 Mar 2023

**HAL** is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire **HAL**, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AGOD-Grasp: an Automatically Generated Object Dataset for Benchmarking and Training Robotic Grasping Algorithms

Mihai Andries, Yoann Fleytoux, Serena Ivaldi, J.-B. Mouret

► To cite this version:

Mihai Andries, Yoann Fleytoux, Serena Ivaldi, J.-B. Mouret. AGOD-Grasp: an Automatically Generated Object Dataset for Benchmarking and Training Robotic Grasping Algorithms. 2025. hal-03983079v3

HAL Id: hal-03983079

https://inria.hal.science/hal-03983079v3

Preprint submitted on 5 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AGOD-Grasp: an Automatically Generated Object Dataset for benchmarking and training robotic grasping algorithms

Mihai Andrieş<fnref n="1" />,² and Yoann Fleytoux² and Serena Ivaldi² and Jean-Baptiste Mouret²

*Abstract*— Robust robotic grasping of objects has broad industrial applications. The reliability of data-driven grasping methods is influenced by the variability of object shapes encountered during training. Most existing objects datasets suffer from human selection bias, lack variability, or are non-reproducible. This paper presents a physically reproducible 3D-printable object dataset for training and evaluating grasping algorithms. It contains exact 3D meshes of 50 objects for simulation and printing purposes. The various objects in the dataset were found using the MAP-Elites algorithm, optimising the variability of objects according to two grasping metrics. We used a Variational AutoEncoder (VAE) as a generative model for voxelgrid object models, which were then converted to meshes and simplified using Volumetric Hierarchical Approximate Convex Decomposition (V-HACD). The dataset is publicly available online, and can be ordered from any 3D-printing service according to given specifications. We hope it will become a standard benchmarking dataset for the robotic grasping community.

## I. INTRODUCTION AND MOTIVATION

Grasping and manipulation are essential abilities for a robot. To compute grasps on objects, two broad approaches exist: (1) analytical (model-based) grasping, which requires information about the manipulated object (weight, center of mass, friction) that may not be readily available, and (2) data-driven approaches, which exploit pre-recorded or simulated grasping data to sample potential grasps and rank them. The efficiency of data-driven approaches depends, among others, on the diversity of objects seen during training. Most object datasets for training and evaluating robotic grasping consist of objects made by humans for humans. This negatively impacts the grasping of broken or unusual objects, or trash in general, which is often the use-case for robots in the recycling industry. Moreover, standardized object datasets require logistic efforts to provide and acquire them, and depend on the availability of these objects on the market.

This paper presents a 3d-printable object dataset designed for evaluating and training robotic grasping algorithms. The dataset was conceived so as to contain objects of various grasping complexity, according to two grasping metrics employed. We provide the mesh models in stereolithography (STL) format for 3d-printing the objects to make the benchmark reproducible, and for tests in simulation. Laboratories lacking 3D-printing capabilities can order the object dataset from a 3D printing service.

This dataset can be used in scenarios where objects lack a definite prior-known shape, like waste sorting and recycling, management of contaminated materials from nuclear power plants, as well as in more traditional use-cases like domestic cleaning robots, bin picking for warehouse robots, etc.

The remainder of the paper is organised as follows: Section II presents the related work on object datasets for grasping. Section III describes the methodology employed for generating the object dataset. Section IV presents our results, an image of the resulting dataset, and discusses the limitations of our approach. We conclude in Section V with opportunities for future work.

## II. RELATED WORK

In this section, we describe the desired properties for an object dataset, and then analyse existing datasets according to these properties.

### A. Desired dataset properties

We are interested in the following properties for an object manipulation dataset:

* object form variability (to counter human bias in object design);

* availability of 3D models for simulation purposes (with complete object models preferred over 3D scans);

* availability of physical versions of objects (in addition to digital 3D models);

* reproducibility, defined by either (a) the possibility to make exact copies, usually using printable 3D models; or (b) distribute-ability: keeping large object stocks provided by reliable and long-lived suppliers, and the ability to deliver an object set (if it cannot be 3d-printed).

### B. Existing object model datasets for grasping evaluation

This section lists object datasets suitable for training and evaluating grasping algorithms. Table I contains a brief description of each dataset in chronological order of their publication, while Table II analyses them according to our criteria of interest mentioned above in Section II-A. A detailed review of existing object model datasets was provided by Calli et al. [2]. Datasets specifically designed for grasping were analysed by Morrison, Corke, and Leitner in [22].

Choi et al. created a list of objects that are difficult to manipulate by motor-impaired patients [5]. The list illustrates categories of objects presenting manipulation difficulties, but does not contain numerical representations of these objects. Similarly, the KIT object models database [13] contains

<footnote n="1">¹ IMT Atlantique, Brest, France
`firstname.lastname@imt-atlantique.fr`

² Université de Lorraine, CNRS, Inria, LORIA, Nancy, France
`firstname.lastname@inria.fr`</footnote>
# AGOD-Grasp: an Automatically Generated Object Dataset for Benchmarking and Training Robotic Grasping Algorithms

Mihai Andries<fnref n="1" />,<fnref n="2" /> and Yoann Fleytoux<fnref n="2" /> and Serena Ivaldi<fnref n="2" /> and Jean-Baptiste Mouret<fnref n="2" />

**Abstract**— Robust robotic grasping of objects has broad industrial applications. The robustness of data-driven grasping methods is influenced by the variability of object shapes encountered during training. Existing objects datasets suffer from generation method bias, human selection bias, can lack variability, and may be non-reproducible. This paper presents a physically reproducible 3D-printable object dataset for training and evaluating grasping algorithms. It contains exact 3D meshes of 50 objects for simulation and printing purposes. The various objects in the dataset were found using the MAP-Elites algorithm, optimising the variability of objects according to two grasping metrics. We used a Variational AutoEncoder (VAE) as a generative model for voxelgrid object models, which were then converted to meshes and simplified using Volumetric Hierarchical Approximate Convex Decomposition (V-HACD). Generated objects are geometrically more complex, are challenging to grasp through their lack of symmetries, and cover better the graspability range than its competitors. The dataset is publicly available online, and can be ordered from any 3d-printing service according to given specifications. It can be used as a standard benchmarking dataset by the robotic grasping community.

## I. INTRODUCTION AND MOTIVATION

Grasping and manipulation are essential abilities for a robot. To compute grasps on objects, two broad approaches exist: (1) analytical (model-based) grasping, which requires information about the manipulated object (weight, center of mass, friction) that may not be readily available, and (2) data-driven approaches, which exploit pre-recorded or simulated grasping data to sample potential grasps and rank them. The efficiency of data-driven approaches depends, among others, on the diversity of objects seen during training. Most object datasets for training and evaluating robotic grasping consist of objects made by humans for humans. This negatively impacts the grasping of broken or unusual objects, or trash in general, which is often the use-case for robots in the recycling industry. Moreover, standardized object datasets require logistic efforts to provide and acquire them, and depend on the availability of these objects on the market.

This paper presents a 3d-printable object dataset designed for evaluating and training robotic grasping algorithms. The dataset was conceived so as to contain objects of various grasping complexity, according to two grasping metrics employed. We provide the mesh models in stereolithography (STL) format for 3d-printing the objects to make the benchmark reproducible, and for tests in simulation. Laboratories

lacking 3D-printing capabilities can order the object dataset from a 3D printing service.

This dataset can be used in scenarios where objects lack a definite prior-known shape, like waste sorting and recycling, management of contaminated materials from nuclear power plants, as well as in more traditional use-cases like domestic cleaning robots, bin picking for warehouse robots, etc.

The remainder of the paper is organised as follows: Section II presents the related work on object datasets for grasping. Section III describes the methodology employed for generating the object dataset. Section IV presents our results, an image of the resulting dataset, and discusses the limitations of our approach. We conclude in Section V with opportunities for future work.

## II. RELATED WORK

In this section, we describe the desired properties for an object dataset, and then analyse existing datasets according to these properties.

### A. Desired dataset properties

We are interested in the following properties for an object manipulation dataset:

* object form variability (to counter human bias in object design);

* availability of 3D models for simulation purposes (with complete object models preferred over 3D scans);

* availability of physical versions of objects (in addition to digital 3D models);

* reproducibility, defined by either (a) the possibility to make exact copies, usually using printable 3D models; or (b) distribute-ability: keeping large object stocks provided by reliable and long-lived suppliers, and the ability to deliver an object set (if it cannot be 3d-printed).

### B. Existing object model datasets for grasping evaluation

This section lists object datasets suitable for training and evaluating grasping algorithms. Table I contains a brief description of each dataset in chronological order of their publication, while Table II analyses them according to our criteria of interest mentioned above in Section II-A. A detailed review of existing object model datasets was provided by Calli et al. [3]. Datasets specifically designed for grasping were analysed by Morrison et al. in [23].

Choi et al. created a list of objects that are difficult to manipulate by motor-impaired patients [6]. The list illustrates categories of objects presenting manipulation difficulties, but

<footnote n="1">¹ IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238 Brest, France
`firstname.lastname@imt-atlantique.fr`</footnote>

<footnote n="2">² Université de Lorraine, CNRS, Inria, LORIA, Nancy, France
`firstname.lastname@inria.fr`</footnote>
object models obtained from 3D scans, but is not physically reproducible. Jiang, Moseson, and Saxena [12] provided a grasping dataset containing RGB-D images of objects, each accompanied by a few positive and negative grasping rectangles. However, no physical objects are provided, nor models for 3D printing.

The Amazon picking challenge [7] involved manipulation of objects commonly sold online. The report describes the practical challenges encountered by robots due to objects of deformable shape, transparent or reflective surfaces, wire-frame objects, etc. The list is valuable due to its practical insights on what makes an object difficult to grasp, but does not constitute an object set.

RoboCup@Home is another competition for domestic robots involving manipulation of objects (both known and unknown). The list of predefined objects includes kitchen-ware, bags, trays, deformable and fragile objects, etc. The object set is of practical interest, but is not exactly reproducible for a benchmark setting outside of the competition.

Google presented a dataset of procedurally generated random rigid objects [24], provided in OBJ and URDF representations for use with the Bullet physics simulator. Since objects were generated according to a single formula, the dataset lacks diversity. Jiang et al. presented a pipeline for augmenting a dataset of 3d objects [11] using an autoencoder, but did not provide any new resulting dataset.

The YCB object and model set [1] has become widely used in the robotics community. However, obtaining a copy takes up to several months due to logistics, and encounters cross-border shipping difficulties (containers with food items, like cans and biscuit boxes, are emptied before shipping) and supply difficulties (some original objects were replaced with similar ones). The object meshes were obtained from object scans, and are not exact numerical representations (this applies in particular to articulated and flexible objects like chains and ropes).

The work most related to ours is that of Morrison, Corke, and Leitner [22], who generated a dataset of 2000 objects for evaluating and training grasping algorithms, using Compositional Pattern Producing Networks (CPPNs) which produce objects with radial symmetry. Of these, the authors subsequently selected and post-processed a set of 49 objects that are printable without support structures using Fused filament fabrication (FFF) 3d-printing. Objects were evaluated using the Ferrari-Canny metric provided by DexNet 2.0 [18]. Similarly, the authors used the MAP-Elites algorithm [23] to guarantee variability in the generated dataset.

Our work differs in two important aspects:

* for the generative model, we use a VAE with 3D convolutions (instead of CPPNs), which processes voxelgrid models of size 64 × 64 × 64,

* for evaluating the generated objects, we use two metrics: (1) one based on the distance between the grasp centroid and the object centroid, and (2) one based on the angular distance between the grasp axis and the face normals passing through the contact points.

### III. METHODOLOGY

This section presents the methodology employed for generating the object set.

#### A. Pipeline

The object shape generation pipeline, shown in Figure 2, has several components:

1) the sampling algorithm (MAP-Elites [23]), which generates a value vector encoding an object shape;

2) a generative model, which transforms a value vector into a 3D voxelgrid shape;

3) the voxelgrid-to-mesh converter, which selects the biggest connected component inside the generated voxelgrid, and converts it into a mesh using the Marching Cubes algorithm [17];

4) the mesh simplifier (V-HACD library [20] integrated into PyBullet), which simplifies the mesh by approximating it with a set of convex parts;

5) the mesh graspability evaluator, which provides two graspability scores for a two-fingered parallel gripper:

* a score based on the distance between the grasp centroid and the mesh centroid, and

* a score based on the angular distance between the grasp axis and the normals of the faces on which the grasp points are located.

#### B. *Sampling algorithm*

We use the MAP-Elites algorithm [23] to sample the search space of object shapes, defined by the latent layer of a VAE described in the next section. MAP-Elites generates value vectors encoding object shapes, feeds these vectors into the pipeline that converts them into 3d object shapes and evaluates their graspability scores, and then projects these objects into a 2D space defined by two grasp-ability metrics. We subdivide the projection space, defined by two object grasp-ability metrics, into $2^{14}$ niches. Dividing the space into many regions allows to have more niches for which samples compete, thus obtaining a more diverse population.

MAP-Elites being an evolutionary exploration algorithm, it needs an initial population from which to start its exploration. The goal of the MAP-Elites initialisation phase is to populate with samples 100 niches, in order to have some diversity to start the evolutionary exploration. We seed the initial population with (1) average latent representations of objects from all the categories of the ModelNet 40 dataset, (2) their pairwise combinations, and (3) ten latent object representations sampled uniformly from the [-2, 2] interval. Since the KL divergence loss of the VAE forces the latent representations to resemble a ($\mu = 0, \sigma = 1$) gaussian, 95% of samples should fall within this interval (which corresponds to $\mu \pm 2\sigma$) according to the 68-95-99.7 rule. MAP-Elites then continues to sample shapes from the gaussian distribution ($\mu = 0, \sigma = 1$) until it either fills 100 niches with samples, or until it reaches 4000 sampled objects. After this phase, MAP-Elites continues with its evolutionary search phase, involving random and directed mutations.
does not contain numerical representations of these objects. Similarly, the KIT object models database [14] contains object models obtained from 3D scans, but is not physically reproducible. Jiang et al. [13] provided a grasping dataset containing RGB-D images of objects, each accompanied by a few positive and negative grasping rectangles. However, no physical objects are provided, nor models for 3D printing.

The Amazon picking challenge [8] involved manipulation of objects commonly sold online. The report describes the practical challenges encountered by robots due to objects of deformable shape, transparent or reflective surfaces, wire-frame objects, etc. The list is valuable due to its practical insights on what makes an object difficult to grasp, but does not constitute an object set.

RoboCup@Home is another competition for domestic robots involving manipulation of objects (both known and unknown). The list of predefined objects includes kitchen-ware, bags, trays, deformable and fragile objects, etc. The object set is of practical interest, but is not exactly repro-ducible for a benchmark setting outside of the competition.

Google presented a dataset of procedurally generated random rigid objects [25], provided in OBJ and URDF representations for use with the Bullet physics simulator. Since objects were generated according to a single formula, the dataset lacks diversity. Jiang et al. presented a pipeline for augmenting a dataset of 3d objects [12] using an autoencoder, but did not provide any new resulting dataset.

The YCB object and model set [2] has become widely used in the robotics community. However, obtaining a copy takes up to several months due to logistics, and encounters cross-border shipping difficulties (containers with food items, like cans and biscuit boxes, are emptied before shipping) and supply difficulties (some original objects were replaced with similar ones). The object meshes were obtained from object scans, and are not exact numerical representations (this applies in particular to articulated and flexible objects like chains and ropes).

The work most related to ours is that of Morrison et al. [23], who generated a dataset of 2000 objects for evaluat-ing and training grasping algorithms, using Compositional Pattern Producing Networks (CPPNs) [27] which produce objects with radial symmetry. Of these, the authors subse-quently selected and post-processed a set of 49 objects that are printable without support structures using Fused filament fabrication (FFF) 3d-printing. Objects were evaluated using the Ferrari-Canny metric provided by DexNet 2.0 [19]. Similarly, the authors used the MAP-Elites algorithm [24] to guarantee variability in the generated dataset.

Our work differs in two important aspects:

* for the generative model, we use a VAE with 3D convo-lutions (instead of CPPNs), which processes voxelgrid models of size 64×64×64. Hence, the generated shapes are rather random (within the bounds of the samples present in the training set of the VAE), and are not symmetrical (a typical aspect of the EGAD objects — a consequence of using CPPNs for their generation).

* for evaluating the generated objects, we use two metrics: (1) the distance between the grasp centroid and the object centroid, and (2) the angular distance between the grasp axis and the face normals passing through the contact points.

### III. METHODOLOGY

This section presents the methodology employed for gen-erating the object set.

#### A. Pipeline

The object shape generation pipeline, shown in Figure 2, has several components:

1) the sampling algorithm (MAP-Elites [24]), which gen-erates a value vector encoding an object shape;

2) a generative model, which transforms a value vector into a 3D voxelgrid shape;

3) the voxelgrid-to-mesh converter, which selects the biggest connected component inside the generated vox-elgrid, and converts it into a mesh using the Marching Cubes algorithm [18];

4) the mesh simplifier (V-HACD library [21] integrated into PyBullet), which simplifies the mesh by approx-mating it with a set of convex parts;

5) the mesh graspability evaluator, which provides two graspability scores for a two-fingered parallel gripper:

* a score based on the distance between the grasp centroid and the mesh centroid, and

* a score based on the angular distance between the grasp axis and the normals of the faces on which the grasp points are located.

#### B. Sampling algorithm

We use the MAP-Elites algorithm [24] to sample the search space of object shapes, defined by the latent layer of a VAE described in the next section. MAP-Elites generates value vectors encoding object shapes, feeds these vectors into the pipeline that converts them into 3d object shapes and evaluates their graspability scores, and then projects these objects into a 2D space defined by two grasp-ability metrics. We subdivide the projection space, defined by two object grasp-ability metrics, into $2^{14}$ niches. Dividing the space into many regions allows to have more niches for which samples compete, thus obtaining a more diverse population.

MAP-Elites being an evolutionary exploration algorithm, it needs an initial population from which to start its explo-ration. The goal of the MAP-Elites initialisation phase is to populate with samples 100 niches, in order to have some diversity to start the evolutionary exploration. We seed the initial population with (1) average latent representations of objects from all the categories of the ModelNet 40 dataset, (2) their pairwise combinations, and (3) ten latent object representations sampled uniformly from the [-2, 2] interval. Since the KL divergence loss of the VAE forces the latent representations to resemble a ($\mu = 0, \sigma = 1$) Gaussian, 95% of samples should fall within this interval (which corresponds to $\mu \pm 2\sigma$) according to the 68–95–99.7 rule. MAP-Elites then
<table><caption>TABLE I: Object datasets suitable for grasping purposes</caption><thead><tr><th>Dataset</th><th>Description</th><th>Format</th><th>Year</th></tr></thead><tbody><tr><td>Matamoros et al. [21]</td><td>Tableware, cutlery, trash bags, bags, disks, books, coat rack, trays, pourable (muesli, cereal), heavy objects, tiny objects, fragile objects, amorphous objects</td><td>Physical objects, handed to Robocup participants</td><td>2009-2019</td></tr><tr><td>Choi et al. [5]</td><td>Medical items, dining items, bathroom items, living room items, bed room items, personal belongings</td><td>Shopping list (outdated)</td><td>2009</td></tr><tr><td>Jiang, Moseson, and Saxena [12]</td><td>1035 images of 280 different objects (different orientations, associated point cloud, background image)</td><td>The raw dataset consists of: (a) images, (b) grasping rectangles, (c) pointclouds, (d) background images and (e) a file containing a mapping from each image to the corresponding background image.</td><td>2011</td></tr><tr><td>Kasper, Xue, and Dillmann [13]</td><td>Boxes, cans, tubes, mugs, bottles, toys, pliers. Limited variation in object types. 3D models are reconstructions, not sources.</td><td>3D triangulated mesh, 2D image data, texture information, grasp information.</td><td>2012</td></tr><tr><td>Calli et al. [1, 3, 4]</td><td>Food items, kitchen items, tools, shape imate, s task items (toys, magazine)</td><td>Point clouds, associated meshes and texture, mass, dimensions, models for integration into planning and simulation software. Meshes reconstructed from scanning (without bottom side of objects).</td><td>2015</td></tr><tr><td>Correll et al. [7]</td><td>25 products commonly sold on Amazon, with various shapes, sizes, deformable shapes, transparent and reflective surfaces, fragile.</td><td>Shopping list of 25 objects commonly sold on Amazon.</td><td>2016</td></tr><tr><td>Google [24]</td><td>Procedurally generated dataset</td><td>3D models of 1000 objects in Wavefront OBJ format.</td><td></td></tr><tr><td>Morrison, Corke, and Leitner [22]</td><td>Objects automatically generated using CPPNs</td><td>2000 objects in total, of which 49 objects are printable without supporting structures</td><td>2020</td></tr><tr><td>AGOD-Grasp dataset (proposed)</td><td>Objects automatically generated in voxelgrid format using a VAE, then converted to meshes.</td><td>50 objects selected out of several hundred thousand objects.</td><td>2023</td></tr></tbody></table>

<table><caption>TABLE II: Object datasets analysed by criteria of interest</caption><thead><tr><th></th><th></th><th>Object variability</th><th>RGB-D scans</th><th>Simulation models</th><th>Physical models</th><th>3D printable</th><th>Physically deliverable</th></tr></thead><tbody><tr><td>[21]</td><td>Robocup @home</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[5]</td><td>ALS dataset</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[12]</td><td>Cornell grasping dataset</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[13]</td><td>KIT object set</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[1]</td><td>YCB dataset</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[7]</td><td>Amazon picking challenge</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[24]</td><td>Google procedurally generated dataset</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>[22]</td><td>EGAD dataset</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td></td><td>AGOD-Grasp dataset</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr></tbody></table>

### C. Shape generation

As a generative model, we use a VAE [14, 25] with an Encoder composed of 3d-convolutional layers, and a Decoder composed of transpose 3d-convolutions. The network layers are interspersed with batch normalisation and rectified linear units (ReLUs). The VAE receives as input binary voxelgrid representations of objects with a resolution of 64 × 64 × 64 pixels, and outputs binary voxelgrids of identical size. The dimension of the latent layer is 512, which is reshaped into a 8 × 8 × 8 cube by the first layer of the decoder, and then upscaled through several layers to the final output size. Consequently, the search space has 512 dimensions, as it is the size of the value vector encoding an object shape. We used a weighted reconstruction loss, penalizing

stronger the network for errors in reconstructing full voxels. This ensures that the reconstructions of voxelgrid object models (which generally contain more empty voxels that filled ones) are not completely empty at the start of training. The VAE was trained to reconstruct 3D shapes from all the object categories of the ModelNet40 object dataset [27]. This ensures that the network has sufficient generative power to create an (arguably) large enough variety of shapes.

The voxelgrid generated by the VAE is rescaled to fit within a cube with sides equal to the width of the gripper, then is converted into a mesh, by first selecting the biggest connected component in the voxelgrid, and then applying the Marching Cubes algorithm [17] implemented in the TriMesh python library [9]. Since meshes converted from voxelgrids tend to have rugged surfaces, we smooth them using the V-HACD algorithm [20] integrated into PyBullet [8], thus reducing the number of mesh faces while maintaining the overall shape. By tweaking the parameter constraining the minimum volume of convex components of the mesh, it allows to thicken object parts that are too thin and can break during 3d-printing or manipulation.

### D. Grasp computation

Grasp candidates are generated using the normals of mesh faces (see Figure 1). 256 randomly selected face normals are used as grasp axes, and for each grasp axis we compute its points of intersection with the mesh, using the TriMesh [9] library, and select the two outermost points as grasp locations. We do this to avoid computing approach paths for grasping, to simplify the computation. If the distance between the two points is larger than the maximum gripper opening, this grasp candidate is discarded.
<table><caption>TABLE I: Object datasets suitable for grasping purposes</caption><thead><tr><th>Dataset</th><th>Description</th><th>Format</th><th>Year</th></tr></thead><tbody><tr><td>Matamoros et al. [22]</td><td>Tableware, cutlery, trash bags, bags, disks, books, coat rack, trays, pourable (muesli, cereal), heavy objects, tiny objects, fragile objects, amorphous objects</td><td>Physical objects, handed to Robocup participants</td><td>2009-2019</td></tr><tr><td>Choi et al. [6]</td><td>Medical items, dining items, bathroom items, living room items, bed room items, personal belongings</td><td>Shopping list (outdated)</td><td>2009</td></tr><tr><td>Jiang et al. [13]</td><td>1035 images of 280 different objects (different orientations, associated point cloud, background image)</td><td>The raw dataset consists of: (a) images, (b) grasping rectangles, (c) pointclouds, (d) background images and (e) a file containing a mapping from each image to the corresponding background image.</td><td>2011</td></tr><tr><td>Kasper et al. [14]</td><td>Boxes, cans, tubes, mugs, bottles, toys, pliers. Limited variation in object types. 3D models are reconstructions, not sources.</td><td>3D triangulated mesh, 2D image data, texture information, grasp information.</td><td>2012</td></tr><tr><td>Calli et al. [2, 4, 5]</td><td>Food items, kitchen items, tools, shape imate, s task items (toys, magazine)</td><td>Point clouds, associated meshes and texture, mass, dimensions, models for integration into planning and simulation software. Meshes reconstructed from scanning (without bottom side of objects).</td><td>2015</td></tr><tr><td>Correll et al. [8]</td><td>25 products commonly sold on Amazon, with various shapes, sizes, deformable shapes, transparent and reflective surfaces, fragile.</td><td>Shopping list of 25 objects commonly sold on Amazon.</td><td>2016</td></tr><tr><td>Google [25]</td><td>Procedurally generated dataset</td><td>3D models of 1000 objects in Wavefront OBJ format.</td><td></td></tr><tr><td>Morrison et al. [23]</td><td>Objects automatically generated using CPPNs</td><td>2000 objects in total, of which 49 objects are printable without supporting structures</td><td>2020</td></tr><tr><td>AGOD-Grasp dataset (proposed)</td><td>Objects automatically generated in voxelgrid format using a VAE, then converted to meshes.</td><td>50 objects selected out of several hundred thousand objects.</td><td>2025</td></tr></tbody></table>

<table><caption>TABLE II: Object datasets analysed by criteria of interest</caption><thead><tr><th></th><th></th><th>Object variability</th><th>RGB-D scans</th><th>Simulation models</th><th>Physical models</th><th>3D printable</th><th>Physically deliverable</th></tr></thead><tbody><tr><td>[22]</td><td>Robocup @home</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[6]</td><td>ALS dataset</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[13]</td><td>Cornell grasping dataset</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[14]</td><td>KIT object set</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[2]</td><td>YCB dataset</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[8]</td><td>Amazon picking challenge</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[25]</td><td>Google procedurally generated dataset</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>[23]</td><td>EGAD dataset</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td></td><td>AGOD-Grasp dataset</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td></tr></tbody></table>

continues to sample shapes from the Gaussian distribution ($\mu = 0, \sigma = 1$) until it either fills 100 niches with samples, or until it reaches 4000 sampled objects. After this phase, MAP-Elites continues with its evolutionary search phase, involving random and directed mutations.

An inconvenience of our sampling approach is that it cannot purposefully explore a given region of the solution space defined using graspability metrics, as we cannot know in advance the graspability score of a shape. Due to this aspect, the initialisation phase that needs to fill a certain quota of the search space before launching the evolutionary algorithm consumes a non-deterministic amount of time. However, this inconvenience is shared by other state-of-the-art evolutionary algorithms for shape generation.

### C. Shape generation

As a generative model, we use a VAE [15, 26] with an Encoder composed of 3d-convolutional layers, and a Decoder composed of transpose 3d-convolutions. The network layers are interspersed with batch normalisation and rectified linear units (ReLU). The VAE receives as input binary voxelgrid representations of objects with a resolution of 64 × 64 × 64 pixels, and outputs binary voxelgrids of identical size. The dimension of the latent layer is 512, which is reshaped into a 8 × 8 × 8 cube by the first layer of the decoder, and then upscaled through several layers to the final output size. Consequently, the search space has 512 dimensions, as it is the size of the value vector encoding an object shape. We used a weighted reconstruction loss, penalizing stronger the network for errors in reconstructing full voxels. This ensures that the reconstructions of voxelgrid object models (which generally contain more empty voxels that filled ones) are not completely empty at the start of training. The VAE was trained to reconstruct 3D shapes from all the object categories of the ModelNet40 object dataset [29]. This ensures that the network has sufficient generative power to create an (arguably) large enough variety of shapes.

The voxelgrid generated by the VAE is rescaled to fit within a cube with sides equal to the width of the gripper, then is converted into a mesh, by first selecting the biggest connected component in the voxelgrid, and then applying the Marching Cubes algorithm [18] implemented in the TriMesh python library [10]. Since meshes converted from voxelgrids tend to have rugged surfaces, we smooth them using the V-HACD algorithm [21] integrated into PyBullet [9], thus reducing the number of mesh faces while maintaining the overall shape. By tweaking the parameter constraining the minimum volume of convex components of the mesh, it allows to thicken object parts that are too thin and can break
<figure><img src="image_2.png" /><figcaption>Fig. 1: Generating grasp candidates: grasp axes are generated from normals of mesh faces. The faces they pierce are coloured in red.</figcaption></figure>

## E. Grasp metrics

We use two metrics for evaluating grasps. The first metric, called centroid distance for brevity, computes the distance between the grasp centroid and the mesh centroid. The grasp centroid is defined as the center point between the grasp contact points. Since we use a parallel gripper with two fingers and compute a single contact point for each finger, the grasp centroid is located at the middle of the segment linking the two contact points. We normalise this value, dividing it by the length of the bounding box diagonal (which is the maximum possible distance between the mesh centroid and a grasp centroid). This makes the metric invariant to object size.

The second metric, called angular distance for brevity, computes the angular distance between the grasp axis and the normals of the two faces on which the contact points are located. More precisely, we calculate the angle in radians between the grasp axis and each of the two face normals. We then normalise each angle, dividing it by $\Pi/2.0$, which corresponds to the largest possible angle between the face normal and an arbitrary line (coming out of the face plane) intersecting it. We sum the squares of these two angles, and normalise again, dividing by two.

## F. Grasp evaluation

Each grasp is evaluated according to the two metrics mentioned above. For each grasp metric, the final graspability score of the object is computed as the 75th percentile value of the sampled grasps' scores.

MAP-Elites works on a normalised projection space, which in our case is a 2D space defined by two grasping metrics, contained within $[0..1] \times [0..1]$. It may happen that some regions of the projection space remain empty, as no objects are found that would fit in those regions. Since we compute the 75th percentile score for our grasping metrics, the samples cluster together more (in the projection space) than when using the best grasp score for each object. As samples compete for fewer regions of the search space, this reduces the population size, impeding the optimisation of the evolutionary algorithm. To counter this clustering effect, we

spread the samples in the projection space, using the 4-th root of their grasp scores:

$$ \text{spreaded grasp score} = \sqrt[4]{\text{grasp score}} \quad (1) $$

We also attempted to use grasp metrics computed by DexNet, but despite considerable effort, we could not replicate the mesh-score computation set-up of DexNet 2.0 [18], which was the last version of DexNet capable of computing grasps on an object mesh, rather than on point clouds (as in later versions). Its online alternative, DexNet as a Service (DNaaS) [16], albeit functional for computing a single grasp metric, hanged when queried for two different grasp metrics on the same object mesh. DNaaS authors confirmed the presence of this problem.

## G. Object selection

After running MAP-Elites long enough for the exploration to reach a plateau (when the number of projection space regions filled with objects stop increasing), we cluster the selected objects using K-Means into 50 clusters. For each cluster, we save the object with the smallest euclidean distance to the cluster centroid. This forms the resulting object set.

# IV. RESULTS AND DISCUSSION

The dataset-generation pipeline filled 575 niches with objects, of which we selected 50. Figure 3 shows the exploration progress, defined by the number of niches filled after each generation of the evolutionary algorithm. Each run consumed about two weeks of sequential computation on a powerful server computer. We ran the algorithm multiple times to confirm the stability of its behaviour.

Figure 4 shows the generated dataset. The bias due to the limited number of object categories in the training set (40 object categories, mostly furniture objects) is arguably visible in the resulting shapes with some objects seemingly being composed of furniture parts.

The dataset is available online<fnref n="1" />. For physical experiments, it should be printed out of Polylactic acid (PLA) with a wall thickness of 1.2mm, and a 15% infill density with uniform inner distribution. This wall width was chosen so as to fit printing nozzles of diameters 0.4mm and 0.6mm. The code used for generating the object set is available online<fnref n="2" />.

## A. Comparison with the state-of-the-art

We compared our object set with EGAD [22], as it is the closest in purpose and methods employed. For a fair comparison, we rescaled the EGAD objects to the same maximum size that we used (0.07m for each side of the object bounding box, corresponding to the maximum opening width of the gripper), and compared it to our object set according to the grasp metrics described in Section III-E. Figure 6 shows a comparison between the niches filled by our dataset, and

<footnote n="1">¹https://github.com/heap-chist-era/object_grasping_dataset/</footnote>

<footnote n="2">²https://gitlab.inria.fr/CHIST-ERA-Heap/shape-generation-code</footnote>
during 3d-printing or manipulation.

### D. Grasp computation

Grasp candidates are generated using the normals of mesh faces (see Figure 1). 256 randomly selected face normals are used as grasp axes, and for each grasp axis we compute its points of intersection with the mesh, using the TriMesh [10] library, and select the two outermost points as grasp locations. We do this to avoid computing approach paths for grasping, to simplify the computation. If the distance between the two points is larger than the maximum gripper opening, this grasp candidate is discarded.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Generating grasp candidates: grasp axes are generated from normals of mesh faces. The faces they pierce are coloured in red.</figcaption></figure>

### E. Grasp metrics

We use two metrics for evaluating grasps. The first metric, called centroid distance for brevity, computes the distance between the grasp centroid and the mesh centroid. The grasp centroid is defined as the center point between the grasp contact points. Since we use a parallel gripper with two fingers and compute a single contact point for each finger, the grasp centroid is located at the middle of the segment linking the two contact points. We normalise this value, dividing it by the length of the bounding box diagonal (which is the maximum possible distance between the mesh centroid and a grasp centroid). This makes the metric invariant to object size.

The second metric, called angular distance for brevity, computes the angular distance between the grasp axis and the normals of the two faces on which the contact points are located. More precisely, we calculate the angle in radians between the grasp axis and each of the two face normals. We then normalise each angle, dividing it by $\Pi/2.0$, which corresponds to the largest possible angle between the face normal and an arbitrary line (coming out of the face plane) intersecting it. We sum the squares of these two angles, and normalise again, dividing by two.

### F. Grasp evaluation

Each grasp is evaluated according to the two metrics mentioned above. For each grasp metric, the final graspability

score of the object is computed as the 75th percentile value of the sampled grasps' scores.

MAP-Elites works on a normalised projection space, which in our case is a 2D space defined by two grasping metrics, contained within $[0..1] \times [0..1]$. It may happen that some regions of the projection space remain empty, as no objects are found that would fit in those regions. Since we compute the 75th percentile score for our grasping metrics, the samples cluster together more (in the projection space) than when using the best grasp score for each object. As samples compete for fewer regions of the search space, this reduces the population size, impeding the optimisation of the evolutionary algorithm. To counter this clustering effect, we spread the samples in the projection space, using the 4-th root of their grasp scores:

$$ \text{spreaded grasp score} = \sqrt[4]{\text{grasp score}} \quad (1) $$

We also attempted to use grasp metrics computed by DexNet, but despite considerable effort, we could not replicate the mesh-score computation set-up of DexNet 2.0 [19], which was the last version of DexNet capable of computing grasps on an object mesh, rather than on point clouds (as in later versions). Its online alternative, DexNet as a Service (DNaaS) [17], albeit functional for computing a single grasp metric, froze when queried for two different grasp metrics on the same object mesh. DNaaS authors confirmed the presence of this problem.

### G. Object selection

After running MAP-Elites long enough for the exploration to reach a plateau (when the number of projection space regions filled with objects stop increasing), we cluster the selected objects using K-Means into 50 clusters. For each cluster, we save the object with the smallest euclidean distance to the cluster centroid. This forms the resulting object set.

## IV. RESULTS AND DISCUSSION

The dataset-generation pipeline filled 321 niches with objects, of which we selected 50. Figure 3 shows the exploration progress, defined by the number of niches filled after each generation of the evolutionary algorithm. Each run consumed about two weeks of sequential computation on a powerful server computer. We ran the algorithm multiple times to confirm the stability of its behaviour.

Figure 4 shows the generated dataset. The bias induced by the training set object shapes (40 object categories, mostly furniture objects) is arguably visible in the resulting shapes, with some objects seemingly being composed of furniture parts. The neighbourhood exploration is visible in shapes that are overall similar, but with small surface differences which cause their predicted graspability metrics to differ, and which make them fill neighbouring niches of the solution space.

The dataset is available online<fnref n="1" />. For physical experiments, we recommend to print it out of Polylactic acid (PLA) with

<footnote n="1">¹https://github.com/heap-chist-era/object_grasping_dataset/</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2: The pipeline for object shape generation and graspability evaluation.</figcaption></figure>

those filled by EGAD. Although our algorithm filled fewer niches, the generated objects are of interest, as they do not suffer from radial symmetry, as compared to EGAD objects. Figure 5 illustrates some of the shapes generated during exploration.

## B. Simulation experiments

We evaluated the set of generated objects in simulation (Gazebo simulator), by attempting to grasp them with a Franka Emika Panda robotic arm, using the following state-of-the-art grasping algorithms: (1) GQCNN [19] (2) GPD [26], and (3) GrConvNet [15] (trained on the Jacquard dataset [10] using only Depth information). Each object in our object set was presented to the algorithms in the same 4 stable orientations (computed using Trimesh) chosen at random. For each object orientation, each algorithm had 5 grasping attempts. Considering that we have 50 objects in the set and 20 grasping attempts per object (1000 grasp attempts in total) per algorithm, the grasping simulations for each algorithm lasted about 20 hours when executed in sequential manner on a desktop computer with the specifications: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz, GPU Quadro P2000 CUDA 11.5. The camera observing the objects was mounted on the wrist of the robotic arm, at a distance of 0.7m from the plane on which the objects were located.

Results are resumed in Table III. Grasping attempts were

classified as either successful or failed. A successful grasp is one in which the object remains in hand for more than 5 seconds after being grasped and lifted. A successful stable grasp is a successful grasp in which the object remains in hand after being lifted and the hand is moved in a waving motion.

GrConvNet was the most successful in our grasping experiments, followed by GPD and GQCNN. Figure 7 shows the grasping success rate of the three grasping algorithms on our generated objects. The generated objects cover almost the full graspability range of these algorithms. Surprisingly, the metrics used for graspability prediction (centroid distance and angular distance) turned out to be unreliable, as they predict poorly the object graspability. It is also interesting to notice the difference in grasping abilities of the three algorithms (which may be due to imperfect contact simulation). To account for imperfections in our implementation algorithms and simulation parameters (rigidity, grasping force) we provide data for a relative comparison of grasping results of these same algorithms on the YCB dataset [1] in Table IV.

## C. Limitations

The variety of object geometries explored was mostly influenced by the chosen 3D representation (binary voxelgrids converted to meshes), and the 3D dataset used for training our 3D generator. Another limitation is that this work focused
<figure><img src="image_3.png" /><figcaption>Fig. 2: The pipeline for object shape generation and graspability evaluation.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Exploration progress, showing the number of niches filled against the number of generations. Generation 0 happens after the initialisation phase of the MAP-Elites algorithm, in which seed objects are randomly generated for the evolutionary exploration. Each generation explores 128 new object samples. The curve represents two weeks of continuous sequential computation on a server computer.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4: 50 objects selected from the generated dataset using K-Means with 50 clusters. Objects were individually scaled for illustration purposes (i.e. they are not-to-scale).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Samples of generated objects. Viewer: MeshLab [7]</figcaption></figure>

a wall thickness of 1.2mm, and a 15% infill density with uniform inner distribution. This wall width was chosen so as to fit printing nozzles of diameters 0.4mm and 0.6mm. The

code used for generating the object set is available online <fnref n="2" />.

<footnote n="2">²https://gitlab.inria.fr/CHIST-ERA-Heap/
shape-generation-code</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 3: Exploration progress, showing the number of niches filled against the number of generations. Generation 0 happens after the initialisation phase of the MAP-Elites algorithm, in which seed objects are randomly generated for the evolutionary exploration. Each generation explores 128 new object samples. The curve represents two weeks of continuous sequential computation on a server computer.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4: 50 objects selected from the generated dataset using K-Means with 50 clusters. Objects were individually scaled for illustration purposes (i.e. they are not-to-scale).</figcaption></figure>

only on rigid objects. Moreover, several practical choices were made to reduce the computational effort for evaluating the generated objects. Among these, the sampling methods may impact the reliability of the graspability estimations: (1) the number of candidate grasps used for predicting the graspability of a generated object, (2) the number of stable poses in which we attempt to grasp the object. Another limitation is caused by the inability of the simulator to perfectly simulate contacts between the objects and the robotic gripper.

## V. CONCLUSION AND FUTURE WORK

We generated a 3d-printable dataset of object models for training and evaluating robotic grasping algorithms. The dataset was optimised for the diversity of objects, according to two grasp metrics. Grasping experiments in

<figure><img src="image_6.png" /><figcaption>Fig. 5: Samples of generated objects. Viewer: MeshLab [6]</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(a) Coverage of the projection space by our (AGOD) dataset</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(b) Coverage of the projection space by EGAD dataset [22]</figcaption></figure>

<caption>Fig. 6: Coverage of the projection space</caption>
A. *Comparison with the state-of-the-art*

We compared our object set with EGAD [23], as it is the closest in purpose and methods employed. For a fair comparison, we rescaled the EGAD objects to the same maximum size that we used (0.07m for each side of the object bounding box, corresponding to the maximum opening width of the gripper), and compared it to our object set according to the grasp metrics described in Section III-E. Our generated objects do not suffer from radial symmetry, as compared to EGAD objects. Moreover, the generated AGOD objects have a higher complexity than those in EGAD object set, as visible in the violin plot in Fig. 6. This higher object morphological complexity [1] (as coded in EGAD) influences the quality of objection perception, which in turn influences the grasp generation pipeline. The similarity matrices in Fig. 7 show that both AGOD and EGAD datasets are diverse, as measured using the Hausdorff Symmetric Distance. Fig. 5 illustrates some of the shapes generated during exploration.

<figure><img src="image_7.png" /><figcaption>Fig. 6: Complexities of objects in AGOD and EGAD [23]</figcaption></figure>

B. *Simulation experiments*

We evaluated the set of generated objects in simulation (Gazebo simulator), by attempting to grasp them with a Franka Emika Panda robotic arm, using the following state-of-the-art grasping algorithms: (1) GQCNN [20] (2) GPD [28], and (3) GrConvNet [16] (trained on the Jacquard dataset [11] using only Depth information). Each object in our object set was presented to the algorithms in the same 4 stable orientations (computed using Trimesh) chosen at random. For each object orientation, each algorithm had 5 grasping attempts. Considering that we have 50 objects in the set and 20 grasping attempts per object (1000 grasp attempts in total) per algorithm, the grasping simulations for each algorithm lasted about 20 hours when executed in sequential manner on a desktop computer with the specifications: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz, GPU Quadro P2000 CUDA 11.5. The camera observing the objects was mounted on the wrist of the robotic arm, at a distance of 0.7m from the plane on which the objects were located.

Results are resumed in Table IV. Grasping attempts are classified as either successful or failed. A successful grasp is one in which the object remains in hand for more than 5 seconds after being grasped and lifted. A successful stable grasp is a successful grasp in which the object remains in hand after being lifted and the hand is moved in a waving motion.

To our surprise, the empirically measured graspability of AGOD objects in simulation revealed to be uncorrelated with the metrics used for graspability prediction (angular distance and centroid distance). Moreover, the graspability results of the evaluated grasping algorithms were not correlated, either. This indicates that there is still room for improvement, both in terms of graspability metrics, and in grasping ability of algorithms. The correlation data is presented in Table III.

GrConvNet was the most successful in our grasping experiments, followed by GPD and GQCNN. Figure 8a shows the grasping success rate of the three grasping algorithms on our generated objects. With the almost linear growth in graspability complexity among AGOD objects, our object set better covers the graspability range of objects than the competing EGAD (Figure 8b). It is also interesting to notice the difference in grasping abilities of the three algorithms (which may be due to imperfect contact simulation). To provide a reference for a relative comparison of the grasping algorithms, so as to avoid algorithm-implementation bias, we provide the grasping results of these same algorithms on the YCB dataset [2] in Table V.

C. *Limitations*

The main limitation of this work is that it focused only on rigid objects with uniform density. The variety of object geometries explored was influenced by the chosen 3D representation (binary voxelgrids converted to meshes), and the 3D object set used for training our 3D generator. Moreover, several practical choices were made to reduce the computational effort for evaluating the generated objects. Among these, the sampling methods may impact the reliability of the graspability estimations: (1) the number of candidate grasps used for predicting the graspability of a generated object, (2) the number of stable poses in which we attempt to grasp the object. Another limitation is caused by the inability of the simulator to perfectly simulate contacts between the objects and the robotic gripper.

V. CONCLUSION AND FUTURE WORK

We generated a 3d-printable dataset of object models for training and evaluating robotic grasping algorithms. Generated objects are geometrically more complex, are challenging to grasp through their lack of symmetries, and cover better the graspability range than its competitors. Grasping experiments in simulation confirmed the diversity of the dataset with regard to grasping complexity. The dataset is available online at https://github.com/heap-chist-era/object_grasping_dataset/. In the future, we would like to investigate the graspability of articulated objects,
<table><caption>TABLE III: Grasping results on our AGOD-Grasp dataset</caption><thead><tr><th></th><th>GrConvNet<br>Jacquard RGBD</th><th>GPD</th><th>GQCNN</th></tr></thead><tbody><tr><td>Successful grasps</td><td>63.5%</td><td>53.1%</td><td>50.1%</td></tr><tr><td>Successful stable grasps</td><td>21.8%</td><td>15.7%</td><td>19.6%</td></tr></tbody></table>

<table><caption>TABLE IV: Grasping results on YCB dataset [1]</caption><thead><tr><th></th><th>GrConvNet<br>Jacquard RGBD</th><th>GPD</th><th>GQCNN</th></tr></thead><tbody><tr><td>Successful grasps</td><td>61.2%</td><td>45.5%</td><td>52.34%</td></tr><tr><td>Successful stable grasps</td><td>23.8%</td><td>20.7%</td><td>26.1%</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>Fig. 7: Grasability variation among generated objects. Each plot presents the same objects, but sorted according to grasability scores obtained using a specific algorithm (GPD/CQCNN/GrConvNet). Generated objects cover almost the full grasability range of these algorithms. The gray dots correspond to predicted grasability scores.</figcaption></figure>

simulation confirmed the diversity of the dataset with regard to grasping complexity. The dataset is available online at https://github.com/heap-chist-era/object_grasping_dataset/. In the future, we would like to investigate the graspability of articulated objects, breakable objects, and objects composed of multiple materials — attributes which influence grasp stability.

## VI. ACKNOWLEDGEMENT

This work was made possible using funds provided by the EU H2020 ERA-NET funding scheme through the CHIST-ERA project HEAP.

## REFERENCES

[1] B. Calli et al. "Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set". IEEE Robotics Automation Magazine 22.3 (Sept. 2015), pp. 36-52. ISSN: 1070-9932. DOI: 10.1109/MRA.2015.2448951.

[2] Berk Calli et al. "Benchmarking in manipulation research: The YCB object and model set and benchmarking protocols". arXiv preprint arXiv:1502.03143 (2015).

[3] Berk Calli et al. "The ycb object and model set: Towards common benchmarks for manipulation research". 2015 international conference on advanced robotics (ICAR). IEEE. 2015, pp. 510-517.

[4] Berk Calli et al. "Yale-CMU-Berkeley dataset for robotic manipulation research". The International Journal of Robotics Research 36.3 (2017), pp. 261-268. DOI: 10.1177/0278364917700714. eprint: https://doi.org/10.1177/0278364917700714. URL: https://doi.org/10.1177/0278364917700714.

[5] Young Sang Choi et al. "A list of household objects for robotic retrieval prioritized by people with ALS". 2009 IEEE International Conference on Rehabilitation Robotics. IEEE. 2009, pp. 510-517.

[6] Paolo Cignoni et al. "MeshLab: an Open-Source Mesh Processing Tool". Eurographics Italian Chapter Conference. Ed. by Vittorio Scarano, Rosario De Chiara, and Ugo Erra. The Eurographics Association, 2008. ISBN: 978-3-905673-68-5. DOI: 10.2312/LocalChapterEvents/ItalChap/ItalianChapConf2008/129-136.

[7] Nikolaus Correll et al. "Analysis and observations from the first amazon picking challenge". IEEE Transactions on Automation Science and Engineering 15.1 (2016), pp. 172-188.

[8] Erwin Coumans and Yunfei Bai. PyBullet, a Python module for physics simulation for games, robotics and machine learning. http://pybullet.org. 2016-2020.

[9] Dawson-Haggerty et al. Software: Trimesh. Version 3.2.0. 2019. URL: https://trimsh.org/.

[10] Amaury Depierre, Emmanuel Dellandréa, and Liming Chen. "Jacquard: A large scale dataset for robotic grasp detection". 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE. 2018, pp. 3511-3516.

[11] Junnan Jiang et al. "Learning Grasp Ability Enhancement Through Deep Shape Generation". Intelligent Robotics and Applications: 15th International Conference, ICIRA 2022, Harbin, China, August 1-3, 2022, Proceedings, Part IV. Springer. 2022, pp. 735-746.

[12] Yun Jiang, Stephen Moseson, and Ashutosh Saxena. "Efficient grasping from rgbd images: Learning using a new rectangle representation". 2011 IEEE International Conference on Robotics and Automation. IEEE. 2011, pp. 3304-3311.

[13] Alexander Kasper, Zhixing Xue, and Rüdiger Dillmann. "The KIT object models database: An object model database for object recognition, localization and manipulation in service robotics". The International Journal of Robotics Research 31.8 (2012), pp. 927-934.

[14] Diederik P. Kingma and Max Welling. "Auto-Encoding Variational Bayes". International Conference on Learning Representations. 2014.

[15] Sulabh Kumra, Shirin Joshi, and Ferat Sahin. "Antipodal robotic grasping using generative residual convolutional neural network". 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE. 2020, pp. 9626-9633.
<figure><img src="image_8.png" /><figcaption>Fig. 7: Similarity matrices for objects in AGOD and EGAD datasets.</figcaption></figure>

<table><caption>TABLE III: Correlation between graspability metrics employed and graspability values measured in simulation.</caption><thead><tr><th></th><th>centroidDistance</th><th>angularDistance</th><th>GPDgraspSuccess</th><th>GQCNNgraspSuccess</th><th>GrConvNetgraspSuccess</th></tr></thead><tbody><tr><th>centroidDistance</th><td>1.00</td><td>-0.04</td><td>0.12</td><td>-0.07</td><td>0.03</td></tr><tr><th>angularDistance</th><td>-0.04</td><td>1.00</td><td>-0.29</td><td>-0.02</td><td>-0.11</td></tr><tr><th>GPDgraspSuccess</th><td>0.12</td><td>-0.29</td><td>1.00</td><td>0.18</td><td>0.18</td></tr><tr><th>GQCNNgraspSuccess</th><td>-0.07</td><td>-0.02</td><td>0.18</td><td>1.00</td><td>0.40</td></tr><tr><th>GrConvNetgraspSuccess</th><td>0.03</td><td>-0.10</td><td>0.18</td><td>0.40</td><td>1.00</td></tr></tbody></table>

<table><caption>TABLE IV: Grasping results on our AGOD-Grasp dataset</caption><thead><tr><th></th><th>GrConvNet<br>Jacquard RGBD</th><th>GPD</th><th>GQCNN</th></tr></thead><tbody><tr><th>Successful grasps</th><td>63.5%</td><td>53.1%</td><td>50.1%</td></tr><tr><th>Successful stable grasps</th><td>21.8%</td><td>15.7%</td><td>19.6%</td></tr></tbody></table>

<table><caption>TABLE V: Grasping results on YCB dataset [2]</caption><thead><tr><th></th><th>GrConvNet<br>Jacquard RGBD</th><th>GPD</th><th>GQCNN</th></tr></thead><tbody><tr><th>Successful grasps</th><td>61.2%</td><td>45.5%</td><td>52.34%</td></tr><tr><th>Successful stable grasps</th><td>23.8%</td><td>20.7%</td><td>26.1%</td></tr></tbody></table>

breakable objects, and objects composed of multiple materials — attributes which influence grasp stability.

## VI. ACKNOWLEDGEMENT

This work was made possible using funds provided by the EU H2020 ERA-NET funding scheme through the CHIST-ERA project HEAP.

## REFERENCES

[1] Joshua E Auerbach et al. "Environmental influence on the evolution of morphological complexity in machines". *PLoS computational biology* 10.1 (2014), e1003399.

[2] B. Calli et al. "Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set". *IEEE Robotics Automation Magazine* 22.3 (Sept. 2015), pp. 36–52.

[3] Berk Calli et al. "Benchmarking in manipulation research: The YCB object and model set and benchmarking protocols". *arXiv preprint arXiv:1502.03143* (2015).

[4] Berk Calli et al. "The ycb object and model set: Towards common benchmarks for manipulation research". *2015 international conference on advanced robotics (ICAR)*. IEEE. 2015, pp. 510–517.

[5] Berk Calli et al. "Yale-CMU-Berkeley dataset for robotic manipulation research". *The International Journal of Robotics Research* 36.3 (2017), pp. 261–268. DOI: 10.1177/0278364917700714.

[6] Young Sang Choi et al. "A list of household objects for robotic retrieval prioritized by people with ALS". *2009 IEEE International Conference on Rehabilitation Robotics*. IEEE. 2009, pp. 510–517.

[7] Paolo Cignoni et al. "MeshLab: an Open-Source Mesh Processing Tool". *Eurographics Italian Chapter Conference*. Ed. by Vittorio Scarano et al. The Eurographics Association, 2008. ISBN: 978-3-905673-68-5.

[8] Nikolaus Correll et al. "Analysis and observations from the first amazon picking challenge". *IEEE Transactions on Automation Science and Engineering* 15.1 (2016), pp. 172–188.

[9] Erwin Coumans et al. *PyBullet*, a Python module for physics simulation for games, robotics and machine learning. http://pybullet.org. 2016–2020.

[10] Dawson-Haggerty et al. *Software: Trimesh*. Version 3.2.0. Dec. 8, 2019. URL: https://trimsh.org/.

[11] Amaury Depierre et al. "Jacquard: A large scale dataset for robotic grasp detection". *2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE. 2018, pp. 3511–3516.

[12] Junnan Jiang et al. "Learning Grasp Ability Enhancement Through Deep Shape Generation". *Intelligent Robotics and Applications: 15th International Conference, ICIRA 2022, Harbin, China, August 1–3, 2022, Proceedings, Part IV*. Springer. 2022, pp. 735–746.

[13] Yun Jiang et al. "Efficient grasping from rgbd images: Learning using a new rectangle representation". *2011 IEEE International Conference on Robotics and Automation*. IEEE. 2011, pp. 3304–3311.

[14] Alexander Kasper et al. "The KIT object models database: An object model database for object recognition, localization and manipulation in service robotics". *The International Journal of Robotics Research* 31.8 (2012), pp. 927–934.

[15] Diederik P. Kingma et al. "Auto-Encoding Variational Bayes". *International Conference on Learning Representations*. 2014.

[16] Sulabh Kumra et al. "Antipodal robotic grasping using generative residual convolutional neural network". *2020 IEEE/RSJ International Conference on Environmental and Robotic Grasping*. IEEE. 2020, pp. 1–6.
[16] Pusong Li et al. "Dex-Net as a Service (DNaaS): A Cloud-Based Robust Robot Grasp Planning System". 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE). IEEE. 2018, pp. 1420-1427.
[17] William E Lorensen and Harvey E Cline. "Marching cubes: A high resolution 3D surface construction algorithm". ACM siggraph computer graphics. Vol. 21. 4. ACM. 1987, pp. 163-169.
[18] Jeffrey Mahler et al. "Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics" (2017).
[19] Jeffrey Mahler et al. "Learning ambidextrous robot grasping policies". Science Robotics 4.26 (2019), eaau4984.
[20] Khaled Mamou. Volumetric Hierarchical Approximate Convex Decomposition (V-HACD). Accessed: 2020-06-015. 2014. URL: https://github.com/kmammou/v-hacd.
[21] Mauricio Matamoros et al. RoboCup@Home 2019: Rules and Regulations (draft). http://www.robocupathome.org/rules/2019_rulebook.pdf. 2019.
[22] Douglas Morrison, Peter Corke, and Jürgen Leitner. "EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation". IEEE Robotics and Automation Letters (2020).
[23] Jean-Baptiste Mouret and Jeff Clune. "Illuminating search spaces by mapping elites". arXiv preprint arXiv:1504.04909 (2015).
[24] Procedurally generated random objects. URL: https://sites.google.com/site/brainrobotdata/home/models (visited on 05/27/2019).
[25] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. "Stochastic Backpropagation and Approximate Inference in Deep Generative Models". International Conference on Machine Learning. 2014, pp. 1278-1286.
[26] Andreas Ten Pas et al. "Grasp pose detection in point clouds". The International Journal of Robotics Research 36.13-14 (2017), pp. 1455-1473.
[27] Zhirong Wu et al. "3D ShapeNets: A deep representation for volumetric shapes". 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 2015, pp. 1912-1920.
<figure><img src="image_9.png" /><figcaption>Object identifiers (sorted for GPD)</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Object identifiers (sorted for GQCNN)</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Object identifiers (sorted for GrConvNet)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Object identifiers (sorted for GPD)</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Object identifiers (sorted for GQCNN)</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Object identifiers (sorted for GrConvNet)</figcaption></figure>

<caption>(a) Graspability variation among AGOD objects.</caption>

<caption>(b) Graspability variation among EGAD objects [23].</caption>

Fig. 8: Graspability variation among generated objects. Each plot presents the same objects, but sorted according to graspability scores obtained using a specific algorithm (GPD/CQCNN/GrConvNet). Generated objects cover almost the full graspability range of these algorithms.

*Conference on Intelligent Robots and Systems (IROS)*. IEEE. 2020, pp. 9626–9633.

[17] Pusong Li et al. “Dex-Net as a Service (DNaaS): A Cloud-Based Robust Robot Grasp Planning System”. 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE). IEEE. 2018, pp. 1420–1427.

[18] William E Lorensen et al. “Marching cubes: A high resolution 3D surface construction algorithm”. *ACM siggraph computer graphics*. Vol. 21. 4. ACM. 1987, pp. 163–169.

[19] Jeffrey Mahler et al. “Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics” (2017).

[20] Jeffrey Mahler et al. “Learning ambidextrous robot grasping policies”. *Science Robotics* 4.26 (2019), eaau4984.

[21] Khaled Mamou. *Volumetric Hierarchical Approximate Convex Decomposition (V-HACD)*. Accessed: 2020-06-015. 2014. URL: https://github.com/kmammou/v-hacd.

[22] Mauricio Matamoros et al. *RoboCup@Home 2019: Rules and Regulations (draft)*. http://www.robocupathome.org/rules/2019_rulebook.pdf. 2019.

[23] Douglas Morrison et al. “EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation”. *IEEE Robotics and Automation Letters* (2020).

[24] Jean-Baptiste Mouret et al. “Illuminating search spaces by mapping elites”. *arXiv preprint arXiv:1504.04909* (2015).

[25] *Procedurally generated random objects*. URL: https://sites.google.com/site/brainrobotdata/home/models (visited on 05/27/2019).

[26] Danilo Jimenez Rezende et al. “Stochastic Backpropagation and Approximate Inference in Deep Generative Models”. *International Conference on Machine Learning*. 2014, pp. 1278–1286.

[27] Kenneth O Stanley. “Compositional pattern producing networks: A novel abstraction of development”. *Genetic programming and evolvable machines* 8 (2007), pp. 131–162.

[28] Andreas Ten Pas et al. “Grasp pose detection in point clouds”. *The International Journal of Robotics Research* 36.13-14 (2017), pp. 1455–1473.

[29] Zhirong Wu et al. “3D ShapeNets: A deep representation for volumetric shapes”. *2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*. June 2015, pp. 1912–1920.