Efficient End-to-End Detection of 6-DoF Grasps for Robotic Bin Picking
Yushi Liu1,2 Alexander Qualmann1 Zehao Yu2 Miroslav Gabriel1
Philipp Schillinger1 Markus Spies1 Ngo Anh Vien1 Andreas Geiger2

Abstract— Bin picking is an important building block formany robotic systems, in logistics, production or in household
use-cases. In recent years, machine learning methods for theprediction of 6-DoF grasps on diverse and unknown objects
have shown promising progress. However, existing approaches
only consider a single ground truth grasp orientation at a grasp
location during training and therefore can only predict limited
grasp orientations which leads to a reduced number of feasible
grasps in bin picking with restricted reachability. In this paper,
we propose a novel approach for learning dense and diverse
6-DoF grasps for parallel-jaw grippers in robotic bin picking.
We introduce a parameterized grasp distribution model basedon Power-Spherical distributions that enables a training basedon all possible ground truth samples. Thereby, we also consider
the grasp uncertainty enhancing the model’s robustness tonoisy inputs. As a result, given a single top-down view depth
image, our model can generate diverse grasps with multiple
collision-free grasp orientations. Experimental evaluations insimulation and on a real robotic bin picking setup demonstrate
the model’s ability to generalize across various object categories
achieving an object clearing rate of around 90% in simulation
and real-world experiments. We also outperform state of theart approaches. Moreover, the proposed approach exhibits its
usability in real robot experiments without any refinement
steps, even when only trained on a synthetic dataset, due tothe probabilistic grasp distribution modeling.

I. INTRODUCTION
Grasp detection is an essential problem for the automationof pick-and-place tasks in industry and logistics, where
robots are used to grasp and manipulate objects in unstructured environments. Recent research applies machinelearning methods to enable model-free grasp detection forparallel-jaw grippers on diverse and previously unknown
objects. Some approaches address the 4-DoF grasp problem [1], [2], primarily considering 3D position and gripper
orientation about a gravity-aligned approach vector. While
this simplifies the grasping problem, it requires assumptions
on the gripper orientation, leading to a top-down grasp
execution limitation. Other approaches propose end-to-end 6DoF grasp detection learning based on a deep neural networkarchitecture for feature extraction from 3D input data [3],[4], [5], [6], [7]. Even though these approaches directly
output multiple complete gripper poses, they only predict
one gripper orientation at a location an thus neglect otherpotential gripper orientations. Jeng. et. al. [8] propose an endto-end network predicting confidence scores for various grasp
orientations, along with refinement values for diverse gripper
orientations per grasp point. However, this approach requires
a specific fine-tuning process of grasp poses. Furthermore,
1 Bosch Center for Artificial Intelligence, Renningen, Germany.
2 University of Tuebingen, Tuebingen AI Center, Germany.

Bin Picking
SystemDepth Image,
Camera Parameters

Multi-Orientation 6-DoF Grasps

Inference
TrainingPθ (g∣V )Normal GridV

Two-Stage Network

Maximum
Likelihood
True Samples

Optimization

Fig. 1: Schematic overview on training and inference
pipelines of the proposed 6-DoF grasp prediction model.
The network is trained to predict the optimal parameter θof the grasp distribution Pθ (g|V ), maximizing the likelihoodof ground truth grasp samples; During inference, the model
infers grasps with multiple orientations including graspability
and collision scores. Green forks indicate 6-DoF grasps.

current methods only consider a single ground truth grasp
orientation at a grasp point during training which then limits
the number of feasible grasps during inference and can
reduce the grasp performance in bin picking with additionalreachability restrictions. In this paper, we propose a novel
approach for predicting dense parallel-jaw grasps in clutteredbin picking scenarios from single-view observations. Our
method predicts multiple orientations per contact point basedon grasp distribution modeling.
Our contributions are as follows: (1) A novel objectivefunction based on a distribution modeling and integrating
uncertainty learning for different grasp orientations at eachcontact point. The function enables learning with multiple
diverse true grasp samples per contact point and enhances
the method’s robustness to noisy input. (2) A two-stage endto-end trainable network for 6-DoF grasp prediction, which
generates dense and collision-free grasps for a single-view
depth image including multiple grasp orientations at eachcontact point. (3) Experimental evaluation in simulation andon a real robot bin picking setup (see Fig. 6). The results
show that our method outperforms state of the art baseline
methods and demonstrates the adaptability of the approachto real robot bin picking setups, even when only trained ona synthetic dataset, due to the integrated probabilistic grasp
distribution modeling.

II. R ELATED W ORK

Gripper Centert g =c+0.5 w b−d a

Grasp detection is a widely researched topic, with notable
surveys like [9] and [10]. In this section, we focus on theparts of this research most relevant to our work.

Gripper
Heightd

A. Data-driven 6-DoF Grasp Detection
Recent 6-DoF grasp detection techniques directly regress
grasp poses using scene sensor data. Many methods treat
each surface point as a grasp center and predict grasp
parameters for every voxel center [5], [6], point [3], [4],or pixel [11]. This approach can be inefficient when the
input space significantly exceeds the grasp translation space.
Another common strategy [12], [13], [14], [15] uses multistage networks to simplify the 6-DoF grasp representation,with each stage predicting specific grasp parameters. Thesemethods typically consider only the closest ground truth
grasp during training, limiting the range of predicted orientations. Our method, in contrast, first identifies contact
point candidates and then predicts grasp parameters based onthese points. Additionally, we define a local grasp orientation
distribution for each contact point. This allows our model
to train on various grasp labels, producing multiple grasp
orientations for each contact. While the work by Jeng etal.[8] is similar in that it also generates multiple orientations
per grasp center using a coarse-to-fine approach, our method
stands out by directly outputting accurate grasp poses foreach contact point using local grasp distribution modeling,eliminating the need for refinement.
B. Scene Representation
Scene representation is crucial for efficient grasp prediction. The Truncated Signed Distance Function (TSDF) is a
popular choice due to its ability to represent near-surface
information. However, generating a comprehensive TSDF
grid requires multi-view depth or RGB inputs [5], [7], [16],which is challenging in bin picking contexts. In contrast,
TSDF grids from monocular view often miss essential geometric details, e.g. the contact surface. While auxiliarytasks such as shape completion or 3D reconstruction can
improve predictions [6], we prioritize reconstructing spaces
crucial for grasping, including the contact region, to boost
training efficiency. As pointed out in [17], [18], [19], [20]that surface normals is of great importance for predicting
grasp orientation, our model utilizes a normal grid where
each voxel encoding a surface normal.
C. Grasp Pose Representation
Recent research favors the 6-DoF grasp representation
due to its ability to offer greater grasping versatility. Thesemethods primarily utilize points from the point cloud orvoxel centers as contact candidates for the gripper’s endpoint position. [21] directly maps grasp poses in SE(3),employing grasp translation and a unit quaternion for rotation. Other methods [19], [5], [6], [22] adopt a similar
representation to predict point-wise grasp configurations.
Additionally, methods like [3], [12], [23] use Gram-Schmidt
orthonormalization to define grasp poses with an approach

Approach Point
t=c−l aOpening Widthwc'Contact
Pairb

Baseline
Vectorl Finger
DepthcContact Pointa Approach Vector

Fig. 2: Contact grasp.
and rotation vector. GDN [8] proposes learning multiple
approach vectors for each feature volume. However, thesemethods predominantly represent only one grasp per contact
point. In contrast, our approach constructs a parameterized
grasp orientation distribution at each grasp contact point
using a Power-Spherical distribution, enabling infinite ways
to grasp objects at a single contact point.
III. P ROBLEM S TATEMENT
We consider the problem of 6-DoF grasp detection for random bin picking scenes with cluttered and unknown objects
located inside a bin and captured by a single top-down view
camera depth image.
A. Notations
3D Normal Grid: We define a normal grid, denoted as V, as
an N 3 voxel grid where each voxel that is close to the object
surface stores the normal vector corresponding to the nearest
surface point relative to the voxel’s center. To create this 3D
normal grid, we initially transform the raw depth image intoa truncated signed distance field (TSDF). Subsequently, we
extract surface normals for voxel centers located closely tothe object surface. These extraction techniques are based on
methods described in prior works such as [24], [25], [26].
Contact Grasp: We use the contact grasp representation,i.e. the Gram-Schmidt orthonormalization, for parallel-jaw
grippers proposed in [3]. Each contact grasp g at a contact
point c ∈ R3 is represented as a 4-tuple:g = (c, b, a, w),
3

(1)
3

where b ∈ R is the grasp baseline vector, a ∈ R is thegrasp approach vector, and w ∈ R+ is the grasp opening
width of the parallel-jaw gripper (Fig. 2).
Approach Point: A reference point t of the gripper handlocation (see Fig. 2). We select this point because it isclose to the contact point and related to approach vector.
We also observed that learning its distribution is beneficial
for predicting collision-free approach directions.
Antipodal Quality: A metric q measuring grasp stability by
calculating the antipodality [1], [19], [27] of the contact pair(c1 , c2 ) of a parallel gripper, which is defined asq(c1 , c2 ) = | cos(b, n(c1 )|| cos(b, n(c2 )|,(2)where b is the baseline vector, n(·) computes the surface
normal vectors at two contact points.
Grasp Configuration: An output representation of our network, representing a set of grasps with different approachdirections at a contact point c (Fig. 3). We denote it as C
and formulate it asrr, w, q).
, {σi }ni=1
C = (c, b, {ai }ni=1(3)r
are derived from the b. EachThe approach vectors {ai }ni=1
ai is obtained by rotating an orthogonal vector b⊥ around b
by a rotation angles γi . The vector b ⊥ is obtained using ther
are calculated byGram-Schmidt process. The angles γi ni=1
dividing an angle range into nr equal steps. This formulation
allows each approach vector to be expressed as a linear
transformation of b, as shown in the equation:ai = Rγi b⊥ , b⊥ = R⊥ b,(4)

where Rγi is the rotation matrix of γi and R⊥ is a constantrotation matrix.
Each approach vector ai is labeled by a binary collisionscore σi , which indicates if the grasp (c, b, ai , w) is
collision-free. The antipodal quality q measures the antipodality (Eq. 2) of the contact pair (c, c + bw).
B. Supervised Learning
The goal is to train a network, which takes a 3D normal
grid V as input and infers a set of grasp configurations. We
use each grasp configuration together with an uncertainty κto parameterize the grasp distribution for each scene. Given
the true grasp samples, the network is trained to predict
optimal parameters that maximize the likelihood (Figure1). We decompose the learning problem into learning thefollowing two functions:f1 : V i → p crf2 : c → (b, κ, {σi }ni=1, w, q)(5)

The first function f1 approximates the contact point distribution, which maps each voxel Vi of the input grid tothe probability that the voxel contains a contact point. The
second function f2 then infers the antipodal quality q andthe grasp parameters given the contact points sampled fromf1 .
IV. METHODS
A. Grasp Distribution ModelingWe introduce a parameterized grasp distribution model
to consider multiple grasp orientations and their orientation
uncertainty at a contact point by representing the grasp
distribution as a joint distribution over the grasp parametersb, a, c, t, w conditioned on the input normal grid V:
P (b, a, c, t, w|V).

(6)

Here, we incorporate the approach point locations denoted
as t into our model, as they indicate the gripper hand’s
position relative to an approach direction, which is criticalfor assessing whether a grasp may lead to collisions. The
grasp distribution can then be factorized into a contact andapproach point distribution P (c|V) and the local grasp
distribution P (b, a, t, w|c, V).

t

γ

c

b

a

V

w

σ i=1

b c∼PS ( νc , κ c)c
The optimalBase Vectorνca(c ,1)=R ⊥ ν c

σ n =0r

σ 1=1ca(c , γ )=R R ν
i

γi

⊥ c

Fig. 3: A baseline vector distribution (left), modeled asa PS distribution P S(νc , κc ); An example of the grasp
configuration (bottom right), where red and green forks
represent collision and collision-free grasps, respectively, at
a contact point; The graphical model (top right) illustrating
the conditional dependency between variables.

1) Contact Point Distribution: We represent each bin
picking scene as a 3D label grid L where each voxel belongs
to one of the three non-overlapping sub-spaces: (1) contact space of the grasp contact point locations; (2) gripper
space of the collision-free gripper poses represented by theapproach point locations; (3) empty space which is thecomplement of the contact space and the gripper space.
Accordingly, each voxel is classified as contact voxel lc ,approach point voxels lt , or empty voxel le . The contact
point distribution is then approximated by the prediction ofcontact voxels.
2) Local Grasp Distribution: We further factorize thelocal grasp distribution into three distributions based on theconditional dependency between the grasp parameters asshown in Fig. 3. The factorization is formulated asZ
P (b, a, t, w|c) =P (b, a, t, w, γ|c)dγγ
Z= P (w|b, c)P (b|c) P (a, t, γ|b, w, c)dγ
Zγ
= P (w|b, c) P (b|c) P (a|b, γ)P (t|b, c, γ)P (γ|b, c)dγ
| {z } | {z } γ{z}opening width baseline |approach(7)Here, we removed V just for a simplification.
3) Baseline Vector Distribution: For a given contact point,
there are multiple baseline vectors that represent a feasible
grasp as shown in Fig. 3. We model the distribution of baseline vectors at a contact point using the the Power-Spherical(PS) distribution [28] parameterized by the optimal baseline
vector νc and the concentration parameter κc , formulated as
P (b|c) = PP S (b|νc , κc ),(8)

where PP S (·|νc , κc ) is the probabilistic density function(PDF) of the PS distribution P S(νc , κc ). The optimal base-Normal GridV

3D U-Net

well as the prior antipodal quality qc0 and nr prior collision0
scores σ1,c, . . . , σn0 r ,c for each contact point. We refer themas ”prior” quality/score because they are conditioned on
the predicted label grid. We compute the quality qc by
multiplying qc0 with the confidence score of the respective
contact point c, formulated asContact Points c(red)

Label Grid L

Blue: L t ; Red: L c
Sampling

Lc
Stage 1
Encoder Feature Grids

qc = I(Lc , c)qc0 ,

q'

σ '1
…

MLP

σ 'n

r

ν

Stage 2fc

Grid Sampleκwline vector refers to the one that represents the most stablecontact point pair with the highest antipodal grasp quality.
4) Approach Vector Distribution: Since the PS distribution is closed under linear transformation [28], based on
Eq. 4, approach vectors with respect to a rotation γ follows
the PS distribution P S(Rγ R⊥ νc , κc ). Thus, we model theapproach vector distribution as a mixture PS distribution
based on Eq. 7 by defining the component and weightfunctions as
P (a|b, γ) := PP S (a|Rγ R⊥ νc , κc ),(9)

P (t|b, c, γ, V)P (γ|b, c) := σ(γ).

(10)

We assume that the weight function σ(γ) is a binary piecewise constant function, indicating whether an approach direction aγ results in collisions. Computing the continuous integral of this mixture model during our training is intractable.
Hence, we approximate the function using nr binary values,rcorresponding to collision-scores {σi }ni=1
of nr rotation
angles (Eq. 3). We can then formulate the approach vector
distribution at a contact point c as
1
Pc (a) = PnrnrXi=1 σi i=1

σi PP S (a|Rγi R⊥ νc , κc ).

where I denotes the trilinear interpolation function. Similarly, a collision-score is calculated by multiplying its priorapproach direction to the confidence score of the predicted
approach points t(i,c) , formulated as0
σ(i,c) = I(Lt , t(i,c) ))σ(i,c),

Fig. 4: The two-stage network architecture.

(11)

B. Two-stage Network
Based on the grasp distribution modeling, we propose a
two-stage neural network shown in Fig. 4.
In the first stage, a 3D Residual U-Net [29], [30] infers
the label grid L given a 3D normal grid. Then, we sample ncontact points from contact voxels. Each voxel of L stores thethree confidence scores [pc , pt , pe ] for the classes of contact
voxel lc , approach voxel lt and empty voxel le . We denotethe grid storing pc as Lc and the grid storing pt as Lt .
In the second stage, according to n contact point locations,
we first grid sample and concatenate encoder features fromeach features grid Fi , which are intermediate outputs of
U-Net encoder layers, forming n multi-resolution encoder
feature vectors fc . Each feature vector is then passed into a
Multi-Layer Perceptron (MLP) generating the optimal baseline vector νc , inverse concentration κ0c , opening width wc , as

(12)

(13)

which aligns with the weight function Eq. 10. To obtain theconcentration parameter κ, we derive it from κ0 using theequation:κ0, κ0 ), 4κ0 )(14)
κ = min(max( 0κ +
where κ0 is a tunable scale hyperparameter, and  is used to
avoid division by zero. In our training, we set κ0 = 25.
1) Loss Functions: The loss function L is a weighted sumof a label loss Ll (the first stage) and a grasp loss Lg (thesecond stage), formulated as
L = φl Ll + φg Lg ,(15)

where Ll is the focal loss [31] for the label grid classificationand φl and φg are weights of two losses, respectively. The
grasp loss is represented as
Lg = ηb Lb + ηa La + ηw Lw + ηq Lq ,(16)

where Lb , La , Lw , Lq are losses from the predictions ofbaseline vectors, approach vectors, opening widths and grasp
qualities at the sampled contact points, respectively andηb , ηa , ηw , ηq are corresponding weights.
Baseline vector and approach vector loss functions are
their negative log-likelihood functions. The opening width
loss Lw is minimum distance between predicted contact
point pair to its true contact point pair. For a better training
convergence, we use the estimated baseline vector ν̂c tocompute predicted approach vectors and contact pairs, which
is an average over baseline vectors of neighbour grasp labels.
Moreover, Lq is the L1 loss function.
C. Training & Inference
We employ a 643 normal grid input with a voxel sizeof approximately 9 mm. Our neural network is trained witha batch size of 1, a learning rate initialized at 10−3 anddecayed exponentially to 10−5 at the training’s end. The
MLP in the second stage starts training after 100 iterationsof U-Net training. During training, we randomly sample
a contact point within each voxel designated as a contact
region. For inference, each contact region voxel is evenly
subdivided into 8 sub-voxels. For each contact point, We
generate 18 approach vectors. We define the rotation angle
range as [ π2 , 3π
2 ] to ensure that all approach vectors pointdownward. To compute the loss, we assign weights of 10,
5, and 0.1 to the label grid classes (contact space, gripper
space, and empty space) for computing the label loss. In thegrasp loss, we set the weights for Lb , La , Lw , and Lq as 1,
0.01, 0.1, 10, and 0.1, respectively. During training, we label
each predicted contact point by searching for a maximum of
16 true contact points within a 3 mm radius. We compute
Lb , La , and Lw only for contact points that have at least
one true neighbor grasp sample.
D. Data Generation
We create a synthetic 6-DoF grasp dataset to train ourmodel, which contains diverse bin picking scenes along withcorresponding dense grasp labels. We generate a total of 1000
training scenes, each accommodating 1 to 20 objects. On average, each graspable object within these scenes is annotatedwith 470 grasps. The simulator used for dataset generation is
PyBullet [32]. Scenes are simulated by randomly dropping
objects into the bin in a simulation environment. We select200 objects from the YCB [33], Google Scanned Objects(GSO) [34] and ABC [35] datasets, splitting them randomly
into training (149 objects) and testing (51 objects) sets.
Objects are also randomly scaled during scene generation.
Two different bin sizes are used for dataset creation. For
each scene we render a top-down view depth images with aresolution of 640x480. We add noise to the rendered images
in simulation using the same additive noise model appliedin [6]. Afterwards, we generate grasp labels for each scene
following the pipeline proposed in [20]. The difference isthat we kept all valid grasps that are collision-free and havea high antipodal quality (above 0.5) for each scene.
V. EXPERIMENTS
We conducted simulated and real-robot experiments toevaluate the performance of our grasp prediction model.
A. Simulated Experiments
Simulated experiments were executed on hardware comprising an NVIDIA Corporation GP107GLM and an Intel(R)Core(TM) i7-8850H CPU. A standard Franka parallel-jaw
gripper was employed. To generate diverse test scenarios, wefollowed the training scene generation pipeline. We created
300 test scenes, evenly distributed among easy, medium
and challenging scenarios, accommodating up to 5, 15,
and 35 objects, respectively. Easy scenes featured objects
with simple structures (box, ball), whereas medium andchallenging scenarios contain all objects from the test set.
Additionally, we add noise to challenging depth images,
replicating the conditions of generating training images.
The dimensions of the bin model in each test scene were
600 × 400 × 280 mm3 . Performance was evaluated basedon a PyBullet [32] simulation using the following metrics
averaged over 100 scenes in each scenario:• Success Rate (SR): The ratio of successful grasps.
• Clearing Rate (CR): The ratio of objects removed.
In each round, a grasp is executed if its antipodal quality
and collision-score are greater than 0.5, and no collision at

its pre-grasp pose. We test grasp configurations in a order ofhigh to low antipodal quality. For each grasp configuration,
we test first 8 out of 18 approach directions according totheir collision-scores until find an executable grasp. A roundends when the bin is empty, no grasp is detected, or threeconsecutive failed attempts occur.
1) Baselines: We compare our approach with the threebaseline methods:• GPD [17]: a two-stage 6-DoF grasp detection algorithm
that generates a large set of grasp candidates sampled
based on an input point cloud and evaluates each ofthem using a neural function.
• VGN [5]: a 3D convolutional neural network takes a
TSDF grid as input and outputs grasps parameters at
each voxel centers.
• GIGA [6]: a two-stage network that jointly detects
6-DoF grasp poses and reconstructs the 3D scene givena single-view TSDF grid.
GIGA and VGN were retrained on our dataset. To thisend, we label each voxel containing at least one grasp
center with the grasp orientation having the highest antipodal
quality. We use the same top-down view depth images totest these baseline methods. We additionally compare theperformance to two ablated versions of our methods: ”wo κ+ 1app and ”w κ + 1app”. For the first one, we trained our
network to learn the optimal baseline vector by maximizing
cosine similarity between each predicted baseline vector andthe average vector over its neighbor ground truth baseline
vectors, without considering the uncertainty κ. The second
one is trained with the uncertainty learning. However, duringexperiments, we only return the optimal approach direction
for each contact point.
2) Results: Our experimental results (Table I) across 100
test scenes in three scenarios demonstrate our method’s
robustness in managing cluttered and noisy input. Even in
the most challenging scenario, it achieves 70% success rate
and 83% clearing rate, outperforming the baseline methods.
Even though ”w κ + 1app” only executes a single approachdirection, it still generates more accurate and denser grasps
than GIGA and VGN (see Fig. 5). Additionally, we compare between models trained with and without uncertainty
learning for the baseline vector prediction, specifically ”w
κ + 1app” and ”wo κ + 1app”. Our observations indicate
a significant performance gap between the two, particularly
in transitioning from easy to challenging scenarios. Thissuggests that uncertainty learning is crucial for improving
grasp prediction accuracy and making the network morerobust in noisy and cluttered input situations. In Fig. 5, we
also present an example of the uncertainty prediction, which
we can see that the model predict low uncertainty values(yellow points) to contact regions that are less clutteredand more easier to approach. Lastly, we also observe themulti-approaches grasp prediction (8app) effectively handles
collisions that occur when the gripper approaching the target
object without prior motion planning. Importantly, this capability was not explicitly trained but emerges in practice. Thisis also reflected by the higher SR and CR when employing

Fig. 5: Predicted grasps from GIGA (left), ours (middle), andour uncertainty prediction (right) on a medium test scene.

multi-approach grasps in medium and challenging scenarios,
in comparison to using single-approach grasps. In terms ofruntime, our model has an inference time of approximately
236 ms on our machine, while GIGA requires about 233 ms,
and VGN about 136 ms. The runtime for GPD varies withdifficulty levels, taking around 373 ms, 496 ms, and 638 ms
in three different scenarios.
MethodGPD [17]
VGN [5]
GIGA [6]Ours (wo κ + 1app)
Ours (w κ + 1app)
Ours (w κ + 8app)

Easy
SR (%) CR (%)
45.0
36.6
58.4
51.9
64.1
59.1
71.0
93.6
81.0
96.6
87.4
97.8

MediumSR (%) CR (%)
33.5
11.1
48.4
27.0
57.8
37.9
59.5
73.5
66.0
82.5
75.8
91.5

Challenging
SR (%) CR (%)
20.7
4.1
39.8
12.0
54.7
32.6
49.9
43.8
62.8
72.3
70.5
83.0

TABLE I: Averaged success rate (SR) and clearing rate (CR)over 100 test scenes in easy, medium and challenging
scenarios. Bold text indicates the optimal performance.

B. Real Robot Experiments
1) Experiment Setup: In the real robot experiments, weused a 7-axis Franka Emika Panda robot equipped a Franka
parallel-jaw gripper with custom silicon fingers (see Fig. 6).
A RealSense D415 RGBD camera with a resolution of
1920×1080 is statically mounted above a source bin with an
distance of around 0.6 m for top-down view image capturingof the bin picking scene. The test object portfolio consists of
10 diverse objects including boxes, tubes, industrial objects,
textiles, a USB cable and a screw driver. In each round, weplace all test objects in the source bin an perform a manual
mixing to generate random object poses within the bin.
Additionally, we remove objects close to the bin walls thatwould not be graspable due to collisions between the usedgripper and the bin and drop them manually from a random
position above the bin. We use the same model weights
and network configuration as in the simulated experiments.
During the experiments, we determine the final grasp among
all grasp predictions by selecting the grasp configuration withthe highest grasp quality score. Then we select the grasp
with the lowest collision score within the previously selected
grasp configuration. In case of collisions between gripper
and bin or objects during a grasp attempt, we first manuallystop the robot and then consider the next highest predictedcollision-score for the selected grasp configuration. With thissetting, we run 10 rounds in total. A round stops if we had
3 consecutive failure grasps.

Fig. 6: Experimental setup with robotic arm, parallel-jaw
gripper, overhead RGBD camera and bins (left), diverse
object portfolio (middle), successful grasp (top right), failure
grasp (middle right), collision grasp (bottom right).

2) Results: The robot experiments result in a clearing rateof 90% and a success rate of 77% with a total numberof 116 grasp attempts and 90 successful grasps. We also
encountered 4 double grasps where two objects were grasped
at once during a grasp attempt. In this case, the grasp
counts as a failed attempt and we dropped the objects in thesource bin from a random position above the bin. The results
show a similar performance as in the simulated experiments
without any adjustments of the trained model which indicatesthe robustness to noisy image input data due to the grasp
distribution modeling.
VI. CONCLUSIONSIn this work, we considered the problem of 6-DoF grasp
detection for cluttered and unknown objects located inside abin. We presented a novel approach to predict dense andcollision-free grasps with multiple orientations per grasp
point based on two-stage network architecture. The approach
includes a new method for grasp distribution modelingbased on Power Spherical distributions that accounts forgrasp orienation uncertainty and enables a training based on
multiple orientations per grasp contact. Hence, the network
can be trained with more potential ground truth grasps andallows to predict more feasible grasps which is essential
when grapsing with limited reachability inside a bin. Using
only a single-view depth image we can predict dense andcollision-free grasps on cluttered object scenes including
multiple grasp orientations at grasp point. We evaluatedour approach with simulated and real robot experiments ondiverse object types resulting in a clearing rate of around90%. We also showed that we outperform state of the art
approaches for 6-DoF grasp prediction in bin picking tasks.
Future work could consider diverse gripper designs in thedataset generation or the network architecture to enable theusage for different robot setups. Moreover, the approach
might be combined with additional robot pushing skills
or advanced grasp selection methods to further increase
the object clearing performance in challenging bin picking
scenarios.

R EFERENCES[1] J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea,
and K. Goldberg, “Dex-net 2.0: Deep learning to plan robust grasps
with synthetic point clouds and analytic grasp metrics,” 2017.
[2] D. Morrison, P. Corke, and J. Leitner, “Closing the Loop for Robotic
Grasping: A Real-time, Generative Grasp Synthesis Approach,” inProc. of Robotics: Science and Systems (RSS), 2018.
[3] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contactgraspnet: Efficient 6-dof grasp generation in cluttered scenes,” in 2021
IEEE International Conference on Robotics and Automation (ICRA).
IEEE, 2021, pp. 13 438–13 444.
[4] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deephierarchical feature learning on point sets in a metric space,” Advancesin neural information processing systems, vol. 30, 2017.
[5] M. Breyer, J. J. Chung, L. Ott, S. Roland, and N. Juan, “Volumetricgrasping network: Real-time 6 dof grasp detection in clutter,” in
Conference on Robot Learning, 2020.
[6] Z. Jiang, Y. Zhu, M. Svetlik, K. Fang, and Y. Zhu, “Synergies
between affordance and geometry: 6-dof grasp detection via implicit
representations,” Robotics: science and systems, 2021.
[7] J. Cai, J. Cen, H. Wang, and M. Y. Wang, “Real-time collisionfree grasp pose detection with geometry-aware refinement using highresolution volume,” IEEE Robotics and Automation Letters, vol. 7,
no. 2, pp. 1888–1895, 2022.
[8] K.-Y. Jeng, Y.-C. Liu, Z. Y. Liu, J.-W. Wang, Y.-L. Chang, H.-T. Su,
and W. Hsu, “Gdn: A coarse-to-fine (c2f) representation for end-to-end
6-dof grasp detection,” in Conference on Robot Learning. PMLR,
2021, pp. 220–231.
[9] H. Tian, K. Song, S. Li, S. Ma, J. Xu, and Y. Yan, “Data-driven robotic
visual grasping detection for unknown objects: A problem-oriented
review,” Expert Systems with Applications, vol. 211, p. 118624, 2023.
[10] K. Kleeberger, R. Bormann, W. Kraus, and M. F. Huber, “A survey
on learning-based robotic grasping,” Current Robotics Reports, vol. 1,
pp. 239–249, 2020.
[11] M. Gou, H.-S. Fang, Z. Zhu, S. Xu, C. Wang, and C. Lu, “Rgb matters:
Learning 7-dof grasp poses on monocular rgbd images,” in 2021 IEEE
International Conference on Robotics and Automation (ICRA). IEEE,
2021, pp. 13 459–13 466.
[12] H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet-1billion: A largescale benchmark for general object grasping,” in Proceedings of theIEEE/CVF conference on computer vision and pattern recognition,
2020, pp. 11 444–11 453.
[13] C. Wang, H.-S. Fang, M. Gou, H. Fang, J. Gao, and C. Lu, “Graspness discovery in clutters for fast and accurate grasp detection,” inProceedings of the IEEE/CVF International Conference on Computer
Vision, 2021, pp. 15 964–15 973.
[14] B. Zhao, H. Zhang, X. Lan, H. Wang, Z. Tian, and N. Zheng, “Regnet:
Region-based grasp network for end-to-end grasp detection in point
clouds,” in 2021 IEEE International Conference on Robotics andAutomation (ICRA). IEEE, 2021, pp. 13 474–13 480.
[15] W. Wei, Y. Luo, F. Li, G. Xu, J. Zhong, W. Li, and P. Wang, “Gpr:
Grasp pose refinement network for cluttered scenes,” in 2021 IEEE
International Conference on Robotics and Automation (ICRA). IEEE,
2021, pp. 4295–4302.
[16] Q. Dai, Y. Zhu, Y. Geng, C. Ruan, J. Zhang, and H. Wang, “Graspnerf:
Multiview-based 6-dof grasp detection for transparent and specular objects using generalizable nerf,” in 2023 IEEE International Conferenceon Robotics and Automation (ICRA). IEEE, 2023, pp. 1757–1763.
[17] A. Ten Pas, M. Gualtieri, K. Saenko, and R. Platt, “Grasp pose
detection in point clouds,” The International Journal of Robotics
Research, vol. 36, no. 13-14, pp. 1455–1473, 2017.
[18] M. A. Riedlinger, M. Völk, K. Kleeberger, M. U. Khalid, and
R. Bormann, “Model-free grasp learning framework based on physicalsimulation,” in International Symposium on Robotics (ISR). Munich,Germany, 2020.
[19] J. Cai, J. Su, Z. Zhou, H. Cheng, Q. Chen, and M. Y. Wang,“Volumetric-based contact point detection for 7-dof grasping,” in 6th
Annual Conference on Robot Learning, 2022. [Online]. Available:
https://openreview.net/forum?id=SrSCqW4dq9[20] C. Eppner, A. Mousavian, and D. Fox, “Acronym: A large-scale grasp
dataset based on simulation,” in 2021 IEEE International Conferenceon Robotics and Automation (ICRA). IEEE, 2021, pp. 6222–6227.
[21] A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational
grasp generation for object manipulation,” in International Conference
on Computer Vision (ICCV), 2019.

[22] H. Liang, X. Ma, S. Li, M. Görner, S. Tang, B. Fang, F. Sun, andJ. Zhang, “Pointnetgpd: Detecting grasp configurations from point
sets,” in 2019 International Conference on Robotics and Automation(ICRA). IEEE, 2019, pp. 3629–3635.
[23] H.-S. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, W. Liu, Y. Xie,and C. Lu, “Anygrasp: Robust and efficient grasp perception in spatial
and temporal domains,” IEEE Transactions on Robotics, 2023.
[24] B. Curless and M. Levoy, “A volumetric method for building complex
models from range images,” in Proceedings of the 23rd annual
conference on Computer graphics and interactive techniques, 1996,
pp. 303–312.
[25] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim,
A. J. Davison, P. Kohi, J. Shotton, S. Hodges, and A. Fitzgibbon,“Kinectfusion: Real-time dense surface mapping and tracking,” in2011 10th IEEE international symposium on mixed and augmented
reality. IEEE, 2011, pp. 127–136.
[26] Q.-Y. Zhou, J. Park, and V. Koltun, “Open3d: A modern library for
3d data processing,” arXiv preprint arXiv:1801.09847, 2018.
[27] R. Newbury, M. Gu, L. Chumbley, A. Mousavian, C. Eppner, J. Leitner, J. Bohg, A. Morales, T. Asfour, D. Kragic, et al., “Deep learning
approaches to grasp synthesis: A review,” IEEE Transactions on
Robotics, 2023.
[28] N. De Cao and W. Aziz, “The power spherical distrbution,” Proceedings of the 37th International Conference on Machine Learning,
INNF+, 2020.
[29] K. Lee, J. Zung, P. Li, V. Jain, and H. S. Seung, “Superhumanaccuracy on the snemi3d connectomics challenge,” arXiv preprint
arXiv:1706.00120, 2017.
[30] A. Wolny, L. Cerrone, A. Vijayan, R. Tofanelli, A. V. Barro, M. Louveaux, C. Wenzl, S. Strauss, D. Wilson-Sánchez, R. Lymbouridou,
et al., “Accurate and versatile 3d segmentation of plant tissues atcellular resolution,” Elife, vol. 9, p. e57613, 2020.
[31] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss
for dense object detection,” in Proceedings of the IEEE international
conference on computer vision, 2017, pp. 2980–2988.
[32] E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” http://pybullet.org,
2016–2021.
[33] B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M.
Dollar, “The ycb object and model set: Towards common benchmarks
for manipulation research,” in 2015 international conference on advanced robotics (ICAR). IEEE, 2015, pp. 510–517.
[34] L. Downs, A. Francis, N. Koenig, B. Kinman, R. Hickman, K. Reymann, T. B. McHugh, and V. Vanhoucke, “Google scanned objects:
A high-quality dataset of 3d scanned household items,” in 2022
International Conference on Robotics and Automation (ICRA). IEEE,
2022, pp. 2553–2560.
[35] S. Koch, A. Matveev, Z. Jiang, F. Williams, A. Artemov, E. Burnaev,
M. Alexa, D. Zorin, and D. Panozzo, “Abc: A big cad model dataset forgeometric deep learning,” in Proceedings of the IEEE/CVF conferenceon computer vision and pattern recognition, 2019, pp. 9601–9611.