Implicit Grasp Diffusion: Bridging the Gap between Dense Prediction and Sampling-based Grasping
Authors: Pinhao Song
Pengteng Li
Renaud Detry
Venue: CoRL 2024 (2024)
Keywords: Grasping
Implicit Neural Representations
Diffusion Models
TL;DR: Using diffusion models to generate grasps based on local features

Abstract: There are two dominant approaches in modern robot grasp planning: dense prediction and sampling-based methods. Dense prediction calculates viable grasps across the robot’s view but is limited to predicting one grasp per voxel. Sampling-based methods, on the other hand, encode multi-modal grasp distributions, allowing for different grasp approaches at a point. However, these methods rely on a global latent representation, which struggles to represent the entire field of view, resulting in coarse grasps. To address this, we introduce \emph{Implicit Grasp Diffusion} (IGD), which combines the strengths of both methods by using implicit neural representations to extract detailed local features and sampling grasps from diffusion models conditioned on these features. Evaluations on clutter removal tasks in both simulated and real-world environments show that IGD delivers high accuracy, noise resilience, and multi-modal grasp pose capabilities.

Implicit Grasp Diffusion: Bridging the Gap between Dense Prediction and Sampling-based Grasping
Pinhao Song', Pengteng Li,,, Renaud Detry
KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics KK Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images, *HKUST (GZ), AI Thrust (pinhao. song, renaud detry @kuleuven. be pengteng li@connect. wwuss-gz...... edu. cn
Abstract: There are two dominant approaches in modern robot grasp planning: dense prediction and sampling-base based methods. Dense prediction calculates viable grasps across the robot's view but is limited to predicting a fixed number of grasps per voxel. Sampling- distributions, allowing for different grasp approaches methods rely global latent representation, which struggles to represent the en- tire field of view, resulting in coarse grasps. To address this, we introduce Implicit Grasp Diffission (IGD), which combines the strengths of both methods by us- ing implicit neural representations to extract detailed local features and sampling grasps from diffusion models conditioned on these features. Evaluations on clut- ter removal tasks both simulated and real-world environments show that IGD delivers high accuracy, noise resilience, and multi-modal grasp pose capabilities. Our code is freely available at https: htt///gtlala giillab.... kuleuven. be be/detry-lab/ public/amplicit-geeen graplliiitttrr-----e wrappdt-t-ga---dfffsiioo git.
methods, on the other hand, encode multi-modal grasp
point. However, these
Keywords: Grasping, Implicit Neural Representations, Diffusion Models
Introduction
Grasping is the most basic activity that allows a robot to have an effect on its environment - by definition, a robot's primary purpose. Grasping is unfortunately an exceptionally challenging task. The robot must compute 6-DoF gripper poses that yield a stable bond with an object, based on incomplete visual information obtained from a single (color or depth) image of cluttered scene, where at least half of object surfaces are self-occluded or occluded by neighboring objects. From this sparse view, the robot needs to understand the scene's geometry, object properties, and relationship to the shape of its gripper.
Today's ' solutions to this problem can be separated into two main categories: dense prediction meth- ods [1. 2, 3] and sampling-bast based methods [4. 5]. For dense prediction methods, the model discretizes the scene in units (pixel [2], voxel [1], or point [3]), then generates fixed number of grasps for each unit by regressing feasible gripper orientations the unit's position, based local geometry. By nature of this discretization. these methods can easily parse large scenes containing many objects. Unfortunately, a fixed number of grasp predictions per unit is a poor approximation of grasping: point on an object can often he grasped via many gripper orientations. Capturing this multi-modality essential to upstream processes that need to reconcile grasping with reachability or obstacle avoid- ance.
Sampling-baas based methods use heuristic rules [6, 7] or generative models [4. 5] to sample grasps. For the latter type, they encode a region of interest into a latent feature and use generative models such as VABs or diffusion models to sample grasps within the region. In principle, this approach provides
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

strong multi-monn modality and good approximation of the real world, since no discretization is applied. Unfortunately, they are expensive and difficult to train, and struggle with scenes that contain multiple objects: because the information encoded by the latent feature is global, when the region of interest contains multiple objects, the model misses local subtleties and generates inadequate grasp poses. Early works [4, 5] were limited cases where single object is shown in the scene. As shown in [8], the performance of SE(G)-DiffusionFieldd [5] achieves acceptable performance only after segmenting each object's 's point cloud and processing each object separately.
We propose to bridge the gap between the two approaches discussed above with a model named Implicit Grasp Diffiusion (IGD), which captures strengths from both sides:
(i Local geometry: Instead of encoding the entire scene into global feature sampling methods do, IGD leverages implicit neural representations [9] to query local features at certain positions. These local features prioritize local geometry information, which proves advantageous in generating valid grasps.
By contrast the discretization operated by dense prediction methods, IGD samples grasps from continuous domain, via implicit neural representations.
(ii) Continuous sampling domain:
(iii) Multi-modal representation: We compute grasp orientations via diffusion at grasp locations, enabling the generation of multiple orientations at one location.
To realize these capabilities, we present two main technical contributions:
1. Deformable attention in implicit space. Feature relevance at grasping point cannot be reduced to proximity alone. Instead, we utilize a Deformable Attention Module to dynamically sample the implicit feature space, determining relevance based on local geometry.
Two-stage probabilistic grasp evaluator. The large imbalance between valid and invalid grasps poses challenges for filtering the diffusion model's grasp suggestions. We propose two-stage ap- proach: Affordance Evaluator followed by Grasp Classifier. To capture grasp-relevant features, we introduce grasp-conditioned Deformable Attention Module, enabling the model to maintain cquivariance to 3D scene translations and rotations.
We conduct experiments on a clutter-removal task both in simulation and on physical hardware. IGD outperforms dense prediction methods (VGN [1], GIGA [10], GraspNet- Ibillion Baseline [11], and GSNet (12]) and sampling-hase based methods (GPD [6], 6DaF-GrsppNet [4), and SEC3)-DiifuusionFiell [5])
2 Related Works
6-Dof grasping methods. Recent work 6-Dof grasping has the emergence of two families
of methods: sampling- based methods [4, 5, 6] and dense prediction methods [1, 3, 10]. In Section 1, we have already discussed the advantages and disadvantages of both frameworks. In summary, the locality and discretization of dense prediction methods reduce the training difficulty but sacrifice representational capability, while the globality and multi-modality of sampling- based methods lead to slow and poor convergence. In this work, the proposed IGD combines the complementary advan- tages of campling-basedd and dense prediction methods. IGD focuses on local geometry and captures the multi-modal distribution of grasps, which makes work well in cluttered scenes.
Diffusion models. Diffusion models [13] are trained to reverse the process of adding noise to data, effectively teaching the model to reconstruct the original data from progressively noisier versions through a series of denoising steps. Detailed explanations of diffusion models can be found in our supplementary material. Recent advances in computer vision [14, 15, 16] show diffusion models produce high-quality images without the drawbacks of mode collapse and unstable training. Due to its strong multi-mode expressiveness, this paradigm has also been applied the robotics field to generate grasps [5], policies [17, 18], and trajectories [19, 20J. Our IGD model leverages diffusion

---

TSDF
3D Feature Grid
Projected 2D Feature Grids
Tri- plane Feature Grids c
Coms
Apgregation
20 Ne
Booe Okiorazion Diftwor
On anraaaer Dituser
Tri- plane Feature Grids
Grasp Canter (k,gur)
eije
DAM
Two ttage Protobilntic Grosp Evaliator
Wicth Fredicto"
<(p>
Agg rogotod Fecture!
Feature
Acgiggen
Oиuuaany Predictor
(F
Query
Figure I: IGD workflow. The scene is captured by TSDF obtained from depth image. A 3D convolutional network extracts features from the TSDE, and the obtained features are projected onto three canonical planes, then aggregated into tri-plane feature grids. Then, Grasp Orientation Diffuser samples grasp orientations conditioned on the aggregated feature queried at the grasp center. Finally, Two-stage Probabilistic Grasp Evaluator estimates grasp quality.
to generate grasps, endowing it with an inherent capacity to probabilistically encode multiple hand- approach modes around one grasp location.
Implicit Neural Representations. Implicit neural representations (INRs) have shown remarkable capabilities in modeling 3D object shapes, synthesizing scene surfaces, and capturing complex struc- tures [21, 22, 9]. INRs use MLPs to map spatial coordinates scene attributes, enabling smooth and continuous representation of shapes in high resolution. INRs eliminate the need for discretiza- tion by effectively mapping continuous indices to corresponding data, such as magnifying images for super-resolution using magnification scale as an index [23] or querying video frames con- tinuous time [24]. In robotics, INRs play a crucial role: [25, 26] use INRs to map gripper poses grasp distances, guiding the gripper towards valid grasp poses, while GIGA [10] uses INRs to learn geometry and affordance jointly for acquiring grasp poses at query positions. In this work, the proposed IGD follows GIGA and uses INRs to eliminate spatial discretization. Combined with the multi-modal sampling of diffusion models, IGD can theoretically sample all the potential grasps in workspace.
3 Implicit Grasp Diffusion
Implicit Grasp Diffusion (IGD) models the distribution p(r, w/p), capturing gripper orientations and gripper widths conditioned on grasp center point p E RR3. As shown in Fig. 1, IGD comprises a Grasp Orientation Diffuser (GOD) that samples a grasp
(p, r) at followed by a Grasp Evaluator that assesses the grasp quality E [0, 1] of and filters out low-quality grasps. Both models are based on a local feature representation of the object's geometry around p, extracted by a Tri-plane Feature Encoder. A Width Predictor then estimates the finger width required for the grasp using the same aggregated feature.
3.1 Tri-plane Feature Encoder
Following GIGA [10], adopt ConvONets [27] the encoder architecture to extract tri tri-plane feature space. The encoder processes a TSDF voxel field with 3D CNN layer, generating fea-

---

ture embedding for each voxel. These 3D features are then projected onto three canonical planes (XY/X2/Y2). A 2D U-Net further refines these feature planes, producing the tri-plane features Cay, Cu. Syz. Given a query position p. we project p to each feature plane and query the local features the projected locations c(p), as:
c(p)
[o(Csy, Psy): (oC Pu)), (Cyys Py:)):
(1)
where Gj. Pij (i.j E x, are the plane features and points projected onto the corresponding plane, and @ represents bilinear interpolation of the feature plane the projected point.
To generate grasp at position we gather features from p's neighborhood by sampling nearby points and querying, then aggregating their features. Crucially, the sampling process is guided by local geometry to focus on areas relevant to the grasp. Inspired by [28], we propose a Deformable Attention Module (DAM, Fig. 2a) to dynamically adjust the receptive field and produce the aggre- gated feature E(p), formulated as:
Q Woe(p), K = Wx(c(p+ Apt)
V= --Wllccp- APK))
(2)
K"Q
ē(p) Wo V Softmax
Vdl
where WQ. WK. K, Wv, Wo learned parameters. Apk is a learned offiet obtained by linear projection from c(p). In effect, ë(p) encodes a local shape descriptor tailored for grasp generation and grasp accuracy evaluation.
3.2 Grasp Orientation Diffuser
Given position p. the Grasp Orientation Diffuser (GOD) models an orientation distribution pa(r) ē(p)) to represent feasible grasps based on the local geometry information encoded in ē(p). During the forward diffusion process, noise is injected into the orientation r (denoted as ro) to obtain
= Vasra + (1 - a.)c,
(4)
where & E (0, I). In the reverse diffusion process, C₁ is approximated with co t) output by noise predictor, which is realized through implicit neural representations, as:
fo ē(p), I) - c,(r, ().
(5)
where fo(-) is MLPs with learned parameters and g (p, r). Thus, To can be reconstructed from pure noise PT iteratively as follows:
1 at
A Pi-1
(Fr
ca (F), + o22,
(6)
Vou
VT-
where z E (O, Through the reverse diffusion process, the predicted grasp 9 = (p. ro) E SE(3) can be obtained. To train the noise predictor, the KL divergence between the forward and backward diffusion processes is minimized, which is equivalent to:
Ladpm
((0r)) [lle - ee( ca(vo + vI - {...
3.3 Two-stage Probabilistic Grasp Evaluator
Not all grasps generated by GOD are satisfactory: (i) Grasps generated far from any object, inside one, should be discarded. (ii) Since grasps produced by GOD are samples, some have a low prob- ability and are inadequate. Consequently, we employ a grasp evaluator to filter out unsatisfactory grasps. However, training grasp evaluator directly challenging due to the severe imbalance be- tween graspable and ungraspable regions, a problem noted similar works [4, 29, 30]. To address this, we propose Two-stage Probabilistic Grasp Evaluator, designed manage the imbalance be- tween positive and negative grasps. We consider the conditional distribution p(r(p) and rewrite it as:
p(rlp) = Σ e
(8)

---

Linear
Scaled Dol-Produst Attention
Scaled Dat Product Attention
Grasp
Lineor
Lineer
X
h
Ap
Local Feature
Anchor Fealures
Control Points
Anchor Points
Faetur
(a)
Figure 2: (a) The architecture of Deformable Attention Module. (b) The architecture of Grasp- conditioned Deformable Attention Module. See text for details.
where affordance a represents the existence of feasible grasps at certain point. p(alp) denotes the probability the existence of feasible grasps position p, and p(rla, p) denotes the feasibility probability of the grasp orientation given the affordance the grasp evaluator can be separated into two factors: Affordance Evaluator (AE) p(alp) and a Grasp Classifier (GC) p(rla, p). To train the grasp evaluator, we maximize the lower bound of log-likelihood, which is an approximation of maximum likelihood estimation (MLE), as:
the position p. Eq. shows that
log(p(r(p)) = log(
Σ
p)p(a)))())
aff0.17
Σ
log(p(rla, p)) + log(p(alp)) (Jensen's 's inequality)
(E(0,,)
The lower bound of the log-likelihood can be regarded as summation the fog-likelihood of AE and GC. Maximizing the lower bound reduces the original goal to independent maximum -inellioio objectives for the first and second stages respectively. In our implementation, implicit neural repre- sentations are used to realize AE and GC.
Affordance Evaluator (AE). We query the grasp center p and the aggregated feature ē(p) to obtain the affordance, as:
(p, ē()) + a [0, 1],
(10)
where fu implemented by several residual fully-connected blocks. AE is trained with the follow- ing cross-entropy loss:
C. = lap + (1 ap) log(1 ap)),
(11)
where åp the predicted affordance, while ap the ground-truth affordance.
Grasp Classifier (GC), We query the grasp and the aggregated feature ē(p) to obtain the grasp score v, as:
fg c(p) + u [].
(12)
To implement Eq. 12, we propose a Grasp-conditioned Deformable Attention Module (Grasp DAM, illustrated Fig. 2b) extract feature for grasp classification. We first define a set of gripper- relative learnable control points u1 uzg order to express these points in the scene's 's base frame, transform them with transformation T defined by g. The feature for grasp classification can be obtained by applying deformable attention to the features of control points,
Q' = Woc(p), K' = W (C(TTTT))))
V Wo
(13)
KTQ' Q'
h W' Softmax
(14)
dd
5

---

where W W WV, Wo are learned parameters. Several residual fully connected blocks are applied h compute grasp score However, due to the sparsity of feasible grasps, GC trained with cross-entropy of infeasible grasps. To alleviate this problem, a large set of negative grasps are randomly sampled from the scene and a focal loss is used to extract valuable information from these negative examples, as:
balanced positive-negative dataset often overestimates the quality
{ {
a(1 (gg) (gg( (1 a) log(1 fg),
0gg 1,
(g(0g)
(15)
0gg
where 6g is the predicted grasp score, while llg is the ground-truth grasp label. a is the balance parameter, and is the focus parameter. The sampling of control points in GraspDAM is SE(3)- equivariant because the geometric relation between the scene and grasp remains fixed, the relation between transformed control points and the scene will be consistent. This contrasts with a straight- forward alternative implementation, where sampling points are generated by neural networks with the input of the grasp and aggregated feature. The sampling equivariant limits the search space and improves the grasp feature extraction consistency, which improves the training stability. Based on Eq. 8, ppr(pp()))) (p) p(rl((rr 1, p)p(a 1(p) as p(rla 0, p) O. Thus, we can obtain the final grasp quality as 9 = u. In the paper's supplementary material, ablation studies demonstrate that the Two-stage Probabilistic Grasp Evaluator has benefits both training and inference.
3.4 Network Training
The training loss of the proposed IGD consists of two parts: the grasp loss and the geometry loss, as [ Cgrasp + Loce. The total grasp loss can be formulated as Cgrap Ladpm + C. + L.g + Lwo where Lospm. La. and Lg are the loss for training the GOD, AE, and GC, respectively. The width loss L and the geometry loss foxe are obtained by calculating the 1.2 distance and the cross-entropy loss between predictions and ground truths, respectively, following the paradigm of GIGA [10].
4 Experiments
4.1 Experimental Setup
We follow the same experimental setup as GIGA [10]. The model is trained with GIGA's open- source data [10]. We implement our method on Franka Research 3 robot arm in both simulation and real-world environments.
Our simulated environment is built using PyBullet, featuring free- floating gripper that samples grasps within tabletop workspace measuring 30 30 30 cm3 For fair comparison, we employ the same object assets VGN [1] and GIGA [10]. including 303 training and 40 test objects from various datasets [31, 32, 33, 34]. The model is validated in two types of simulated scenes: pile and packed. In the pile scene, objects are randomly dropped into box of the same dimensions as the workspace, resulting in cluttered pile once the box removed. The packed scene features a subset taller objects placed at random locations on the table in their canonical pose. single-view depth map serves as the model's input for grasp generation. The process involves repeatedly predicting and executing a grasp, followed by removing the grasped object from the workspace until one of three conditions is met: all objects cleared, two consecutive failures occur, simulation rounds using 5 different random seeds.
Simulation Environment:
grasp is detected. Performance metrics are averaged over 100
In real-world experiments, 15 rounds of experiments are performed for both the packed and pile scenes, respectively. Everyday objects are used to conduct the experiments (see supplementary material). In each round, objects are randomly selected and placed on the table. In each grasp trial, pass the TSDF point cloud from side-view depth camera to the model and execute the physically feasible grasp with the highest score.
Real-world Environment:

---

Table 1: Quantitative results of clutter removal. We report the mean and standard deviation of GSR and DR. N denotes sampling rounds in IGD. The best performances are highlighted in bold. Our method shows equal or better performance than all other methods, with competitive latency.
Packed
Pile
Method
Latency (ms)
GSR (%)
DR (%)
GSR (%)
DR (%)
VGN [1]
72.5:
76.7
59.3+2.9
43,
GIGA [10]
84.8=2.2
85.1
69.5
49.0+3.4
17
GraspNet-1B Baseline [1I]
49.9-2.3
40.1+2.2
50.2+44.2
30.0+2.3
73
GSNet [12]
67.8-2.5
60.113.2
58,313.8
51.31:,66
149
GPD [6]
41.8 2:
34.11:3.4
22.7.11.1
9.0
2138
GraF-GrapNeetttc [4]
179:1088
11.91009
15.5.12.9
6,91111
2220
SE(3)-Dif [5]
7.2+1.5
1.3+1.0
7.6+1.8
3.0=0.8
5643
IGD (Ours, N=l)
92.9-1188
86.7+11.8
68.2+
50.6+
217
IGD (Ours, N=11)
91.2=0.9
88.8+11.5
718+22.2
55.7+2.66
1823
Table 2: Quantitative results of clutter removal in the real-world experiment. We report GSR, DR, successful grasp numbers, and total grasp trial numbers (in brackets). The best performances are highlighted in bold. Our method outperforms VGN and GIGA on both scenes and both metrics.
Packed
Pile
Method
GSR (%)
DR (%)
GSR (%)
DR (%)
VGN [1]
77.3 (58/71)
81.7
65.3 (47/72)
62.7
GIGA [10]
81.3 (65/80)
86.7
77.4 (65/84)
86.7
IGD (Ours)
88.3 (68/77)
90.7
82.7 (67/81)
89.3
Metric: (i) Grasp Success Rate (GSR # successffll grasps
#total grasps that measures the ratio of successful grasps
to total grasps; (ii) Declutter Rate (DR =
# grasped objects
#total objects that measures the ratio of objects removed
successfully to the number of total objects presented.
4.2 Training and Inference Details
We implement the proposed IGD with PyTorch and train the models with the Adam optimizer for 12 epochs. An initial learning rate of 10-4 is set. The step learning scheduler leveraged with decay factor set to 0.1, and the scheduler works at the 9th and 1 11 epochs.
The final grasp pose is obtained by sampling from the trained IGD. We discretize the volume of the workspace into 40 x 40 x 40 voxel grids and use the centers of all voxel cells as grasp centers. We then evaluate affordances at grasp centers, filtering out those with low affordances. Then, for each remaining grasp center, we conduct multi-round grasp sampling and retain the grasp with the highest grasp score. Next, we mask out impractical grasps. Finally, grasp qualities are the product of affordances and grasp scores, and the grasp with the highest quality is selected if the quality is beyond the threshold. If no grasp has a quality above the threshold, we declare that there is no feasible grasp in the scene.
4.3 Baselines
We compare our method against seven strong baselines:
Dense Prediction Methods: (i) VGN [1]: Volumetric Grasping Network, which generates large number of grasps in parallel given input TSDF volume. (ii) GIGA [10]: Grasp detection via Im- plicit Geometry and Affordance, which leverages implicit neural representations and geometrical supervision. (iii) GraspNeet-1bllien
Baseline []]] A baseline method training the GraspNet- Ibillion dataset. (iv) GSNet [12]: A model based on GraspNet- 1biliin Baseline with the proposed graspness as a refined grasp quality label.
Sampling-based Methods: (v) GPD [6]: Grasp Pose Detection, which generates large set of grasp candidates and classifies each of them. (vi) 6DoF -frFpGGttpeet [4]: A grasping model based on VAE. (vii) SE(3)-Dif [5]: A grasping model based on SE(3) score-based diffusion models.
7

---

Figure 3: The visualization of the denoising trajectory. Blue squares are the starting points of the denoising process, while red triangles are the endpoints. Green stars are ground-truth grasps. Starting from pure noise, the data converge different ground truths during the denoising process, demonstrating IGD's multi-modal expressiveness.
For all the methods mentioned above, we use their pre-trained models for comparison.
4.4 Grasp Detection Results
We report GSR and DR for different scenes in Table Sampling- methods struggle gen- erate effective grasps in cluttered environments, performing worse than dense prediction methods. IGD outperforms both dense prediction and sampling- based methods with just one sampling round in both packed and pile scenes. In packed scenes, IGD shows an 8.1% improvement in GSR over GIGA while maintaining the same DR level. Increasing the sampling rounds to 11 boosts IGD's DR in packed scenes to 88.8%, exceeding GIGA by 3.7%. In pile scenes, IGD's 's performance matches that of GIGA, and with 11 sampling rounds, it achieves 71.8% GSR and 55.7% DR. The supe- rior performance of IGD results from its integration of the strengths of both sampling- and dense prediction methods: (i) local information enables effective handling of cluttered scenes, and (ii) the strong multi-modal expressiveness of the diffusion model captures grasp distributions. Ta- ble 2 presents grasp detection results from real-world experiments, aligning with simulation findings where IGD outperforms GIGA in both packed and pile scenes. This enhanced performance is partly attributed to IGD's greater robustness in noisy real-world environments, which will be analyzed further in the supplementary material.
4.5 Multi-modal Grasp Modeling
The introduction of diffusion models aims capture the multi-modal distribution of feasible grasps. To validate this, conduct six rounds of diffusion processes various valid points containing feasible grasps and visualize the denoising trajectories using t-SNE, shown in Fig. 3. Each fig- ure displays four ground-truth grasps. The results indicate that starting from pure noise, the data converges to different ground truths throughout the denoising process, demonstrating that IGD ef- fectively captures the multi-modal nature viable grasps.
Conclusion
In this work, we introduce Implicit Grasp Diffusion (IGD), a novel grasping framework that lever- ages the strengths of both dense prediction and sampling-based methods, exhibiting robust locality and multi-modality. Our framework features Grasp Orientation Diffuser and Two-stage Prob- abilistic Grasp Evaluator for regressing and evaluating grasps. We assess IGD in both simulated and real-world environments, comparing it against state-of-the-art approaches. Experimental results highlight IGD's high grasp accuracy, strong noise robustness, and expressive multi-modal capabili- ties.

---

Acknowledgments
This work supported by Interne Fondsen KU Leuven/Internal Funds KU Leuven. We would like thank Yang Chen at Shenzhen BIT-MSU University for providing computational resources and working environment.
References
[1] M. Breyer, J. J. Chung, Ott, R. Siegwart, and J. Nieto. Volumetric grasping network: Real- time 6 dof grasp detection clutter. In Conference on Robot Learning, pages 1602-1611. PMLR, 2021.
[2] D. Morrison, P. Corke, and J. Leitner. Learning robust, real-time, reactive robotic grasping. The International journal of robotics research, (112233:183-222 2020.
[3] Y. Qin, R. Chen, H. Zhu, M. Song, J. Xu, and H. Su. S4g: Amodal single-view single-shot se (3) grasp detection cluttered scenes. Conference on robot learning, pages 53-65, PMLR, 2020.
[4] A. Mousavian, C. Eppner, and D. Fox. 6-dof graspnet: Variational grasp generation for object manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2901-2910, 2019.
[5] Urain, N. Funk, Peters, and G. Chalvatzaki. (3)ddiffusionfields:: Learning smooth cost functions for joint grasp and motion optimization through diffusion. In 2023 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pages 5923-5930. IEEE, 2023.
[6] A. Ten Pas, M. Gualtieri, K. Saenko, and R. Platt. Grasp pose detection in point clouds. The International Journal of Robotics Research, (6(13-14):1455-144 1473, 2017.
[7] Mahler, J. Liang, Niyaz, M. Laskey, R. Doan, X. Liu, A. Ojea, and K. Goldberg. Dex- 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics, arXiv preprint arXiv: 09312, 2017.
[8] H. Ryu, J. Kim, J. Chang, H. S. Ahn, J. Seo, T. Kim, J. Choi, and R. Horowitz, Diffusion-edfs:: Bi-equivariant denoising generative modeling (3) for visual robotic manipulation. arXiv preprint arXiv:2309.02688 2023.
[9] B. Mildenhall, P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[10] Z. Jiang, Y. Zhu, M. Svetlik, K. Fang, and Y. Zhu. Synergies between affordance and geometry: 6-dof grasp detection via implicit representations. arXiv preprint arXiv:2104.01542. 2021.
[11] H.-S. Fang, C. Wang, M. Gou, and Lu. Graspnet-lbilion:: A large-scale benchmark for general object grasping. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 11444 11453, 2020.
[12] C. Wang, H.-S. Fang, M. Gou, H. Fang. J. Gao, and C. Lu. Graspness discovery in clutters for fast and accurate grasp detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15964-15973, 2021.
[13] Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-663 68510,68811110 2020.
[14] P. Dhariwal and A. Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 118880887744 2021.

---

[15] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06/25, 1,2) 2022.
[16] R. Rombach, A. Blattmann, D. Lorenz, Esser, and Ommer. High-resolution image syn- thesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[17] Z. Wang. J. J. Hunt, and M. Zhou, Diffusion policies as an expressive policy class for offline reinforcement learning. In Deep Reinforcement Learning Workshop NeurIPS 2022, 2022.
[18] C. Chi, Feng. Du, Z. Xu, E. Cousineau, Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint WrXiv:2303.04137, 2023.
[19] T. Power, R. Soltani-Zarrin, S. Iba, and D. Berenson. Sampling constrained trajectories us- ing composable diffusion models. In IROS 2023 Workshop on Difserentiable Probabilistic Robotics:
Emerging Perspectives on Robot Learning, 2023.
[20] J. Carvalho, A. T. Le, M. Baierl, D. Koert, and J. Peters. Motion planning diffusion: Learn- ing and planning of robot motions with diffusion models. In 2023 JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 1916-1923. IEEE, 2023.
[21] R. Chabra, J. E. Lenssen, E. llg. T. Schmidt, J. Straub, S. Lovegrove, and R. Newcombe, Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX 16, pages 608-625. Springer, 2020.
[22] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IFFF/CVF conference on computer vision and pattern recognition, pages 4460 4470, 2019.
[23] S. Gao, X. Liu, B. Zeng, S. Xu, Y. Li, X. Luo, J. Liu, X. Zhen, and B. Zhang. Implicit diffusion models for continuous super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10021-10030 2023.
[24] Z. Chen, Y. Chen, J. Liu, X. Xu, V. Goel, Z. Wang, H. Shi, and X. Wang. Videoinr: Learn- ing video implicit neural representation for continuous space-time super-resolution. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2047-2057, 2022.
[25] Weng, Held, F. Meier, and M. Mukadam. Neural grasp distance fields for robot manip- ulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1814-1821. IEEE, 2023.
[26] Y.-C Chen, A. Murali, B. Sundaralingam, W. Yang, A. Garg, and D. Fox. Neural motion fields: Encoding grasp trajectories as implicit value functions. arXiv preprint 2at::2266.18854. 2022.
[27] S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys, and Geiger. Convolutional occupancy networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 520-540. Springer, 2020.
[28] X. Zhu, W. Su, L. Lu, B. Li, Wang, and J. Dai. Deformable detr: Deformable transformers for end-to-end object detection. In International Conference on Learning Representations, 2020.
[29] X. Zhou, X. Lan, H. Zhang, Z. Tian, Y. Zhang, and N. Zheng. Fully convolutional grasp detection network with oriented anchor box. In 2018 IEEE/RS/ International Conference Intelligent Robots and Systems (IROS), pages 7223-7230. IEEE, 2018
10

---

[30] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact- Contact-graspner graspnet: Efficient 6-dof grasp generation in cluttered scenes. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pages 13438-13444 13444. IEEE, 2021.
[31] B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pages 510-517. IEEE, 2015.
[32] D. Kappler, J. Bohg, and S. Schaal. Leveraging big data for grasp planning. In 2015 IEEE international conference robotics and automation (ICRA), pages 4304-4311, IEEE, 2015.
[33] A. Kasper, Z. Xue, and R. Dillmann. The kit object models database: object model database for object recognition, localization and manipulation in service robotics. The International Journal of Robotics Research, 31(8):227-934, 2012.
[34] A. Singh, J. Sha, K. S. Narayan, T. Achim, and P. Abbeel. Bigbird: A large-scale 3d database of object instances. 2014 IEEE international conference on robotics and automation (ICRA), pages 509-516. IEEE, 2014.

---

Supplementary Material of Implicit Grasp Diffusion
Pinhao Song', Pengteng Li", Renaud Detry
KU Leuven, Dept. Mechanical Engineering, Research unit Roboties, Automation and Mechatronics KK Leuven, Dept. Electrical Engineering. Research unit Processing Speech and Images, *HKUST (GZ), AI Thrust
{pinhao. song, renaud. detry @kuleuven. pengteng. li@connect hkust-ga gz. edu. cn
1 Preliminary: Diffusion Models
Diffusion models aim to model an unknown data distribution ((x)) with a parameterized model Po())) The procedure consists of two steps: the forward and the reverse diffusion processes. The forward process iteratively injects small Gaussian noise in By to obtain 01.7:
T II ((xx
(<11T/=0)
where g(xx+((;-1) = Nax vI- 1, 3e1) is the per-step noise injection following variance
schedule B1, .... Br. This leads to the distribution ((0))=)) = Maa v Vārao: (1 (1- ãe) ot) where Oç Br and 0₁ = I Oş Since ae x O, or (O, I). The reverse diffusion learns to denoise the data starting from ør following po(x: (x,) (N pa 1), 3,1) where:
Br
no (ate 1)
co (x 1)).
V
VI-a; at
The parameterized model en (0),() turbations and the noising schedule by the score-matching objective:
() is called the score function, and it is trained to predict the per-
arg min E 200. [||e ea GalVo + vI- -,E 1)]
In particular, such score function represents the gradient of the learned probability distribution as:
V æw log
co( t).
VI- cz.
Experiment Scenes
The objects used the real-world experiment are shown in Fig. la. Fig. lb and Fig. Ic illustrate examples of packed and pile scenes.
Visualization of Grasp Detection
We visualize the top-10 10-score grasps with threshold 0.5 in some challenging cases in Fig. 2. Compared to GIGA, IGD generates collision-free good-quality grasps and gives lower scores to bad-quality grasps.
We also visualize some failure cases of IGD in Fig. 3. There are three main reasons for these failures. First, using diffusion models allows for sampling a large range of orientations, which increases the burden on the grasp evaluator. When the range of feasible orientations limited, such as with large objects (Fig. 3 (a-b)), smooth curved surfaces (Fig. (c-d)), and lying boxes (Fig. (d)), the proposed method is prone to failure. Second, single-view method based TSDF generated
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

(b)
(c)
Figure I: (a) Objects for the real-world declutter experiment. (b) An illustration the pile scene.
(c) An illustration of the packed scene.
(a)
(b)
(c)
(d)
(e)
Grasp visualization in some challenging cases. The first row denotes GIGA, and the second row denotes IGD, (a-c) are in pile scenes, while (d-f) illustrate packed scenes. Green grasps denote successful grasps, while red grasps denote failed grasps.
Figure 2:
from depth map, it struggles to analyze the geometry and borders of occluded objects (as shown in Fig. 3 (e-f)), leading to further failures. Third, IGD uses low-resolution TSDF without the guidance of RGB information, resulting in a loss of subtle geometry details. For example, in Fig. 3 (g), IGD generates grasps between two objects, incorrectly assuming they are connected. In Fig. 3 (h), IGD fails to distinguish between thin-lying objects and the table, resulting in an unsuccessful grasp attempt.
4 Robustness to Different Noises
Table shows the performance of GIGA and IGD in different kinds of noises. According [1], noise depth camera includes the following noises: noise from stereo matching, lateral noise, blur, and noise in depth estimation. We simulate those noises and add them in the depth image to evaluate the performance of models. Our baseline condition is the dex noise environment, where models are trained. According to Table 1, both IGD(N=I) and IGD(N=9) outperform GIGA in different noise conditions. Besides, the performance decrease of IGD also lower than GIGA. The results demonstrate the strong robustness of IGD. The robustness of IGD comes from two factors: (i) Probabilistic two-stage grasp evaluator can precisely estimate the true quality of grasps; (ii) Multi- round sampling increases the chance sample good grasps. From Table 1, increasing sampling rounds decreases performance drop. Due to its strong robustness, IGD performs well in real-world experiments.

---

(a)
(b)
(c)
(d)
(h)
Figure 3: Different failure cases of IGD.
Table I: Robustness experiments. We evaluate the performance under four types of noises: dex, stereo, lateral, blur, and depth. N denotes sampling rounds in IGD. We set dex noise as the baseline calculate the performance decrease (in bracket). The best performances highlighted in red.
Packed
Pile
Noise Type
Method
GSR (%)
DR (%)
GSR (%)
DR (%)
GIGA
84,8:22,2
85.1=2.5
19.5+11.3
49.0:t3.4
Dex (baseline)
IGD (N=l)
92.9+t 1.8
86.7=1.8
68.2+11.9
50.6+:1.5
IGD (N=9)
91.2++0.9
88.8-1.5
71.0++0.7
55.0++1.6
GIGA [2]
72.0.11. 12.8)
78.4.12.6 (3 6.7)
51.811.2 (L 17.7)
47.0.1..3 2.0)
Stereo
IGD (N=1)
83.6:11.8 9.3)
86.412.1 0.3)
60.410099 (.8)
48.6111. ( (.0)
IGD (N=9)
85.6++1.3 (1 5.6)
88.3+ (1 0.5)
61.9++1.5 9.1.
54.8++2.5 () 0.2)
GIGA [2)
75.11114 (1 9.7)
(L.
56.311155 (L 13.2)
53.812.8 4,8)
Lateral
(N=1)
83.711.2 (1 9.2)
86.4218 0.30. 0.3)
63.6:11.5 (L 4.6)
53.24224 2.6)
IGD (N=9)
85.2+1.6 (L 6.0)
87.2
68.6:22.9 (.44
61.2+2.2 (1 6.2)
GIGA [2]
69.612.4 15.2)
72441..2 12.7)
5381199 (1 15.7)
48.242.77 (1 0.8)
Blur
IGD (N=1)
81.9
79.842.2 (2.77
60.912.9 7.3)
43.5#3.1 (L 7.1)
IGD (N=9)
1. (..0)
84.5+177 (14.3)
63.4+2.4 (L
48.4 6.6)
GIGA [2]
75.3+117 (L
(
51.8+188 (7.17)
47.6+2.1 (L 1.4)
Depth
IGD (N=1)
89.940.6 (L 3.0)
58.5+0.8 (. (L
49.2#1.4 1.4)
IGD (N=9)
90.0+ 1.2)
(1
62.1+113 (L.9.)
56.5+3.1 (1 1.5)
5
Ablation Studies
We also conducted extensive ablation studies to validate each module our proposed IGD. Ablation studies were mainly performed in pile scenes because of data abundance and higher task difficulty compared packed scenes. Table shows the ablation studies of IGD. The proposed DAM effec- tively improves the performance in both GSR and DR. Besides, the performance deteriorates with GC alone compared to the AF. Combining the AE and GC achieves the best performance, which demonstrates the effectiveness of the proposed probabilistic two-stage grasp evaluator. To further investigate what brings the performance improvement, we train AE and GC together and deactivate one them in the inference (shown as "GC*" and "AE*" in Table 2). If we deactivate the GC, the performance decreases drastically, while the performance decline from deactivating the AE limited. This result contrary solely training the AE and GC. We can conclude that the per- formance improvement is mainly from the loss supervision instead of simple ensemble of two modules. Negative grasp sampling is also important to train the grasp evaluator. Introducing neg-

---

Table Ablation Studies of the proposed IGD, "DAM" denotes the deformable attention module. "AE" denotes the affordance evaluator. "GC" denotes the grasp classifier. "Neg." denotes the negative grasp sampling for the grasp classifier. "AE*" and "GC*" denote that AE and GC are trained together, but they are used solely in the inference. The best performances are highlighted in red.
DAM
AE
GC
Neg.
AE*
GC*
GSR (%)
DR (%)
59.1
42.8+3.1
62.2+3.0
45.6=2.5
53.5+2.6
42.5+4.0
7.2
33.5+2.0
62.8+3.1
47.9#4.3
68.2+1.9
50.6+1.5
59.5+0.9
47.5=0.9
64.342.3
48.0=3.1
Table 3: Ablation Studies of anchor points in Table 4: Ablation Studies of focal loss. The best DAM. The best performances are highlighted in performances are highlighted in red. red.
7
GSR (%)
DR (%)
Anchor points
GSR (%)
DR (%)
0
0.5
68.5.1..
46.113.9
23
68.211.9
50.2
1
0.25
65.7+
13.6+1144
33
65.7+2..
49.0+2..7
1
0.5
66.5
49.7+114
44
62.3:1..9
46.2
1
0.75
63.242.6
47.5+2.6
2
0.25
66.34..1
46.943.1
2
0.5
68.2+1.9
50.6+1.5
2 0.75
65.243.5
50.412.9
3
0.5
62.713.5
45.813.3
92
70
CSR/ (80)
GSR/ DR (%)
82
G5R
50
G5R
45
13
11
13
Sampling rounds
Sampling rounds
(a)
(b)
Figure 4: The ablation study of sampling rounds. (a) Packed scene. (b) Pile scene.
ative grasp sampling with GC alone largely increases GSR (53.5% to 72.2%) but decreases DR (42.5% to 33.5%). When it comes to probabilistic two-stage structure, negative sampling improves both GSR and DR. The results mean that the grasp evaluator needs to excavate information from negative samples to learn to distinguish feasible grasps in the large grasp space.
Number of sampling rounds. Since can sample multiple rounds to obtain different grasps the same grasp position, ablation study about sampling rounds is conducted to analyze the effect of this hyper--prameee which is shown in Fig. 4. In packed scenes, there is no improvement in GSR

---

Figure 5: The visualization of the sampled points Aps (shown in blue) for the grasp (shown in green) position p. The first column denotes the packed scene, and the second column denotes the pile scene.
as sampling rounds increase, while DR increases in general. contrast, in harder pile scenes, both GSR and DR increase as sampling rounds increase. The results show the ability of IGD to obtain good grasps by increasing grasp samples, which is inherited from sampling- based methods.
Sampled points in DAM. Table 3 shows ablation study sampled points in DAM. Because the anchor points are initialized as the points spatial-uniformly sampled in the cube around the grasp center, we select 23, 33, and 48 sampled point numbers to evaluate their performance. From Table 3, the best performance appears in the 2 setting. We also visualize the sampled points in DAM in Fig. 5. packed scenes, the sampled points are primarily distributed in the region between the grasping pose and the object's surface. Since grasps are generated by the features aggregated by DAM, we believe these sampled features contribute more accurate grasp generation. In pile scenes, in addition to points localized around the grasp area, points are distributed over farther range. This visualization demonstrates that DAM can dynamically sample features based on the local geometry, aiding in accurate grasp generation.
Focal loss hyper-parameters. Table 3 shows an ablation study of focal loss hyper-parameters in GC. ? reduces the focal loss to a normal cross-entropy loss and shows inferior performance. ? is to tune the level of hard example mining, and is balance the weight of positive and negative samples. According to the results, we select the best hyper-parameter setting as 0.5 and = 2.
6 Limitations and Future Work
Although we have demonstrated the effectiveness of IGD in both simulation and real-world systems, there are limitations that future work can improve. First, we can't 't directly obtain the point with the highest affordance value. In GIGA, 40 x 40 x 40 points are sampled to query the grasp to obtain the best grasp. IGD inherits this limitation from GIGA. Second, diffusion models have higher computational costs and inference latency compared dense prediction methods, especially IGD where grasp sampling is separated into two stages: position sampling and orientation sampling. Future work exploit the latest advancements in diffusion model acceleration methods reduce the number of inference steps required, such new noisy schedules [3], inference solvers [4], and consistency models [5]. Third, in our GOD, we directly apply diffusion models to the generation of quaternions. However, quaternion in the diffusion process is not in close form. Current works have proposed a lot of methods to achieve rotation diffusion [6, 7, 8]. Although our effort to apply these works to IGD fails, it still worth exploring.

---

References
[1] T. Mallick, P. P. Das, and A. K. Majumdar. Characterizations of noise in kinect depth images:
A review. IEEE Sensors journal, 1416):173 2014.
[2] Z. Jiang, Y. Zhu, M. Svedik, K. Fang, and Zhu. Synergies between affordance and geometry: 6-dof grasp detection via implicit representations. arXiv preprint arXtv:2104.015422 2021.
[3] T. Chen. On the importance of noise scheduling for diffusion models. arXiv preprint arXiv:2301.1097., 2023.
[4] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[5] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. arXiv preprint arXiv:2303.0166, 2023.
[6] J. Urain, N. Funk, Peters, and G. Chalvatzaki. Se (3)-diffusionnfeels:: Learning smooth cost functions for joint grasp and motion optimization through diffusion. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5923-5930. IEFE, 2023.
[7] A. Leach, S. M. Schmon, M. T. Degiacomi, and C. G. Willcoeks. Denoising diffusion proba- bilistic models on so (3) for rotational alignment. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning, 2022.
[8] Y. Jagvaral, F. Lanusse, and R. Mandelbaum. Unified framework for diffusion generative models in so (3): applications in computer vision and astrophysics. arXiv preprint urXiv:2312.11707 2023.


Reviews:
[{"review_id": "8ZSG8ki0rz", "reviewer": "Reviewer_cuMq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "In this work, the authors consider the task of SE(3) grasp prediction. Noting the difficulty of capturing the multimodaltiy, precision, and sparsity of grasps in a real scene, the authors seek to combine both dense prediction and local sampling approaches in order to fit the underlying distribution. The authors propose a novel deformable attention module to effectively capture local features near a query point, and incorporate this attention module into various implicit functions which are used to sample and score grasps. The authors show that their method outperforms the next closest method, GIGA, as well as various other recent methods for grasp generation.", "summary_of_recommendation": "Overall there is enough novelty in the method - and downstream improvement over baselines - to warrant acceptance. If more details were explored, and failure modes characterized, I might be willing to increase my rating.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "One area that is not well-motivated is the Tri-plane feature grid. Why was this chosen, and are there benefits to simply using a 3D voxel grid?\n\nCan you characterize behavior of the attention modules a bit more?\n\nWhat are the failure modes of the grasping? What leads to poor predictions in some cases?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j7BGh47Ga2", "reviewer": "Reviewer_bjQA", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a new method for grasp generation using Implicit Neural Representations, a two-stage grasp evaluator and a diffusion process for calculating the orientation of the grasp. The proposed approach uses features from the Implicit Neural Representation together with the proposed Deformable Attention Module to estimate the affordance of sampled points and the grasp score when rotation is computed using a diffusion process. In such a way, the method combines the advantages of using dense prediction and sampling-based methods. In the experiments, the proposed approach is compared against several baselines showing promising performance.", "summary_of_recommendation": "I would recommend accepting this paper for publication given that the authors add a few additional technical details and improvements for clarity.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "* Why Deformable Attention Module is not used in the diffusion process to generate orientation samples?\n* How the learned offset \\delta p_k is generated?  How do you ensure that generated offsets are different from each other? Why not use the same control points as for the grasp-conditioned Deformable Attention Module with a fixed orientation?\n* What makes the Grasp Classifier SE(3)-invariant?\n* What rotation representation are you using and how do you normalise it during the diffusion process?\n* What is actually shown in Figure 3?\n\nIf my concerns expressed in the limitations are addressed and the questions above answered, I’m willing to improve my score.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0yqxsfKBXf", "reviewer": "Reviewer_TcPf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents an extension of the GIGA algorithm, incorporating deformable feature aggregation, a two-stage probabilistic grasp evaluator, and diffusion-based orientation proposals. The enhanced method generates a tri-plane feature grid by processing TSDF with ConvONet, applies deformable heads for predicting grasp affordance and width, and employs a diffusion model for orientation generation. The authors demonstrate improvements over GIGA in both simulated and real-world grasping experiments.", "summary_of_recommendation": "I recommand weak accept.", "technical_quality": 2, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "**Additional Concerns**\n\n1. **Technical and Expository Clarifications Needed:**\n    - Lack of thorough investigation of related works, particularly missing citations for ConvONet in 3.1.\n    - Several terms and symbols (e.g., \\phi_p) are undefined.\n    - Is your f actually mapping? isn’t this a probabilistic model?\n    - In Affordance Evaluator box of Figure 1, is usage of local feature correct? Isn’t it aggregated local feature?\n    - Term Grasp diffuser is confusing. It only generates orientations.\n    - Methods for generating negative grasps and determining ground truth for affordance are not adequately explained.\n    - The voxel resolution for position sampling needs specification.\n2. **Efficiency Concerns:**\n    - Increased latency in grasp generation should be analyzed, identifying the main bottlenecks, possibly related to the grasp diffuser.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers generally agree that this is a valuable contribution with some novel ideas and well-demonstrated performance improvements. On the other hand, all three point out weaknesses in terminology and formal description (regarding Eqn. 1, SE(3) invariance, generation/sampling/discretization). There are also questions around the attention module and about limitations/failures.

The authors carefully addressed most of the reviewer's questions to their satisfaction.
Decision: Accept