A Planar-Symmetric SO(3) Representation for Learning Grasp Detection
Authors: Tianyi Ko
Takuya Ikeda
Hiroya Sato
Koichi Nishiwaki
Venue: CoRL 2024 (2024)
Keywords: Grasp Detection
Rotation Representation
Parallel Gripper
TL;DR: We propose a novel SO(3) representation that can parametrize a pair of planar-symmetric poses with a single parameter set by leveraging the 2D Bingham distribution.

Abstract: Planar-symmetric hands, such as parallel grippers, are widely adopted in both research and industrial fields.
Their symmetry, however, introduces ambiguity and discontinuity in the SO(3) representation, which hinders both the training and inference of neural network-based grasp detectors.
We propose a novel SO(3) representation that can parametrize a pair of planar-symmetric poses with a single parameter set by leveraging the 2D Bingham distribution.
We also detail a grasp detector based on our representation, which provides a more consistent rotation output.
An intensive evaluation with multiple grippers and objects in both the simulation and the real world quantitatively shows our approach's contribution.

A Planar-Symmetric SO(3) Representation for Learning Grasp Detection
Tianyi Ko*
Takuya Ikeda" Woven by Toyota, Inc.
Woven by Toyota, Inc. t cianyi.... kwwwoven. toyota
..kuuaa.keda@@woen.. toyota
Hiroya Sato The University of Tokyo h-satol jsk, imi u-tokyo. .c. ip
Koichi Nishiwaki Woven by Toyota, Inc.
koichi ni waki @woven toyota
(c) Small varionee
(d) Median vairance
(4) Large variance
(a) Non-symmetric representation
(b) Our proposal
Figure 1: (a) For symmetric grippers, two distinct rotations (180%-fipped around the approach di- rection) representing the same grasp cause inconsistency and ambiguity. (b) We propose novel planar-symmetric SO(3) representation that express pair of poses with single parameter set. (c-e) It also provides deviation information, which is beneficial in the inference time.
Abstract: Planar-symmetric hands, such as parallel grippers. are widely adopted in both research and industrial fields. Their symmetry, however, introduces ambi- guity and discontinuity in the SO(3) representation, which hinders both the train- ing and inference of wewuralnnetwrr-breee S3(3) representation that can parametrize pair of planar-symmetric poses with single parameter set by leveraging the 2D Bingham distribution. We also detail grasp detector based on our representation. which provides more consistent rota- tion output. An intensive evaluation with multiple grippers and objects both the simulation and the real world quantitatively shows our approach's 's contribution. supplementary video is available at https://youtu. be 2ee/243229 75cT0.
grasp detectors. We propose a novel
Keywords: Grasp Detection, Rotation Representation, Parallel Gripper
Introduction
Parallel grippers are commonly selected for grasping tasks due to their reliability, availability, and cost. For those grippers, flipping the gripper by 180 around the approaching direction results in the same grasp due to their planar symmetry, forming a bi-modal distribution in SO(3). While reported that having multiple feasible solutions introduces ambiguity [1] and representation explicitly manage such multi-modal distribution is beneficial [2], many neural-network-based grasp detectors [3, 4, 5, 6, 7] directly regress the gripper rotation without enough consideration on this bi-modal ambiguity. In this paper, propose rotation representation that can represent the two feasible gripper rotations in a single parameter set, forming the problem into uni-modal distribution.
Equally contributed.
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Bingham [8] proposed an antipodal symmetric distribution on the (hyper-)sphere. Peretroukhin et al. [9] and Sato et al. [10] discussed the application of 3D Bingham distribution for SO(3) represen- tation by leveraging the fact that a pair of antipodal quaternions (+q and -q) represent a single rotation 3D. While [9, 10] handles a single 3D rotation, in this work, we propose a representation that can handle a pair of 3D rotations by leveraging 2D Bingham distribution to represent one of the basis vectors of a rotation matrix (See Fig. 1.)
Our approach is applicable to any grasp detector that explicitly regresses the gripper rotation. With- out modifying the training data, it only requires (i) changing the network's rotation output channel to 9, (ii replacing the rotation term of the loss function with our proposed one, (iii) additional eigenvector decomposition or sampling process for the inference time.
The contributions of this paper are summarized follows:
1. We propose novel SO(3) representation where one of the basis vectors of the rotation matrix is expressed by the 2D Bingham distribution so that pair of two planar-symmetric poses are expressed by a single parameter set.
2. We propose a learned grasp detector with the planar-symmetrie SO(3) representation. We qualitatively show its continuous properties and efficacy of the distribution information. We quantitatively show its superiority over the existing representation through simulation and real-robot experiments.
2 Related Works
2.1 Rotation Representations for Grasp Detectors
Morrison et al. [11] tackled 2D planar grasp generation and incorporated the symmetry of their two- fingered gripper by limiting the yaw angle representation the range of [0, x). Qin et al. [3] detected grasp poses in SE(3) by inferring the translational displacement and the rotation matrix [12] for each point on the object surface with a PointNet [13] backbone. Sundermeyer et al. [6] treated the point cloud as candidates of the gripper's 's contact points and estimated per-point rotation matrix. Breyer al. [4] employed 3D fully convolutional network to infer per-voxel rotation in quaternio form from TSDF [14] input. Jiang et al. [5] followed [4] in the sense of rotation expression. Ko et al. [7] expressed the gripper rotation by the rotation matrix in their fully convolutional network. Our approach targets this line of work, in which the gripper rotation is explicitly learned by neural network with a rotation loss function. In [3, 4], the planar-symmetry problem was already pointed out and tackled by selecting the smaller one of the two rotation losses for the ground truth rotation and the flipped one for the backpropagation. This paper quantitatively shows that our approach can better handle such problems.
There is another line of work for which our representation is not applicable. Zhao et al. [15] ex- pressed the rotation by the direction the fingers move and the 1D angle around that direction. Wang et al. [16] first selected "view direction" that corresponded to the hand approach direction, then inferred the remaining 1D in-plane rotation against the cropped and reprojected point cloud, with both rotations discretized. Cai et al. [17] inferred five degrees of freedom (DoF) of the grasp pose in SE(3) by the location and the surface normal of the point cloud and discretized the remaining one DoF, which can handle multimodal grasp pose. Huang et al. [18] represented grasps pairs of points on the point cloud and avoided direct rotation representation by analytically deriving it from the position and normal of the points. This work does not discuss which architecture is superior since it depends on the application, sensor, and hardware. Instead, we focus on providing a handy add-on boost the performance of pnieecr-ttation-rgeesiin--gge grasp detectors.
2.2 Rotation Representations for Symmetric Objects
To our best knowledge, there are symmetry- SO(3) representations for learning grasp de- tection. On the other hand, there are several such representations in the fields of object pose esti-

---

mation. Sato et al. [10] showed that the combination of the 3D Bingham distribution and nega- tive log-likelihood (NLL)) loss can well represent the rotation of axial symmetry. However, a sin- gle 3D Bingham distribution can only handle a unimodal distribution over rotations. To handle a multimodal distribution, Riedel et al. [19] and Deng et al. [2] utilized the Bingham Mixture Model (BMM). Manhardt al. [20] used multiple quaternions to represent multiple hypotheses of rotations that come from object symmetry. Zhang et al. [1] employed a continuous 6D rotation representa- tion [12] and applied diffusion model [21] to manage multiple feasible rotations. Although these approaches have the capability to handle the multimodality of rotations that come from discrete symmetries, they increase the number of hyperparameters or the complexity of the loss function compared to approaches with unimodal representations.
Since our target is the specific multimodal distribution of planar-symmetrie grippers, which is bimodal distribution, there can be simpler and more efficient rotation representation without lack of capability. As such, our approach extends the work by Zhou et al. [12] by expressing one of its basis vectors with the 2D Bingham distribution to allow planar symmetry. Unlike the approaches based on a mixture distribution, the explicit constraint of antipodality to manage planner symmetry is unnecessary, thanks to the nature of 2D Bingham distribution.
Method
3.1 2D Bingham Distribution
The Bingham distribution [8] which has the property of antipodal symmetry is probability distribu- tion on the unit sphere gd- C IR. We set d = 3 because we only consider S throughout this paper. The 2D Bingham distribution is parametrized by a 3 x 3 symmetric matrix. Letting Sym, be the set of n-dimensional symmetric matrices, the parameter of the 2D Bingham distribution A E Symy can be written with parameters a E I using the function triu a E RR - E Sym3.
For every real symmetric matrix, we can choose orthogonal matrix D 1xx3 and real vector À E R satisfying
(1) where the operator diag: R - 3x3 puts the elements of X on the diagonal of the matrix. This de- composition can always be performed so that a S A2 I X3 is satisfied. Using this decomposition, the 2D Bingham distribution is defined as follows.
A = D diag(A) A)
1
B B(A)(u)
C(X) exp (v Au),
where v 2 and E Symg. Here C(A) is the normalizing constant of the Bingham distribution defined as below:
C(A)
exp (v diag dga (w),
Jessa
where (-) is the uniform measure on the 32, Note that C(A) depends only on X, the eigenvalues of A. Importantly, the below equation holds for any E
B (D diag(A + +)) B(D diag
(4)
where A + == (A1 G, .....,.............. ......,, As + e). Hence, the entries of X can be shifted to satisfy the below.
\ S A2 S A 0
(5)
follows directly from the Rayleigh's quotient formula [22].
arg max " (D diaag(A)) ) Ux UEB2
(6)
where UA is a column vector of D corresponding to the maximum entry of A. The mode vector U>s coincides with the right-most column vector of D when À is sorted as Eq. 5. In the following parts, it is assumed that the condition Eq. 5 is always met.
3

---

10x-0x40410
TSDF 30 30m=33cm
0220
3D Conv
gravity-reeection score 40x40x40
gucees label
appreach direction 40x4 40x40x4333
3x40
symmettic plane normal 40x 40x00x0x3
hand orientation +
symmetric plane normal with Bingham representation
rotation confidence 30x40x60
A
Architecture of our grasp detection network with the hand symmetric plane's normal vector e; expressed in the 2D Bignham representation. We can either sample e; from the Bingham distribution (A) or directly perform eigenvalue decomposition of A and take the eigenvector with the largest eigenvalue as e, while taking the difference of the three eigenvalues as the confidence.
Figure 2:
3.2 A Planar-Symmetric SO(3) Representation and Its Loss Function
We propose novel planar-symmetric SO(3) representation that can express a pair of poses with a single parameter set. Our proposal is 9-parameter representation follows.
T, x 1 E R², where a E R x E
The 6-parameter vector a is the parameter of 2D Bingham distribution, that are described in Sec- tion 3.1. We make the mode antipodal-vector of the 2D Bingham distribution orthogonal to the symmetric plane. The 3 parameters æ are a unit 3D vector that is parallel to the symmetric plane.
To train neural network with planar-symmetric SO(3) representation, from training dataset and express it rotation matrix (ea Ey e.] R3x3 where, using and defined in Eq. 7, corresponds to æ and e₂ is mode vector of B(triu(a)). The loss function is then expressed as
first sample a pose
(8) where Less (a, b) a b/ (|]b||| denotes the cosine similarity loss and LANLL denotes the 2D Bingham negative ligeliiel loss (see Appx. B for more details). Note that while Loos has the same input size for æ and Ca., A (= triu(a)) and ez have a different sizes for LANLL be- cause we implicitly represent é, with the symmetric matrix A. In addition, since LBNLL ex) (A, ez), we only need to sample one pose per each symmetric pair from the training data. In Eq. 8, we add the two losses without scaling. Even though cosine and log have different natures, we empirically observed that LELLL typically converges from range of 2-3 after the first epoch to range of +0.5, which is close to that of Lees Our quantitative evaluation in Sec. shows that this simple addition works well for the grasp detection.
Leos ex)) + (eNua(rriu(q),e)
3.3 Grasp Detector with Planar-Symmetric SO(3) Representation
In this section, introduce rotation representation to [7]. Figure 2 illustrates its network architecture. The network takes a 40 TSDF [14] volume as input and employs 3D fully- convolutional encoder-decoder feature extractor to acquire a 40x40x40x 16 channel feature vol- ume. The feature volume is further processed by three distinct fully convolutional heads. A gravity- rejection score head estimates, for each voxel, the magnitude of disturbance in the gravity direction that the grasp can resist if the TCP (tool center point) located at that voxel. The grasp-success classification head classifies whether the grasp is feasible. Lastly, the grasp-rotation head estimates the ver-voxel gripper rotation. More training details are in Appx. C.
In the original work [7], the grasp rotation head outputs 6 channels, which is a concatenation of e, and e, (red and blue vector in Fig. (a), respectively) of the rotation matrix. To introduce the -ymmeric SO(3) representation, we extend the output channels to 9 to parameterize Eq. 7. At the training time, we feed the network output and a Eq. 8. At the inference time, we perform an eigenvalue decomposition on A to acquire its eigenvalues X; and their corresponding eigenvectors A

---

The implementation of LBLLL is based on [23], with minor modification to support 2D Bingham distribution. While the original code takes a few days to train our network, re-writing the NumPy- based operations with CuPy [24] makes the training time comparable to other rotation representa- tions. With an Nvidia V100 GPU, four training runs took an average of 190 minutes, 90 minutes for the minimum case and 350 minutes for the maximum case. The rotation matrix versions took 200 minutes in average, and the quaternion version took 170 minutes. Considering the variance, the training cost overhead of our representation can be regarded as negligible. The overhead on the memory footprint is also small since our representation only changes the last layers of the network.
There are three ways for reconstructing the hand rotation during the inference time. The simplest one is directly use the primary eigenvector dY as Cz. We analyze its effect in Section 4.1. In order make use of the information the uncertainty, derived from A. A mask to reject voxels with low confidence can be added to the output of the grasp-success classification head. We quantitatively evaluate this second approach in Section 4.3. The third approach is to sample from B(AA based on the method in [25]. This solves a major limitation of the original works: each voxel can only represent one grasp pose. We evaluate this case Section 4.2. In this case, in exchange for lower sample efficiency, we can get a diverse set of grasp poses, which is beneficial for considering additional constraints such as placement and collision avoidance.
introduce confidence threshold
In any case, there is no guarantee that e, is orthogonal to the network output e..- Following [12], we perform a -chmmi orthogonization to reconstruct a valid rotation matrix. [7] reported that keeping e, and modifying e: provides a superior performance since e, typically coincides with the normal of the object's surface and easier for the network to learn. We follow this approach by first normalizing eg then modifying e as e, (ex ez)eg.
Experiment and Evaluation
This section evaluates our approach with the vanilla version [7] as our baseline, which uses the rotation matrix representation proposed by [12]. For more comprehensive comparison, also trained a quaternion version, similarly to [4]. To train the baselines, we followed [3, 4] where computed the rotation losses for both non-flipped and 180%-flipped rotation, then selected the smaller one for the backpropagation.
4.1 Analysis of Rotation Continuity
This section compares the continuity of the rotation volume output. 60 60x 200m-sized box- shaped object was placed the workspace origin, and single depth image was captured from [0.5, 0, 0.5] m. Figure shows comparison of our approach and the baseline rotation matrix version by plotting the gravity-rejection score field and gripper rotation field at 57 mm. In this case, the basis vector e, of the hand's rotation matrix (blue vector in Fig. (a)) must be aligned with the y-axis to grasp the thin and long object.
The rotation field for the baseline not consistent: some voxels have +y output, and others have -y. This is natural because the network is trained to regress either of them, and both of them are the correct answer. The problem is that some voxels (surrounded by the red square) have intermediate value of the two correct modes. If those voxels are selected for the grasp execution, the grasp will fail. In the case of our proposal, on the other hand, the vector field is consistent and smooth. Notice that for the proposed representation e, and -e is expressed by the same parameter set (see Fig. (b)); therefore, we plot both.
4.2 Analysis Rotation Distribution
In order to analyze the distribution of the grasp rotation field, we placed 60 mm tall and 70 mm diameter flat cylinder next to the thin and long box and acquired the grasp orientation field in the same process Section 4.1. This time, sampled 30 e, from B ((A) and overlaid them in Fig. 4.
5

---

-191
position [m]
position [m]
Figure 3: Cross-section of the network output grasp score field and rotation field at 57 mm
plane when a long box is aligned with the workspace origin. Left: baseline. Right: ours.
possitio
-3.35
position [m]
Figure 4: Same plot as Fig. 3 but a flat cylinder is placed next to the box. This time we sample 30 e, from (()). The uncertainty the box is small because there grasp the long box, which corresponds to the case in Fig. ((c). The distribution on the center of the cylinder is close uniform because any downward grasp on the region is affordable. This corresponds to the case of Fig. (e).
other possible choices to
For the voxels the thin and long box, the uncertainty is small and e, converges single grasp rotation. This is reasonable because the grasp will only succeed when is aligned with the y-axis, otherwise the finger will collide with the box. This matches the case illustrated in Fig. (c) well. On the other hand, the uncertainty is large for the voxels on the cylinder. The uncertainty especially large when it is aligned with the center of the cylinder, allowing arbitrary hand rotation around the s- axis. This corresponds the case illustrated in Fig. (e). For those near the edge of the cylinder, the uncertainty takes an intermediate value. While this may be reasonable, such inaccurate poses may cause grasp failures. As the uncertainty can be analytically acquired by the ratio of the eigenvalues, proper threshold to reject those poses can be effective. Such cases are illustrated in Fig. (d).
4.3 Quantitative Evaluation with a Physics Simulation
We performed experiments in simulation to compare our proposal with our baseline quantitatively. We followed the experiment protocol of existing works [4, 3, 15, 17, 18, 5, 7]: multiple objects are randomly dropped to a scene, and the robot declutters them into a bin. The performance is measured by two metrics: success ratio (SR), which represents the number of successful grasps divided by the number of total trials, and clear ratio (CR), which represents the number of successful grasps divided by the total number of objects. We used Drake [26] with hydroelastic contact model [27] as the simulator. We simulated the whole declutter process, including the pre-contact motion, the interaction between the object and the hand's passive parallel link mechanism, and the lift motion until the object is dropped into a container placed near the workspace. Before each grasp, we rendered a single depth image to mimic the partial observation in the real world.
We trained the networks with two grippers: (i) Robotiq 2F-85 gripper with its passive compliance mode activated as an example of an underactuated gripper, and (ii) Franka-Emika" gripper, as an

---

Figure 5: Capture the simulation experiment with two kinds of grippers and objects.
example of rigid parallel-jaw gripper. We created two evaluation datasets: (i) one consisting of subset of meshes provided by Breyer et al. [4] evaluate the grasp performance household objects, (ii) one with large flat boxes that require a highly accurate grasp pose due to small clearance (see Appx. for more details).
Figure shows simulation captures, and the supplementary video contains more examples. Table summarizes the results. "2D Bingham" represents the case where used the primary eigenvector VAg e, of the rotation matrix; "2D Bingham w/ Conf. Tresh." represents the case with an identical setup to "2D Bingham," except that the voxels where (As - (2)->2) + (Ax A1) < 15 were masked out as an invalid grasp. Qualitatively, we observed two major failure modes. One was an incorrect approach direction corresponding to an inaccurate e,- We could see some improvement in the rotation matrix version over the quaternion version could not see the advantage of our approach. This is natural because our approach does not contribute to this. The other failure mode was an inaccurate yaw angle corresponding to ez, frequently occurring when the object surface and required top-down grasp. We observed a clear improvement in approach for this failure mode. We also saw that despite the underactuated gripper being effective for power grasping, its compliance is fragile against collision between the fingertip and the object during the approach. The result that our method had the highest effect for the combination of Robotiq gripper and large- boxes, quantitatively suggests that it improves tried the 3D Bingham representation proposed by [10]. Surprisingly, underperformed all other representations, indicating that naively adopting the Bingham distribution is not beneficial.
laid the
hand's yaw accuracy. As reference, we also
Table 1: Success ratio (SR) and clear ratio (CR) simulation
Robotiq 2F-85
Hand
Tranka-Emike Hand
VGN Dataset
Large Box
VGN Dataset
Large Box
2D Bingham (Ours) w/ Conf. Thresh.
SR [%]
79.0
62.9
63.1
67.7
CR
76.6
70.6
64.9
80.6
SR [%]
73.7
54.3
63.6
66.2
2D) Bingham (Ours)
CR [%]
74.9
59.9
66.6
79.4
SR [%]
65.5
46.1
58.7
48.7
Rotation Matrix [12, 7]
CR [%]
70.2
52.0
63.8
57.5
[%]
58.3
22.2
47.8
38.5
Quaternion [4]
CR [%
62.4
24.6
48.2
48.4
SR [%]
45.6
16.4
39.1
20.5
3D Bingham [10]
CR [%
47.4
17.5
39.8
23.4
4.4 Real-Robot Evaluation
To validate our approach in the physical world, we performed a real-robot evaluation with a Franka- Emika Panda robot, a Robotiq gripper with custom fingertips, and wrist-mounted ZED X Mini stereo camera. We selected nine kinds of household objects from YCB [28] dataset. We first created random scene in the simulator to ensure random yet consistent arrangement of objects for all the test cases. Before each experiment, we manually aligned the objects in the real world to match

---

Bingham (Ours) w/
SR
70.0%
Rotation
Conf. Thresh.
CR
74.5%
SR
53.6%
Rotation Matrix
CR
58.8%
Bimghum (Ours)
SR
55.9%
Quaternion
CR
64.7%
Table 2: Success ratio (SR) and clear ratio (CR) against rotation expressions in real-robot evaluation
Figure 6: Examples of failure modes that happen only for the baseline but not for ours.
the scene created in the simulation. For each grasp, the robot first moved the wrist a predefined pose and captured single depth image. We then performed the inference and sorted the voxels in the gravity-rejectionsson order, searching for the first applicable robot trajectory that fulfills: (i) the gripper linearly moves down in the -z direction 30 cm to the pre-grasp pose, (ii) the gripper moves forward for 10 em to grasp the object, (iii) the gripper is linearly lifted by 30 cm. In order to avoid collision with the environment, we first used the camera to scan the table and used a planar patch detection [29] provided by Open3D [30] to acquire set of static collision shapes. We did not explicitly avoid the collision between the gripper and objects other than the object be grasped since colliding grasps expected be rejected during network inference [4, 7].
Table 2 summarizes the result, and a full video of the experiment is included in the supplementary material. These results prove that our representation also outperformed the baseline in the real world.
Qualitatively, our approach worked better in three typical top-down grasp cases: (i) a T-shaped object, such as the electric drill; (ii) large and flat object with limited position error tolerance, such the laid-down mustard bottle; (iii) a small and thin object, such as the whiteboard marker. Similarly to the case in simulation, the baselines occasionally provided clearly wrong yaw rotation, which could not be seen for our representation. Figure 6 shows some typical cases. Appendix E provides more details on target objects and their major failure modes.
Limitation and Scope of This Work
Our work only applies to planar-symmetric grippers since it relies on the nature of the Bingham distribution to represent pair of antipodal vectors. Thus, it is not applicable to the works using three-fingered grippers or anthropomorphic hands. Another limitation that our work applies only grasp detectors that explicitly regress the gripper rotation. For example, EdgeGraspNet [18] estimates points on the object and uses their location and normal information reconstruct the gripper rotation, leaving room for approach. It's also difficult apply method works such [17, 16] which discretize the grasp rotation and avoid direct regression. Since each hardware and detector architecture has pros and cons, in this work, we focus on investigating the benefits of our approach as an add-on, leaving the comparison against those approaches as a future work.
Conclusion
In this paper, we proposed a (3) representation that can parameterize two planar- symmetri poses with a single parameter set by leveraging the 2D Bingham distribution. We also detailed the imple- mentation neural-network- tation. Qualitative analysis showed the benefits of representation for consistent network output. Finally, performed an intensive quantitative evaluation with multiple grippers and objects in both the simulation and the real world to demonstrate efficacy of our approach.
grasp detector that uses our planar-symmetric SO(3) represen-

---

References
[1] J. Zhang. M. Wu, and H. Dong. Generative category-level object pose estimation via diffusion models, Advances in Neural Information Processing Systems, 36, 2024.
[2] H. Deng, M. Bui, N. Navab, L. Guibas, S. llie, and T. Birdal. Deep bingham networks: Dealing with uncertainty and ambiguity in pose estimation. Int. J. Comput. Vis., 130(7):1277-154, July 2022.
[3] Y. Qin, R. Chen, H. Zhu, M. Song, J. Xu, and H. Su. S4g: Amodal single-view single-shot (3) grasp detection in cluttered scenes. In Proc. of Conf. Robot Learning, pages 53-65. PMLR, 2020.
[4] M. Breyer, J. J. Chung. Ott, R. Siegwart, and J. Nieto. Volumetric grasping network: Real- time 6 dof grasp detection in clutter. In Proc. of Conf. on Robot Learning, pages 1602-1611. PMLR, 2021.
[5] Z. Jiang, Zhu, M. Svetlik, K. Fang, and Zhu. Synergies between affordance and geometry: 6-dof grasp detection via implicit representations. arXiv preprint arXiv:2104.01522 2021.
[6] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes. In Int'l. l. Conf. on Robotics and Automation, pages 13438- 13444. IEEE, 2021.
[7] T. Ko, T. Ikeda, T. Stewart, R. Lee, and K. Nishiwaki. Gravity- aawirty-wwareaae grasp generation with im- plicit grasp mode selection for underactuated hands. arXiv preprint arXiv:2312. 1804, 2023.
[8] C. Bingham. An Antipodally Symmetrie Distribution on the Sphere. The Annals of Statisties, 2(6): 1225, 1974. 0of:10.22144ao/111763333333 URL fittps://doi... 1176342877.
[9] V. Peretroukhin, M. Giamou, D. M. Rosen, W. N. Greene, N. Roy, and J. Kelly. A Smooth Representation of SO(3) for Deep Rotation Learning with Uncertainty. In Robotics: Science and Systems, Jul. 12-16 2020.
[10] H. Sato, T. Ikeda, and K. Nishiwaki. A Probabilistic Rotation Representation for Symmetric Shapes With Efficiently Computable Bingham Loss Function. In Int'l Conf. Robotics and Automation, pages 6929.6929. IEEE, May 2023.
[11] D. Morrison, Corke, and Leitner. Closing the Loop for Robotic Grasping: Real-time, Generative Grasp Synthesis Approach. In Robotics: Science and Systems, 2018.
[12] Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li. On the continuity of rotation representations in neural networks. In Conf. on Computer Vision and Pattern Recognition, pages 5745-5753. IEEE/CVF, 2019.
[13] C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++:: Deep hierarchical feature learning on point sets in metric space. Advances neural information processing systems, 30, 2017.
[14] B. Curless and M. Levoy. A volumetric method for building complex models from range images. In Proc. of the 23rd annual Conf. on Computer graphics and interactive techniques, pages 303-312, 1996.
[15] B. Zhao, H. Zhang, X. Lan, H. Wang, Z. Tian, and N. Zheng. Regnet: Region-based grasp network for end-to-end grasp detection in point clouds. In Proc. of Int'l, Conf. on Robotics and Automation, pages 13474-133 13480. IEEE, 2021.
[16] C. Wang, H.-S. Fang, M. Gou, H. Fang. Gao, and C. Lu. Graspness discovery in clutters for fast and accurate grasp detection. In Proc. of the IEEE/CVF International Conference Computer Vision, pages 15964-15973, October 2021.

---

[17] J. Cai, J. Cen, H. Wang, and M. Y. Wang. Real-time collision-free grasp pose detection with geometry-aware refinement using high-resolution volume. IEEE Robotics and Automation Letters, 7(2):1888-1895 2022.
[18] H. Huang. D. Wang, X. Zhu, R. Walters, and R. Platt. Edge grasp network: A graph-based se (3)-invariant approach to grasp detection. arXiv preprint arXiv:2211.00191, 2022.
[19] S. Riedel, Z.-C. Marton, and S. Kriegel. Multi-view orientation estimation using bingham mixture models. In 2016 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), pages 1-6. IEEE, May 2016.
[20] Manhardt, D. M. Arroyo, C. Rupprecht, B. Busam, T. Birdal, N. Navab, and Tombari. Explaining the ambiguity object detection and 6d pose from visual data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6841-6850, 2019.
[21] Y. Song, Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arxiv, 2020.
[22] R. J. Plemmons. Matrix analysis (roger a. horn and charles r. johnson), 1988.
[23] Binghamnll.
[24] R. Okuta, Y. Unno, D. Nishino, Hido, and C. Loomis. Cupy: numpy-compatible library for nvidia gpu calculations. In Workshop on Machine Learning Systems The Annual Conf. on Neural Information Processing Systems, 2017. URL
org/
16.
[25] J. Kent, A. Ganeiber, and K. Mardia. A new method to simulate the bingham and related distributions in directional data analysis with applications. arxiv, 10 2013.
[26] R. Tedrake and the Drake Development Team. Drake: Model-based design and verification for robotics, 2019. URL httptttps:/ddeee //ttpp::://rrkkee/ www.mmttuuu
[27] R. Elandt, Drumwright, M. Sherman, and A. Ruina. pressure field model for fast, robust approximation of net contact force and moment between nominally rigid objects. In Proc. of Int'l. Conf. on Intelligent Robots and Systems, pages 0238-8245 IEEE/RSJ, 2019.
[28] B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar. The yeb object and model set: Towards common benchmarks for manipulation research. In Int'l I. Conf. on Advanced Robotics, pages 510-517. IEEE, 2015.
[29] A. M. Araújo and M. M. Oliveira. robust statistics approach for plane detection in unorga- nized point clouds. Pattern Recognition, 100:107115, 2020.
[30] Q.-Y. Zhou, J. Park, and V. Koltun. Open3d: A modern library for 3d data processing. arXiv preprint 1rXiv:1801.09887,,2 2018.
[31] T. Kent. Asymptotic expansions for the bingham distribution. Journal the Royal Statistical Sociery. Series C (Applied Statistics), 36(2):139 144, 1987. URL http: /www jator. crg/stable/234755
[32] K. V. Mardia and E. Jupp. Directional Statistics. Wiley Series in Probability and Statisties.
John Wiley & Sons, Inc., Hoboken, NJ, USA, jan 1999. ISBN 9780470316979. doi:10.10022 9780470316979, URL http: : //oi..i../////// wi wile eey com 10. 1 002 /
10

---

(-1000, -0 1000,00 0)
(-1000, -)
(-10, 10,00
(-10, -1,0)
(-1, 11,0)
D
1
0
(100)) [iog)
(20R
(0 (oog)
68.00 60 8.61E-04)
(68.00, 43 2.206-02)
(66. C0, 1.288-02) (208.72, 188 6.41--03)
(308 360.45 9-418-08
Figure 7: Visualizations of 2D Bingham distribution which is sampled by the method in Kent al. [25]. The blue arrows represent the peak the distribution. The parameter D in Eq. 1 controls the peak direction. The parameters satisfying Eq. 5 correlate the shape of the distribution. The values of (999% are represented by tuples whose first element the 99th percentile approximated by Eq. 12 and second is calculated directly from 1M sampled points. The second element is in the form PMC + MC. where PMC is the mean of 100 calculation of percentiles and OMC is the standard deviation of PMC.
25
-3.0
-25
no
05
+ +
+ td2
2sin
-30
-55
00
05
Figure 8: The visualization of the sampled points from Bídiag( (50, - 10, 0)). Note that d; = e; whose i-th component is and the others are In the left figure, the sampled points are depicted as blue scatter points. The domain between the red planes denotes A, (0) (99%)) ). which contains 99% of the sampled points. Similarly, the domain between the green planes (0!99))) 2D figures, the upper (repr., lower) figure shows the projection of the sampled points onto the plane spanned by (dj, ds) (repr., (dz, dz)). where the red line (repr., the green line) corresponds the red plane (repr., the green plane) in the left figure.
). In the right
Appendices
A Understanding of 2D Bingham Distribution
To show correlations between shape parameters, A = (Ai, Aa), and output distributions, we visualize several results in Fig. 7.
Furthermore, conduct intuitive analysis about shape parameters, A = (A1, d2, As). Let A Sym3 be the parameter matrix of Bingham distribution and be diagonalized as Eq. 1. We sorted

---

the entries of satisfying Eq. 5. Letting (c1, C2, oz) be the coordinates of æ relative to the frame spanned by the column vectors D, we get
(c) = =Dxx.
(9)
We define a subset g as
A.(O,) = Ex E ==s00
(10)
The visual description of A,(),) is provided in Fig. 8. Letting X B(A) be random variable that follows the Bingham distribution with the parameter A, we can define 0!²) E [0, T] as 0, satisfying the following equation.
Prob (X E A.((,)) p.
(11)
In this paper, p = 99% is used for specific calculations.
Kent [31] implies that if both À Å and À A2 sufficiently large, the distribution of 20, asymptotically approaches von Mises distribution with the shape parameter (As 22
Approximating von Mises distribution as Gaussian [32], we can yield the following approximation.
2 erf (p)
(12)
A3 X
Here erf: - (-1,1) is defined by Eq. 18, and erf- (-1,1) its inverse function.
B 2D Bingham Negative Log-Likelihood Loss
The negative log-lik eigllikeihhod loss [10] for the 2D Bingham distribution is defined as:
LENLI Ug):
2g Aug: In C(A)-
(13)
The normalizing constant C(A) can be calculated as
C(A) e h//
Σ
u((nhh)FFnnh A) cnk
(14)
where
F(t,)) II
(->x
(15)
The derivative 0C/0x can be obtained by substituting 0F/0A for F in Eq. 14. Let e, h be defined as
Nuin
2nd(1 +
C
V
r r and
S wa 1,
(16)
2 + r)wa'
waN
where d is any positive number satisfying c, and N is a positive integer satisfying N > Nmin- The function w in Eq. 14 can be parametrized by P1, P2-
1
Nh
waNh
w (x) erfe
P2
P1
P2 =
(17)
P1
Wi
4
where erfe is the complementary error function which defined by erfe(x) = 1 erf(xx). Here erf IR + (-1,1) i) is the error function defined by
2 1" -t dd,,
erf(x) = VT oo
(18)
As described in Sato et al. [10], set d c/2, Nmin 15, r 2.5, wa 0.5, N = 200 here.
12

---

C
Additional Training Details
Our model architecture and training pipeline follows [7]. We first created 90 meshes daily house- hold objects. For each object, we first sampled antipodal grasps and then randomized the grasp pose. For each grasp pose, we performed grasp simulation in gravity-less simulator and measured the relative pose between the hand and the object. This allows create a grasp pose dataset with both precision grasps and power grasps. We then project the grasp poses to the scenes, which contain multiple objects in a randomized way.
We trained the network against three labels. (i) Gravity-rejection score (trained via I.2 loss) stands for how strong a grasp can resist against gravity-direction the simulator based the approximation proposed in [7]. It serves encourage power grasping, which is more robust than precision grasping. (ii) Grasp-validness score (trained via cross-entropy loss) stands for the probability that grasp is a valid one. This label is generated by filling all voxels with a grasp by one and those without a grasp by zero. It serves to reject grasps that are colliding with the objects or trying to grasp parts that are too wide to grasp. Finally, (iii) per-voxel grasp rotation trained via the loss proposed in Eq.
direection disturbance. This label generated in
D Large Flat Box
The Robobiq 2F-85 gripper has maximum grasp width of 85 mm. We, therefore, created box object with 30 mm 70 mm x 200 mm size. When the box is laid down, the clearance between the object and the finger surface is 7.5 mm. This corresponds to 22.6 yaw tolerance if the hand's position is perfectly aligned with the box, considering the finger's 22 mm width (See Fig. 9). The effective tolerance, however, is smaller because both the network input TSDF volume and the grasp position are discretized by the voxel size of 7.5 mm (30 cm workspace divided by 40 resolution). This makes the decluttering task challenging and requires high position/rotation accuracy of the grasp pose.
22. 6
85
70
Figure 9: The combination of 70mm-wide box and the gripper with 85 mm open width only allows a 7.5 mm translation error on each side, which is the same value as the voxel size of the network's input/output. Even when the gripper is at the perfect location, the yaw tolerance is limited to 22.6".
E
Additional Details of Real-Robol Evaluation
Figure 10 shows the target objects used in the real-robot evaluation described in Sec. 4.4. Among the objects, the most challenging one was 035_powee 135_power.dril1, with more than 1kg of weight unevenly distributed to the adversarial shape. Its slippery plastic surface also makes the grasping hard. In many cases, even when the grasp looked good, it was dropped when the robot tried lift it. Other challenging objects include 005 aamato. souppppoopuu.po...ooooo can and 010 potted meat can. While the official YCB objects are shipped with their insides vacant, replaced them with unopened
13

---

SPAM
f0tm Francht
Figure 10: Target objects for real-robot evaluation.
ones to avoid damaging them. However, this makes the objects heavier (roughly 300g) and more rigid (meaning smaller contact regions). Combined with their slippy metal surface, the grasp was significantly more challenging. We also observed that C06 mustard bottle was challenging when it was laid down the surface. Even though it popular object that is used in various evaluations, when laid down, it has 90 mm width in the widest part and 80 mm width in the narrowest part, leaving almost zero tolerance for our hand with an 85 mm maximum aperture. Similarly, 010-pot eddmeat.caa -can was also challenging with its 80 mm height and 95 mm width when it was laid down.
14


Reviews:
[{"review_id": "IUsdYer2lY", "reviewer": "Reviewer_Ueu5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposed to model the grasping detection with a planar-symmetric SO(3) representation, parameterized using 2D Bingham distribution. In this way, it can handle the 180-degree ambiguity of the symmetric grippers and avoid inconsistencies in the learned representations caused by this symmetry, while being able to model the uncertainty of the estimated grasping pose.", "summary_of_recommendation": "The contribution is focused and clear, though relatively simple conceptually. The proposed representation could be valuable in general grasping pose modeling.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "I did not understand the experiment setup in section 4.4. Could you explain this paragraph further? \"We then iterated over each output voxel in the gravity-rejection score order to find the first applicable robot trajectory that fulfills: (i) the gripper linearly in the z direction 30 cm to the pre-grasp pose, (ii) the gripper moves forward for 10 cm to grasp the object, (iii) the gripper is linearly lifted by 30 cm.\" What do you mean by  \"iterating over each output voxel in the gravity-rejection score order to find the first applicable robot trajectory\"?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zBagHFSpB0", "reviewer": "Reviewer_pqFb", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes an SO(3) representation using a 2D Bingham distribution for grasp detection, which captures the bilateral symmetry of a parallel jaw gripper. By comparing it against other rotation representations (i.e., rotation matrix and quaternion), the paper demonstrates that the proposed representation achieves higher grasp success rates than these traditional representations.", "summary_of_recommendation": "While the proposed method is intriguing, the empirical results currently presented are not sufficiently compelling. If the bottleneck is the backbone network, I would suggest the authors apply their method in one of the SOTA grasp detection algorithms and improve the experimental results.", "technical_quality": 2, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Questions:\n1. While the paper compares the learning of bilateral symmetry using a loss term as proposed in [4], it would be beneficial to also discuss and compare other methods that leverage bimanual symmetry.\n2. Given the focus on grasping, my biggest concern is the performance shown in Table 1, which underperforms some existing baselines, as mentioned in the weaknesses section. Can the authors comment on this? Is the experimental environment more challenging than those used in other works? Can the authors show the performance of the proposed method in one of the standard benchmarks, like the GIGA benchmark [5]?\n3. Similarly, in the real world, the proposed method only achieves a 70% grasp success rate. What are the main failure modes?\n4. It would be helpful to include a figure in the appendix to show the selected 9 objects in the real-world experiment.\n5. Can the authors comment on why the proposed method performs better than the rotation matrix in the cases shown in Figure 6?\n6. In the limitations section, the authors claim, \"Another limitation is that our work applies only to grasp detectors that explicitly regress the gripper rotation.\" I wonder if one can use a sample-based method and send the Bingham distribution as the input to the network? This might also solve the multimodality problem of the grasp rotation distribution, which is something the regression-based method will suffer from.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w22q4R1x0S", "reviewer": "Reviewer_oYED", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The work proposes to deploy a 2D Bingham distribution in order to incorporate an antipodal planar symmetry in the context of grasping.", "summary_of_recommendation": "I believe that despite the weaknesses mentioned above, the work presents an original and interesting idea. Therefore, I lean towards accepting it for publication.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "I would like the authors to comment on the weakness raised above regarding the comparison with other representations for rotations.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Here are the strengths and weaknesses of the paper as identified by the reviewer and area chair:

Strength:
- the direction of leveraging the symmetry in robotic grippers for better grasp detection is interesting and relevant 
- The approach of how to incorporate this symmetry is novel 
- the paper contains experiments in simulation and the real world 

Weaknesses:
- while there are substantial experiments reported, the choice of baselines should be improved. For example, there are other ways to represent rotations. Or there are other SOTA grasp detectors that could be evaluated on the proposed dataset to understand their performance relative to the proposed approach. 
- The dataset on which the authors compared is not one of the standard dataset and therefore prevents the reviewers to understand how much the proposed approach improves on existing grasp synthesis methods for parallel yaw grippers. It would be good if the authors could run their approach on existing datasets while at the same time also providing more details on the dataset they evaluated on. 
- the reviewers make several suggestions to improve writing clarity of the paper

For the rebuttal, the authors have clarified their contributions and baselines. Specifically, they compare against and significantly outperform [7]  (rotation matrices - IROS paper) in both sim and real. The authors also run a set of (approximate) comparisons against [17,4]. However, this analysis is not quite at the level of rigor as the comparison to [7] (in my mind especially a comparison to [17] seems important) to demonstrate that they actually have better or at least comparable grasp performance given the new representation. A watertight comparison against [17,4] would need to be done following the same methodology as the comparison against [7]. 

For the camera-ready version I recommend the authors to make it more clear that the comparison against using rotation matrices is actually from a different paper and therefore a SOTA method. The authors should only include any conclusion on the performance compared to [17,4] if they follow the exact same methodology and data as done for [7]. I don't recommend to include the arguments laid out in the rebuttal.
Decision: Accept