Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose Estimation
Authors: Huan Ren
Wenfei Yang
Xiang Liu
Shifeng Zhang
Tianzhu Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: category-level object pose estimation
spherical representations
shape-independence
correspondence prediction
Primary area: applications to robotics, autonomy, planning

Abstract: Category-level object pose estimation aims to determine the pose and size of novel objects in specific categories. Existing correspondence-based approaches typically adopt point-based representations to establish the correspondences between primitive observed points and normalized object coordinates. However, due to the inherent shape-dependence of canonical coordinates, these methods suffer from semantic incoherence across diverse object shapes. To resolve this issue, we innovatively leverage the sphere as a shared proxy shape of objects to learn shape-independent transformation via spherical representations. Based on this insight, we introduce a novel architecture called SpherePose, which yields precise correspondence prediction through three core designs. Firstly, We endow the point-wise feature extraction with SO(3)-invariance, which facilitates robust mapping between camera coordinate space and object coordinate space regardless of rotation transformation. Secondly, the spherical attention mechanism is designed to propagate and integrate features among spherical anchors from a comprehensive perspective, thus mitigating the interference of noise and incomplete point cloud. Lastly, a hyperbolic correspondence loss function is designed to distinguish subtle distinctions, which can promote the precision of correspondence prediction. Experimental results on CAMERA25, REAL275 and HouseCat6D benchmarks demonstrate the superior performance of our method, verifying the effectiveness of spherical representations and architectural innovations.

Published as a conference paper at ICLR 2025
LEARNING SHAPE-INDEPENDENT TRANSFORMATION VIA SPHERICAL REPRESENTATIONS FOR CATEGORY-LEVEL OBJECT POSE ESTIMATION
Huan Ren Wenfei V..3 Xiang Liu
Shifeng Zhang
Tianzhu Zhang1.2+
3 UUniversity of Science and Technology of China
Naational Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory
"Jianghuai Advance Technology Center rh th_hr_66 ww._r6666aaal. uutc.. wustc.edu.c. edu.cn
*Dongguan University of Technology "Sangfor Technologies
{yangwf, (zzhaan]eustc.eddu
ABSTRACT
Category-level object pose estimation aims to determine the pose and size of novel objects specific categories. Existing correspondence-base cally adopt point-based representations to establish the correspondences between primitive observed points and normalized object coordinates. However, due to the inherent shape-dependence of canonical coordinates, these methods suffer from semantic incoherence across diverse object shapes. To resolve this issue, we in- novatively leverage the sphere shared proxy shape of objects to learn shape- independent transformation via spherical representations. Based on this insight, we introduce novel architecture called SpherePose, which yields precise corre- spondence prediction through three core designs. Firstly. We endow the point- wise feature extraction with (33)aiinrae, which facilitates robust mapping between camera coordinate space and object coordinate space regardless of rota- tion transformation. Secondly, the spherical attention mechanism is designed propagate and integrate features among spherical anchors from comprehensive perspective, thus mitigating the interference of noise and incomplete point cloud. Lastly, hyperbolic correspondence loss function is designed to distinguish sub- tle distinctions, which can promote the precision of correspondence prediction. Experimental results CAMERA25, REAL275 and HouseCat6D benchmarks demonstrate the superior performance of our method, verifying the effectiveness of spherical representations and architectural innovations.
approaches typi-
INTRODUCTION
Object pose estimation, which involves predicting the 3D rotation R SO(3) and 3D translation t E I of observed objects, has received considerable attention from the research community due its crucial applications in augmented reality (Marchand al., 2015; Su et al., 2019), robotic manipulation (Liu et al., 2023a; Wen et al., 2022), and hand-object interaction (Lin et al., 2023c), etc. While many prior instance-level object pose estimation methods (Peng al., 2019; Wang al., 2019a; 2021a) have achieved promising performance, their dependence on CAD models restricts the generalization ability. To mitigate this problem, category-level object pose estimation has been introduced in Wang et al. (2019b), which aims to reason about the 6D pose {R, t E SE(3) and 3D size 8 E 3 of unseen objects in specific categories, without the need for their CAD models.
Existing category-level methods can be mainly divided into two groups, i.e., direct eggression-based al., 2022; Lin et al., 2023b) and correspondence-basee meth- ods (Wang et al., 2019b; Tian et al., 2020; Liu et al., 2023b; Lin et al., 2024). The former approaches seck to directly regress object pose in an end-to-end manner. Although conceptually simple, they struggle with the posc-sensitive feature learning due to the non-linearity of the entire pose search space SE(3) (Lin et al., 2022a; 2023b). In contrast, the latter methods focus on establishing the correspondence between camera coordinate space and the Normalized Object Coordinate Space
methods (Chen et al., 2021;
"Corresponding author. Project page: https://renhuan1999 1999 hww.aa/mmmmmo.iui.iio/o/eeeeeeoe

---

Published as a conference paper at ICLR 2025
Mean 313
Hard to Leam
Observation
Methods
Feature Extractor
32.3
Shape Dependennt
Representations
NOCS Predictor
Point- hared Repessenttiooes
Stage Review
NOCS Predictor
NOCS Angle Emor
Shape Variation
Observation
Sphanical Repessnntiionss
NOCS Coordmmttes
Shape Indepeendestt
Pose Fitting
Method
Pose
(a) Corespondence- based Pipeliie
(b) Comparison Between Our Method and Previous Methods
Figure 1: We propose novel approach for category-leve object pose estimation based on spherical representations. (a) The classical pipeline of correspondence- based methods, where representations denote the organization form of observation data, e.g., point-based representations straightforward and commonly adopted in previous methods. (b) The overview of comparison be- tween our method and previous methods. Note that each XYZ. position of representations and NOCS coordinates is visualized as RGB tuple. Our method employs spherical representations to learn shape-independent transformation, yielding smaller NOCS angle errors compared to DPDN (Lin al., 2022b), which adopts point-based representations and suffers from the shape-dependence.
R which is
(NOCS) (Wang al., 2019b), and then solve the pose through the Umeyama algorithm (Umeyama, 1991) or deep estimators (Lin et al., 2022b), as illustrated in Figure (a)). We follow this paradigm, where the correspondences in the linear coordinate search space R are easier learn.
Nevertheless, existing correspondence-bait methods typically adopt point-based representations, where discrete points back-projected from observed depth maps serve as the representations of ob- servation data. Subsequently, the NOCS coordinates of these points are derived by geometric nor- malization and alignment of object shapes, which inherently shape-dependent. As result, observed points with similar semantics across diverse object shapes are mapped to distinct NOCS coordinates, also known as semantic incoherence Wan al. (2023). For example, points on the camera lens are mapped to different coordinates in NOCS if the camera length is different. Since the correspondence prediction need to take into account both pose information and large shape vari- ation, it is hard to learn and generalize, which results in large NOCS angle errors as demonstrated resolve the above issue, we propose innovatively leverage the sphere a shared proxy shape of objects to learn shape-independent transformation via spherical rep- resentations. Specifically, we first uniformly divide the sphere with the Hicrarchical Equal Area iso-Latitud Pixelation (HEALPix) grids (Gorski et al., 2005), each of which is represented by its center anchor. Then observed points are projected onto the spherical grids, with point-wise fea- tures assigned to the corresponding spherical anchors, which serves the representations of observation. Since these spherical anchors are coherent across various object shapes, the spherical NOCS coordinates
Figure (b)(TT)))
shape-independent, and thus more precise correspondence prediction can be achieved by focusing only on the pose information, shown in Figure (Bb)to)))
Based on the sphere representations, estimation, termed SpherePose. Given a centralized and scale-normalized observation' point-wise features are first extracted and then assigned spherical anchors with HEALPix spherical projec- tion, yielding the spherical representations. The shape-independent transformation is subsequently
introduce novel architecture for category- object pose
Note that since translation and size relatively easier he estimated, we follow VI-Net (Liin et al., 2023h) leverage lightweight PointNet++ for prediction, and focus on more challenging rotation estimation.
2

---

Published as a conference paper at ICLR 2025
learned between the spherical anchors and the spherical NOCS coordinates for rotation estimation. In order to boost the precision of correspondence prediction, we further present three core designs. The first design is the (3)vainnaa point- wise feature extraction. It is intuitive that no mat- ter how an object is rotated, the feature of a specific point on that object should remain invariant as it is mapped to a static coordinate in NOCS. Driven by this insight, for deep image features, we em- ploy pretrained DINOv2 (Oquab et al., 2024) features with patch-wise consensus semantics, which are robust rotations (Chen al., 2024). As for deep point cloud features, we propose the Color- PointNet++ network by making minimal adjustments to PointNet++ (Qi et al., 2017), which derives input features from RGB values rather than absolute X coordinates, inherently ensuring (3) invariance by focusing on the color information and relative proximity of point cloud. The second design is the spherical feature interaction. Due to self-occlusion, depth cameras fail to perceive the occluded regions on the backside of objects, resulting in incomplete point cloud. Consequently, after projecting point-wise features onto the sphere, there exist numerous spherical anchors with no assigned features. To this end, we adopt the attention mechanism (Vaswani, 2017) with learnable position embedding to facilitate the interaction and propagation of features among anchors, where initially unassigned spherical features can be reasoned out. Moreover, by holistically integrating the relationship among spherical anchors, we acquire comprehensive spherical features which are ro- bust to noise. The third design is hyperbolic correspondence loss function. For the supervision of correspondences, we derive the gradients of several typical loss functions and realize that they exhibit minor gradients around zero, which is prone yield ambiguous prediction. To achieve more precise prediction, we leverage the computation of correspondence distance in hyperbolic space (Lin al., 2023a), which yields higher gradients near zero and can discern more subtle distinctions.
To sum up, the main contributions of this paper are as follows:
We present an innovative approach that utilizes the sphere as a proxy shape of objects to learn shape-independent transformation via HEALPix spherical representations, which ad- dresses the semantic inconsistency of depe-deeendent canonical coordinates that plagues existing correspondence- methods with point-based representations.
We introduce an architecture for category-level object pose estimation that achieves precise correspondence prediction through three core designs: (33)a-inariiant point-wise feature extraction, spherical feature interaction, and a hyperbolic correspondence loss function.
Extensive experiments on existing benchmarks demonstrate the superior performance of our method over state-of-the-art approaches, verifying the effectiveness of spherical repre- sentations and architectural innovations.
RELATED WORK
The task of category-level object pose estimation is first introduced in Wang et al. (2019b) to predict the 6D pose and size of unseen objects in a given category, and existing methods can be categorized into two groups, i.e., direct regression-busee methods and correspondence-base based methods.
Direct Regression- based Methods. This category of approaches intends to directly regress the ob- ject pose in end-to-end manner. FS-Net (Chen al., 2021) proposes decouple the rotation into two perpendicular vectors that simplifies the prediction, and utilizes a 3D graph convolution (3D-GC) autoencoder for feature extraction. GPV-Pose (Di al., 2022) enhances the learning of pose-sensitive features with geometric consistency, and HS-Pose (Zheng et al., 2023) further ex- tends 3D-GC to extract hybrid scope latent features that combine both global and local geometric information. VI-Net (Lin et al., 2023b) presents a novel rotation estimation network that decouples the rotation into viewpoint rotation and an in in-plane rotation, simplifying the task by leveraging spherical representations. Based on the decoupled rotation, SecondPose (Chen et al., 2024) extracts (co3s)-consitt semantic and geometric features to further enhance the pose estimation. However, these methods struggle with the pose-sensitive feature learning due to the non-linearity of SE(3).
Correspondence- Methods. This group of approaches aims to establish the correspondence between camera coordinate space and object coordinate space, and then acquire the pose through pose fitting algorithm, such as Umeyama (Umeyama, 1991). As a cornerstone, the Normalized Object Coordinate Space (NOCS) is introduced in Wang et al. (2019b), which serves as shared canonical representation to align object instances within a given category. Later, SPD (Tian et al., 2020) proposes to handle the intra-class shape variation by reconstructing the 3D object model from
3

---

Published as a conference paper at ICLR 2025
pre-learned categorical shape prior, and then match the observed point cloud to the reconstructed 3D model. Inspired by it, CR-Net (Wang et al., 2021b) and SGPA (Chen & Dou, 2021) are presented to improve the reconstruction quality. DPDN (Lin et al., 2022b) learns the shape prior deformation in the feature space, and replaces the traditional Umeyama al algorithm with a deep estimator to directly regress the pose and size. Query6DoF ( (Wang et al., 2023) and IST-Net (Liu et al., 2023b) eliminate the dependence on explicit shape priors with implicit counterparts and implicit space transforma- tion, respectively. More recently, AG-Pose (Lin et al., 2024) proposes detect a set of sparse keypoints to represent the geometric structures of objects and establish robust keypoint-level corre- spondences for pose estimation. Although these methods have demonstrated strong performance, they all adopt point-based representations, where the NOCS coordinates are derived by geometric normalization and alignment of object shapes. When handling diverse object shapes, the NOCS coordinates become semantically incoherent, which are hard to learn and generalize. To tackle this issue, Semantically-awarr Object Coordinate Space (SOCS) is presented in Wan et al. (2023), where semantically meaningful SOCS coordinates are built by keypoint-guided non-rigid object alignment to the categorical average shape. However, due to the non-rigid alignment process, there is a mis- match between the geometric structure of the observed point cloud and the SOCS coordinates. This discrepancy plagues the pose fitting, as the transformation may not be optimal for all parts of the object. In this work, we instead resort spherical representations, where the sphere is leveraged as shared proxy shape of objects to learn hlape-independent transformation between spherical anchors and spherical NOCS coordinates. Thus precise correspondence prediction can be achieved without the need to account for large shape variation, thereby enhancing subsequent pose estimation.
3 METHOD
Target. Given an RGB-D image, we first acquire the instance segmentation masks with an offline Mask R-CNN (He et al., 2017), which then yield the cropped RGB image I Xxxx3 and segmented depth image for each instance. Partially observed point cloud po E RNNx is derived by back-projecting and downsampling the segmented depth image, where N denotes the number of points. With I and po as inputs, our method predicts the rotation R E SO(3), translation t E IR,, and size s R. of the observed instance.
As illustrated in Figure 2, the proposed method mainly consists of two components. First, in the spherical representation projection phase (Section 3.1), point-wise features are extracted and then assigned to spherical anchors with HEALPix spherical projection, yielding the spherical representations. Subsequently, in the spherical rotation estimation phase (Section 3.2), spherical anchors perform feature interaction with each other through the attention mechanism, and then they are mapped to the spherical NOCS coordinates for rotation estimation. In Section 3.3, we elaborate on the training and testing workflow, including additional translation and size prediction for com- plete category-level object pose estimation. Finally, in Section 3.4, we emphasize the distinctions between our method and some relevant works on spherical representations and the proxy shape.
Overview.
3.1 SPHERICAL REPRESENTATION PROJECTION
(0n))-iinrriant Point-wise Feature Extraction. Taking image I and normalized points pa as inputs, we start with point-wise feature extraction. Note that for point-wise image features, we first employ a 2D feature extractor to I and then leverage P to perform point-wise selection on the extracted features. To facilitate the transformation between camera coordinate space and object coordinate space, we recognize that the wise features FP E TR. should be (0n))-invariant:
(F - =F" VG E SO(3),
(1)
where the indicates the transformation of features when rotating points P by rotation G. Intuitively, the feature of a particular point on an object should remain invariant regardless of the object's is ro- tation, given that the point is mapped to fixed coordinate in NOCS. To this end, we synthesize the features of observation with SO(3) -ivaarinee from a comprehensive perspective. Specifically, for image features, we employ RGB values with low-level texture information and the pretrained DINOv2 (Oquab et al., 2024) features with high-level semantic information. Owing to the pretrain- ing on large-scale datasets, DINOv2 possesses robust generalization capabilities and can provide
a E R Nx3 is centralized and noreanorma from 0d with a lightweight PointNet+ (Qi et al,, 2017).
4

---

Published as a conference paper at ICLR 2025
Spherical Representation Projection
44
Spherical Rotation Estimation
F
RGB
LEA
Sphetical
NOCS Prodictor
R
DINOV2
Projection
Rotation Fitting
ColrPrintNett
Transfomme Encoder
Need trantog
Radivas
A
Coocatmation
RGB rolor an reagred with top prine ormggnents from PCA fietum 15a00 projection
01A
RGB songpee XX postbrrs of iipertical archom or NOUS coordraates
Figure 2: Overview of the proposed SpherePose. Given the observation I and P, we first extract -invaaant point-wise features from four distinct perspectives and assign them to the spherical anchors A with HEALPix spherical projection, yielding initial spherical features FA, Then, the Transformer encoder module is employed for spherical feature interaction and integrates compre- hensive spherical features F. Finally, predict the corresponding spherical NOCS coordinates O via a NOCS predictor, which is applied to the estimation of rotation R.
semantically consistent patch-wise features that are robust to rotations (Chen et al., 2024). As for point cloud features, we adopt radius values with low-level spatial information, along with fea- tures with high-level geometric information from the proposed ColorPoin(Net++, which is a variant of PointNet++ (Qi et al., 2017). The input features of the original PointNet++ are derived from absolute XYZ coordinates, which are inherently (03))eqquivarian To validate the importance of for correspondence prediction, features, by replacing raw XYZ coordinates with RGB values. Given the -invaannnee of color attributes and kNN operation in grouping and interpolating. our ColorPointNer++ yields SO(3) iinvariant point-wise features, as detailed in Appendix A. The final point-wise features F" e IR N x are the concatenation of all four point-wise features that capture distinct information from the observation, followed by feature dimension reduction to C.
make minimal adjustments to adapt the archi-
tecture for extracting
HEALPix Spherical Projection. Once point-wise features are extracted, previous methods adopt point-based representations to directly predict the NOCS coordinates corresponding to the observed points, which causes semantic incoherence across diverse object shapes, as introduced in Section Instead, we propose to employ the sphere a shared proxy shape of objects to learn shape- independent transformation via spherical representations. Specifically, we first uniformly divide the sphere with HEALPix grids (Gorski et al., 2005) as illustrated in Figure 4(b), each of which is equal in area and represented by its center anchor Am R3mm E {I....,AI where M denotes the number of grids. We then project the normalized points P onto the spherical grids, and assign point-wise features P to the corresponding spherical anchors, which serve as the new represen- tations of observation with spherical features FA E R MxC_ In detail, consider grid indexed by m, we follow VI-Net (Lin et al., 2023b) to search within this grid region for the point with the largest radius value, denoted as Pa. and assign its feature FP the anchor A.. If there is point projected onto the grid, we set m = 0. Since these spherical anchors are coherent across diverse objects, the corresponding spherical NOCS coordinates are hhape-independent.
3.2 SPHERICAL ROTATION ESTIMATION
Spherical Feature Interaction. Due to self-occlusion, depth cameras fail to perceive the occluded regions on the backside of objects, causing incomplete point cloud. This phenomenon deteriorates when projected onto spherical grids, leaving almost half of the spherical anchors without assigned features. To this end, we employ a Transformer encoder with attention mechanism ( (Vaswani, 2017) to facilitate feature interaction and propagation among spherical anchors A e R in global per- spective, yielding comprehensive spherical features E M from the initial spherical features FA with sequence of self- layers and MLP blocks, formulated follows:
F++1 = MLP P1+1
LE (0,... [O....,--11. L- 1},
(2)
p1+1 = Attention F*) F!
Radius refers to the Fuclidean distance from the origin, i.e., the 1.? norm of 3D point's coordinates.
5

---

Published as a conference paper at ICLR 2025
where L is the number of layers and F FF, F FA. Due to the disorder of spherical anchors, the learnable position embeddings Er RMxC are assigned to corresponding anchors and contribute to the computation of attention for feature propagation. In formula, the self-attention in 1-th layer is defined as:
(Q'W9) (K'W*) (
Attention F*) = Softmax
(v'w"),
(3)
VC
Q' = E K'= E V V'=F', F²,
(4)
where WQ/KiV RCCC denotes learnable projection matrix for query, key and value, respectively, with independent parameters across layers. Through global feature interaction and propagation, those spherical features that are initially unassigned can be reasoned out. Moreover, by attending to the relationship among spherical anchors from holistic perspective, integrate pose-coherent spherical features which can alleviate the interference of noise.
Correspondence-basee Rotation Estimation. Given the spherical features F, follow previous methods (Lin et al., 2022b; 2024) to first predict the spherical NOCS coordinates O E RMx3 corresponding spherical anchors A with MLP-based NOCS predictor. Since the spherical anchors and NOCS coordinates are on the unit sphere, we L normalize the output of MLP:
MLP(F
E (I,..,,,,
(MLP(FM)ll
After acquiring the correspondences between spherical anchors and spherical NOCS coordinates, there are several approaches to acquire the rotation R E SO(3), such as the Umeyama algo- rithm (Umeyama, 1991) or deep estimators (Lin et al., 2022b). We adopt the Umeyama algorithm with RANSAC (Fischler & Bolles, 1981) for outlier removal as it does not require extra training.
3.3 CATEGORY- OBJECT POSE ESTIMATION
Training. For the estimatiin of translation & E R$ and size 8 E R3, we follow VI-Net (Lin et al., 2023b) employ a simple and lightweight PointNet++ (Qi al., 2017). takes the observed point cloud po and the point-wise selected RGB attributes from image I as input, and directly regresses t and s with L loss as follows:
Le = 1le 171 + +|s 80
(6)
As for the estimation of rotation R E SO(3), adopt the correspondence- paradigm pervise the spherical NOCS coordinates O. Specifically, we generate ground truth spherical NOCS coordinates Ogt by mapping spherical anchors A into NOCS using the ground truth rotation Ret.
Oat (? Am, m {l,..., M),
and the prediction emrors e can then measured by the distance between them, defined as:
|0m O 1, L' distance em |0 - 0 l2, as L distance
m E (1,..,,
(8)
Previous methods employ either e directly as the loss function or the smooth L loss (Wang et al., 2019b), which uses squared term e² if fall below a certain threshold and the term e otherwise. However, we examine the gradients of these loss functions in Figure 3 and discover that they ex- hibit minor gradients around zero, which prone to generate ambiguous prediction. To promote the precision of correspondence prediction, we leverage the computation of distance in hyperbolic space (Lin et al., 2023a), which yields higher gradients near zero and can discern more subtle dis- tinctions. In detail, the arcosh(1 + æ) function is employed the errors with 2 distance, and the final hyperbolic correspondence loss function for O is formulated as follows:
1 M
1
Leorr
arcosh (1 + em) =
M arcosh (1 + 1|0m -0
(9)
Inference. Given the cropped RGB image and point cloud pO, we first predict the translation t and size s via PointNet++, and then normalize* the point cloud as P = (PO t)/||s|/2. which is fed into our SpherePose for the estimation of rotation R.
'Note that the predicted t and used for centralization and saaee-oormalizatoon in the inference stage, while the ground truth r and " used instead during the training stage.

---

Published as a conference paper at ICLR 2025
Non-uniforn Area
Uniform Area
Rej== Rol-m?
[ie nrenche) +
['(e)
Rell
1.3
2.5
(a) Equirectangular Grids
(b) HEALPix Grids
Figure 3: Illustration of several correspondence loss functions and their gradients.
Figure 4: Overview of (a) equirectangular grids and (b) HEALPix grids.
3.4 DISCUSSION
Spherical Representations. Although several previous methods (Lin et al., 2021; 2023b; Chen et al., 2024) have also adopted spherical representations for category-level object pose estimation, there are two fundamental distinctions. Firstly, they employ the equirectangular grids (Driscoll & Healy, 1994), which are evenly divided along latitude and longitude, as shown in Figure 4(a). How- ever, this partition of sphere is non-uniform in area, with a higher sampling density near the poles, which is suboptimal for representing the point cloud data. This characteristic is particularly detri- mental for objects with diverse poses, as non- non-uniform sampling of distinct semantic components re- sults in a biased distribution of sampled data. Instead, we resort to the HEALPix grids (Gorski et al., 2005), which consistently yield the uniformly sampled spherical representations across various ob- ject poses, as illustrated in Figure 4(b). This uniform spherical partition is a more appropriate choice for the capture of point cloud structures, thereby enhancing the performance of pose estimation. See- ondly, they intend to directly regress the pose in an end-to-end manner, which struggle with the pose- sensitive feature learning due to the non-linearit, of SE(3)/SO(3) DualPoseNet (Lin et al., 2021) employs spherical convolution in the spectral domain learn SO(3) -qquiivrarn features. However, the Fourier transforms require tensors in the Fourier domain of SO(3) that scale with the cube the bandwidth, limiting the resolution. VI-Net (Lin et al., 2023b) utilizes spherical convolution in the spatial domain to learn viewpoint-equivariinn features. Although it simplifies the regression process by decoupling the rotation into viewpoint rotation and in-plane rotation, the spatial convolution has a limited receptive field. Different from them, we follow with the correspondence based paradigm to establish the correspondences between spherical anchors and spherical NOCS coordinates, which is easier to learn in the linear space IR3 As for the spherical feature extraction, we adopt the atten- tion mechanism to facilitate the feature interaction and propagation among spherical anchors from a holistic perspective, yielding the comprehensive spherical features.
Proxy Shape. SAR-Net (Lin et al., 2022a) aims to deform the category-level proxy template point cloud to align with the observed point cloud for implicitly representing its rotational state. The ob- ject rotation is then solved from the categorical and deformed template point clouds by Umeyama algorithm. Although it learns shape-independent transformation by leveraging the category-shares template shape, the representations of the proxy shape are not well exploited to facilitate corre- spondence prediction. Specifically, the rotation information for guidance during the deformation process is derived from a single global feature vector of the observed point cloud, which leads to difficulties with rotation-sensstiive feature learning. In contrast, we resort the sphere as shared proxy shape to learn shape-independent transformation via spherical representations. By projecting the observed point cloud onto the spherical grids, we preserve the intact information from the ob- servation. The comprehensive spherical features are further extracted through the interaction and integration of features among spherical anchors from a holistic perspective, which can mitigate the interference of noise and promote the correspondence prediction. The proxy shape is also incorpo- rated in MFOS (Lee et al., 2024) for image-aligned pose representation, which transforms the pose into a 2D image by rendering the 3D coordinates of proxy shape (e.g., a cuboid or ellipsoid) positioned according to the pose. Since MFOS operates with only 2D images as input, the proxy shape provides mechanism for achieving 3D geometric perception, and the object pose is recov- ered by solving a PnP problem from the dense 2D-3D mapping. We differ by directly learning the 3D-3D transformation of the proxy sphere between the camera and object coordinate spaces.
7

---

Published as a conference paper at ICLR 2025
Table 1 Performance comparison with state-of-the-art methods on REAL2275 dataset.
Method
Representation
IaUso
1oUgs
5"2cm
3"5em
10"2cm
10"5cm
GPV-Pose (Di et 2022)
Point-based
64.4
32.0
42.9
73,3
HS-Prse (Zheng et al., 2023)
Point-hased
82.1
74.7
46.5
55.2
68.6
82.7
GenPose (Zhang
2023)
Point-based
52.1
60.9
72.4
84.0
Direct Regression
DualPoseNet (Lin al., 2021)
Spherical
79.8
62.2
29.3
35.9
50,0
66.8
VI-Net (Lin et ol., 2023b)
Sphericul
50.0
57.6
70.8
82,
SecondPose (Chen al,, 2024)
Spherical
56.2
63.6
74.7
NOCS (Wang
2019b)
Point-based
78.0
30.1
7.2
10.0
13.8
25.2
SPD (Tian et al., 2020)
Point-based
77.3
53.2
19.3
21.4
43.2
54.1
SGPA (Chen & Dou, 2021)
Point-based
80.1
61.9
35.9
39.6
61.3
70.7
DPDN (Lin ct al., 2022b)
Point-based
83.4
76.0
46.0
50.7
70.4
78.4
Correspondence
IST-Net et al., 2023b)
Point-based
82.5
76.6
47.5
53.4
72.1
80.5
Query6DoF (Wang et 2023)
based
82.5
76.1
49.0
58.9
68.7
AG-Pose (Lin et al., 2024)
Point-based
83.7
79,5
54.7
61.7
74.7
83.1
SpherePose
Spherical
84.0
79.0
58.2
67.4
76.2
88.2
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We evaluate methods three benchmarks including CAMERA25, REAL275 (Wang et al., 2019b) and HouseCat6D (Jung et al., 2024). CAMERA25 is a synthetic dataset that con- tains 275K training images and 25K testing images from 6 object categories, which are generated by rendering foreground objects with real-world backgrounds using the mixed-reality technique. REAL275 real-world dataset that consists of 4.3K training images of 7 scenes and 2.75K test- ing images of 6 scenes, which shares the same categories with CAMERA25. HouseCat6D is an emerging real-world dataset that comprises 20K training frames of 34 scenes, 3K testing frames of 5 scenes and 1,4K validation frames of 2 scenes from 10 household categories. This collection encompasses a diverse range of photometrically challenging objects, including glass and cutlery, which are captured in comprehensive manner across various viewpoints and occlusions.
Evaluation Metries. Following previous works (Wang et al., 2019b; Chen et al., 2024), we report the mean Average Precision (mAP) of n m em for 6D pose estimation, which denotes the percentage of prediction with rotation error less than n² and translation error less than m em. We also report the mAP of Intersection over Union (IoU,) for 3D bounding boxes with thresholds of x%.
Implementation Details. For a fair comparison, we employ the same instance segmentation masks as DPDN (Lin et al., 2022b) from Mask R-CNN (He et al., 2017). The number of sampled points is N = 2, 048. For DINOv2 features, images are first cropped and resized to 224 x 224 and then fed into the frozen DINOv2, followed by bilinear interpolation upsampling the original resolution for point-wise selection. After extracting four point-wise features, we concatenate them at the feature dimension, followed by linear layer activated by GeLU (Hendrycks Gimpel, 2016) that reduces the dimension to C' = 128. The resolution of HÉALPix grids is expressed by the parameter Naide and we set it to 8, which results in M 768 grids. The number of the Transformer encoder layers is set to L 6 by default. All experiments are conducted on single RTX3090TI GPU with a batch size of 64 for 200K iterations. More details can be found in Appendix A.2.
4.2 COMPARISON WITH SOATE-OF-THE METHODS
Results on REAL275 and CAMERA25 Datasets. Table shows the comparison of our method with existing direct regression-buss methods and correspondence-buseee methods on REAL275 dataset. should be noted that our approach represents pioneering application of spherical repre- sentations within the correspondence-hasee paradigm. From the results we can see that SpherePose outperforms the prior state-of-the-art methods on the precision 6D pose estimation. Specifically, when compared with direct regression-based methods, SpherePose surpasses SecondPose (Chen et al., 2024) by 2.0% on 5 and 2.2% on 10m5cm, which also employs spherical representations and DINOv2 (Oquab et al., 2024) backbone. And when compared with correspondence- based meth- ods, SpherePose outperforms AG-Pose (Lin et al., 2024) by 3,5% on 5cm and 5.1 % on 10"5cm, which encounters the intra-class semantic incoherence arising from point-based representations. As
8

---

Published as a conference paper at ICLR 2025
Table 2: Performance comparison with state-of-the-ard methods on HouseCat6D dataset.
Method
Representation
IoU2s
IoUsa
5"2cm
5"5cm
10"2cm
10"5cm
Ne-Net (Chen et al., 2021)
Point-based
74.9
48.0
3.3
4.2
17.1
21.6
GPV- (Di et al,, 2022)
Point-hused
74.9
50.7
3.5
4.6
17.8
22.7
Direct Regression
VI-Net (Lin et al., 20236)
Spherical
80.7
56.4
8.4
10.3
20,5
29.1
SecondPose (Chen
2024)
Spherical
83.7
66.1
11.0
13.4
25.3
35.7
AG-Pose (Lin et al., 2024)
Point-bused
81.8
62.5
11.5
12.0
32.7
35.8
Correspondence
SpherePose
Spherical
88.8
72.2
19.3
25.9
40.9
55.3
for the precision of 3D bounding boxes, achieve comparable performance. Similar results CAMERA25 dataset can be found in Table 8, please refer to Appendix A.3 for the details.
Results on HouseCat6D Dataset. Table 2 provides the quantitative results of existing methods HouseCat6D dataset. As can be seen from the table, SpherePose achieves the best performance against state-of-the-art approaches by large margin under all metries. Specifically, SpherePose surpasses SecondPose (Chen et al., 2024) by 6.1% on IoUso and 8.3% on 5e2m, and outperforms AG-Pose (Lin et al., 2024) by 10.3% on IoU30 and 7.8% on 5 When compared under the 5em metric, our method surpasses them by almost 20% (19.6% and 19.5%, respectively). It notable that this dataset includes objects with photometric complexities and extensive viewpoint distributions, which is extremely challenging. The significant improvements on this benchmark further demonstrate the effectiveness of the proposed method.
Results of Correspondence Errors. To validate the effectiveness of the proposed spherical rep- resentations compared point-based representa- tions in correspondence prediction, we calculate the mean NOCS errors the REAL.275 dataset. As shown in Table 3, the NOCS errors of Sphere- Pose are lower than DPDN (Lin et al., 2022b) and AG-Pose (Lin et al., 2024) on both the angle and Euclidean distance, which indicates that the proposed method achieves more accurate correspon- dence prediction by learning chape-independent transformation on the proxy sphere.
Table 3: Comparison of correspondence errors.
Mean NOCS Error
Method
Angle (*)
Distance (cm)
DPDN (Lin et al., 2022b)
16.62
10.50
AG-Pose (Lin al., 2024)
10.36
5.88
SpherePose
4.52
3.89
4.3 ABLATION STUDIES
To shed more light on the superiority of our method, we conduct comprehensive ablation studies on REAL275 dataset, as detailed below.
Efficacy of 80(3))invarian) Point-wise Feature Extraction. Table 4 illustrates the impact of
Table 4: Ablation studies on the deep point cloud backbone.
Point Cloud Backbone
5"2cm
5"5cm
10"2cm
10"5cm
Parameters
different deep point cloud back- bones the performance of ob- ject pose estimation. The origi- nal PointNet++ (Qi al., 2017)
None
55.9
64.6
74.3
86.7
PointNet++
56.3
64.7
73.2
84.4
1,391,104
ColorPointNete+
58.2
67.4
76.2
88.2
1,389,664
inherently lacks SO(3)-invariance due to the injection of (033)-equivarian absolute XYZ coordi- nates. To validate the importance of (O(3)-invariant features for correspondence-based methods, we introduce minimal adaptations to PointNet++ to derive our ColorPointNett+, which enables the extraction of inherently (3(3)-invarian) point cloud features while maintaining the architecture and the number of parameters almost unchanged. As shown in the table, with comparable number of parameters, our ColorPointNet** outperforms the original PointNet++ by 1.9% on 5 and 3.8% on 10 5cm. Notably, PointNet++ performs worse than point cloud backbone at all, showing a 2.3% drop on 10 5cm. These results demonstrate the effectiveness of ( invvariant point-wise features for correspondence-bases object pose estimation methods.
Efficacy of Spherical Feature Interaction. Table 5 illustrates the impact of varying numbers L of the Transformer encoder layers. When the Transformer encoder is not employed, there is significant drop in performance to 20.7% on 2em, while when two layers of encoder are applied, the performance 5 goes up 53.6%. This result demonstrates the effectiveness of spherical

---

Published as a conference paper at ICLR 2025
Table 5: Impact of the number of encoder layers.
Table 6: Impact of distinct loss functions.
Encoder Layers
5
5'5cm
10m
10m
Loss Function
5emm
5
10m2cm
10
20.7
23.9
56.2
65.8
L
56.0
64.6
74.4
86.1
53.6
61.3
74.0
85.8
Smooth L
54.6
63.2
72.6
83.3
55.7
86.5
Hyperbolic L
58.0
66.5
73.5
85.0
58.2
67.4
76.2
88.2
1.2
54.2
62.6
72.2
83.2
57.9
66.7
74.9
86.2
Hyperbolic L
58.2
67.4
76.2
88.2
feature interaction, which yields comprehensive spherical features from holistic perspective. As the number of encoder layers increases, the precision of pose estimation also improved, reaching saturation at six layers. Further increasing the number of encoder layers does not lead to enhanced performance, but rather increases the inference overhead. Therefore, we set = our method.
Efficacy of the Hyperbolic Correspondence Loss Function. Table 6 presents the impact of dis- tinct loss functions on correspondence. As for the error with I,- distance in Equation 8, we conduct evaluations using L loss, smooth L loss and hyperbolic L' loss, respectively. Since the smooth L loss has minor gradients near zero, it exhibits the worst performance of 54.6% on 2cm Con- versely, the hyperbolic I,- loss augments the gradients around zero distinguish subtle distinctions, which improves the 2cm metric from 56.0% of the L loss to 58.0%. With regard to the error with also derive the hyperbolic L loss, which results in improvement
I distance in Equation 5 from 54.2% of the L loss to 58.2%. We assume that the L distance in Euclidean space is better measure of the correspondence errors, we adopt the hyperbolic L loss for correspondence prediction. For detailed formulas of these loss functions, please refer to Appendix A.2.
Choice of Spherical Grids. Table 7
Table 7: Impact of the choice of spherical grids.
provides the performance comparison of
adopting different spherical grids. To en-
Spherical Grids
5-2cm
5"5em
10"2cm
10"5cm
sure fair comparison, we use M 768 grids for both HEALPix (Gorski et al., 2005) and Super-Fibonacci Spirals (Alexa, 2022) grids. For Equirectangular (Driscoll & Healy, 1994) grids, the latitude and longitude resolution is set to 28 x 28, resulting in 784 grids, which is close to 768 in number. Since the HEALPix grids are area-uniform for the partition of sphere and can better capture the structure of point cloud, it achieves 2.5% and 4.6% higher per- formance on the 502cm and 10m metrics, respectively, compared to the Equirectangular grids. In addition, we experiment with other nearly uniform spherical partitions, such as Super-Fibonacci Spirals, which also outperforms Equirectangular grids with 3.6% gain on 10%5cm though still slightly below the performance of HEALPix.
HEALPix
58.2
67.4
76.2
88.2
Fquirectangular
55.7
64.7
72.1
83.6
Super-Fibonaee Spirals
56.5
65.0
75.8
87.2
5 CONCLUSION
In this work, we investigate the prevalent issue of intra-class semantic inconsistency the canon- ical coordinates within existing correspondence--asee approaches for category-level object pose estimation. We attribute this issue to shape-dependent point-based representations and propose to leverage the sphere as a shared proxy shape of objects to learn shape-independent transformation via spherical representations. In light of this insight, we introduce SpherePose, a novel architecture that achieves precise correspondence prediction through three core designs: (33))iinariiant point-wise feature extraction for robust mapping between camera and object coordinate space, spherical fea- ture interaction for holistic relationship integration, and hyperbolic correspondence loss function for diagnosis of subtle correspondence errors. Experimental evaluations on existing benchmarks demonstrate the superiority of our method over state-of-the-art approaches.
Limitations and Future Works. Although the employment of spherical representations equipped with spherical attention mechanism yields superior performance, the computational complexity of the attention mechanism, which scales quadratically with the number of tokens, limits the resolution of spherical grids. When the sphere is evenly partitioned into M = 768 grids, each grid corresponds to area of approximately 53.71 square degrees. For future work, may explore linear attention mechanism (Wang et al., 2020) to facilitate spherical feature interaction. This could potentially enhance the resolution of spherical partitioning and improve the quality of spherical representations.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was partially supported by the National Nature Science Foundation of China (Grant 62306294, 62071 122), Basic Strengthening Program Laboratory Fund (No.NKLDSE2033000)) Dreams Foundation of Jianghuai Advance Technology Center (NO.2023M7433855, Youth Innova- tion Promotion Association of CAS.
REFERENCES
Marc Alexa. Super-fibonacci spirals: Fast, low-discrepancy sampling of so (3). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8291-8300, 2022. 10
Kai Chen and Qi Dou. Sgpa: Structure guidedd prior adaptation for category-level 6d object pose estimation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp- 2773-2782, 2021. 4, 8,
Wei Chen, Xi Jia, Hyung Jin Chang, Jinming Duan, Linlin Shen, and Ales Leonardis. Fs-net: Fast shape-basee based network for category-level 6d object pose estimation with decoupled rotation mecha- nism. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1581-1590, 2021. 1, 3, 9, 16
Yamei Chen, Yan Di, Guangyao Zhai, Fabian Manhardt, Chenyangguang Zhang, Ruida Zhang, Fed- crico Tombari, Nassir Navab, and Benjamin Busam. Secondpose: Se (3)-consistent dual-stream feature fusion for category-level pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1959-99659 2024. 3,5, 7, 8, 9, 14, 16, 18
Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, and Federico Tombari. Gpv-pose: Category-level object pose estimation via geometry-guider point-wiss vot- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6781-6791, 2022. 3, 8, 9, 16
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An im- age is worth 16x16 words: Transformers for image recognition at scale. In International Confer-
ence on Learning Representations, 2020. 15
James R Driscoll and Dennis M Healy. Computing fourier transforms and convolutions on the 2-sphere, Advances in applied mathematics, 15(2):202-250, 1994. 7,
Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24
(6):381-395, 1981.
Krzysztof M Gorski, Eric Hivon, Anthony J Banday, Benjamin D Wandelt, Frode K Hansen, Mstvos Reinecke, and Matthia Bartelmann. Healpix: framework for high-resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759, 2005. 2, 5, 7, 10, 15
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016. 16
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017. 8
Dan Hendrycks and Kevin Gimpel.
Gaussian eгтor linear units (gelus). arXiv preprint
1XXiv:606.0000 15, 2016. 8
HyunJun Jung, Shun-Cheng Wu, Patrick Ruhkamp, Guangyao Zhai, Hannah Schieber, Giulia Riz- zoli, Pengyuan Wang, Hongcheng Zhao, Lorenzo Garattoni, Sven Meier, et al. Housecat6d-a large-scale multi-modal category level 6d object perception dataset with household objects in alistic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattrnn Recognition, pp.
2024. 16

---

Published as a conference paper at ICLR 2025
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015. 15
Jongmin Lee and Minsu Cho. 3d equivariant pose regression via direct wigner-d harmonics predic- tion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 18
JongMin Lee, Yohann Cabon, Romain Brégier, Sungjoo Yoo, and Jerome Revaud. Mfos: Model- free & one-shot object pose estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 29112291111 1-2919, 2024.7
Fangzhou Lin, Yun Yue, Songlin Hou, Xuechu Yu, Yajun Xu, Kazunori D Yamada, and Ziming Zhang. Hyperbolic chamfer distance for point cloud completion. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 14595-14606, 2023a. 3,
Haitao Lin, Zichang Liu, Chilam Cheang, Yanwei Fu, Guodong Guo, and Xiangyang Xue. Sar-net: Shape alignment and recovery network for category-level 6d object pose and size estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6707- 6717, 2022a. 7,7
Jiehong Lin, Zewei Wei, Zhihao Li, Songcen Xu, Kui Jia, and Yuanqing Li. Dualposenet: Category- level 6d object pose and size estimation using dual pose network with refined learning of pose consistency. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp- 3560-3569, 2021. 7, 8, 16
Jichong Lin, Zewei Wei, Changxing Ding, and Kui Jia. Category-level 6d object pose and size estimation using self- superviuee deep prior deformation networks. European Conference on Computer Vision, pp. 19-34. Springer, 2022b. 2, 4, 6, 8, 9, 17, 18, 19
Jiehong Lin, Zewei Wei, Yabin Zhang, and Kui Jia. Vi-net: Boosting category-level 6d object pose estimation via learning decoupled rotations on the spherical representations. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 14001-14011, 2023b. 2, 5, 6, 7, 8, 9, 15, 16, 17, 18
Xiao Lin, Wenfei Yang, Yuan Gao, and Tianzhu Zhang, Instance-adaptive and geometric-aware keypoint learning for category-level 6d object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21040-21049, 2024. 1, 4, 6, 8, 15, 16, 17, 18, 19
Zhifeng Lin, Changxing Ding, Huan Yao, Zengsheng Kuang, and Shaoli Huang. Harmonious feature learning for interactive hand-object pose estimation. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12989-12998, 2023c. 1
Jian Liu, Wei Sun, Chongpei Liu, Xing Zhang, and Qiang Fu. Robotic continuous grasping system by shape transformer-guice guided multiobject category-level 6-d pose estimation. IEEE Transactions on Industrial Informatics, 19(1 19(11)11171-11181 2023a. 1
Jianhui Liu, Yukang Chen, Xiaoqing Ye, and Xiaojuan Qi. Ist-net: Prior-free category-level pose estimation with implicit space transformation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 13978-13988, 2023b. 1,4.8,
Eric Marchand, Hideaki Uchiyama, and Fabien Spindler. Pose estimation for augmented reality: A hands-on survey. IEEE Transactions on Visualization and Computer Graphics (TVCG), 22(12)) 2633-2651, 2015.
Octave Mariotti, Oisin Mac Aodha, and Hakan Bilen. Improving semantic correspondence with corpooint-guidee spherical maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19521-19530, 2024. 18
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez. Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024.
ISSN 2835-8856 4, 8, 14, 15, 16
12

---

Published as a conference paper at ICLR 2025
Sida Peng, Yuan Liu, Qixing Huang, Xiaowei Zhou, and Hujun Bao. Pvnet: network for 6dof pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4561-45777 2019.
Pixel-wise voting
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 3, 4, 6, 9, 14, 16
Yongzhi Su, Jason Rambach, Nareg Minaskan, Paul Lesur, Alain Pagani, and Didier Stricker. Deep multi-state object pose estimation for augmented reality assembly. In 2019 JEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 222-227. IEEE, 2019.
Meng Tian, Marcelo H Ang. and Gim Hee Lee. Shape prior deformation for categorical 6d object pose and size estimation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow; UK, August 23-28, 2020, Proceedings, Part XXI 16, pp. 530-546. Springer, 2020. 3, 8, 16
Shinji Umeyama. I Least-squares estimation of transforration parameters hetween two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence, 13(04):376-380, 1991. 2, 3, 6, 14
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
3,5
Boyan Wan, Yifei Shi, and Kai Xu. Socs: Semantically-aware object coordinate space for category- level 6d object pose estimation under large shape variations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14074, 2023. 2,4
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martin-Martin, Cewu Lu, Li Fei-Fei, and Silvio Savarese. Densefusion: 6d object pose estimation by iterative dense fusion. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 3343-3352, 2019a.
Gu Wang, Fabian Manhardt, Federico Tombari, and Xiangyang Ji. Gdr-net: Geometry guided direct regression network for monocular 6d object pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16611-16621, 2021a.
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. Pro- ceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp- 2642- 2651, 2019b. 1, 2, 6, 8, 16
Jiaze Wang, Kai Chen, and Qi Dou. Category-level 6d object pose estimation via cascaded relation and recurrent reconstruction networks. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4807-4814. IEEE, 2021b. 4
Ruiqi Wang, Xinggang Wang, Te Li, Rong Yang, Minhong Wan, and Wenyu Liu. Query6dof: Learn- ing sparse queries as implicit shape prior for category-level 6dof pose estimation. In Proceedings of the IEFE/CVF International Conference on Computer Vision, pp. 14055-14064, 2023. 4, 8, 16
Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint 10xx::2000604768.. 2020. 10
Bowen Wen, Wenzhao Lian, Kostas Bekris, and Stefan Schaal. You only demonstrate once: Category-Ievel manipulation from single visual demonstration. Robotics: Science and Systems (RSS), 2022. 1
Jiyao Zhang, Mingdong Wu, and Hao Dong Generative category-level object pose estimation via diffusion models. Advances in Neural Information Processing Systems, 36, 2023.
Linfang Zheng, Chen Wang, Yinghan Sun, Esha Dasgupta, Hua Chen, Aleš Leonardis, Wei Zhang, and Hyung Jin Chang. Hs-pose: Hybrid scope feature extraction for category- level object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 17163-17173, 2023. 8, 16
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 CLARIFICATIONS ON (0(3)-INVARIAN)
In this section, we first provide detailed explanation of SO(3)-invariance and equivariance. Subse- quently, we elaborate on the SO(3) (033a-invri of DINOv2 and ColorPointNet++, respectively.
SO(3 -n-varriance/Euuivrrianee (1) Rotation estimation network should be S0(3) -quiivaauva The rotation estimation task can be defined as R = Net(P) where Net denotes the rotation esti- mation network. The (e3)_vqquvariaace between the input observed point cloud P IR and the output estimated rotation R E SO(3) is then formulated as:
Net(wo((P) = (Net(P)))
(10)
(2) Correspondence predictor should ensure S0(3) (ifa)-iirriince. Correspondence-based methods can be expressed as R = Nel(P) ((P, Corr ()), where + is the rotation solver (e.g., the Umeyama (Umeyama, 1991) algorithm) and Corr the correspondence (NOCS) predictor. Then, (03 equivariance between the network input and output is formulated as:
( Corr(va(
( Corr (P)))
(1I)
Given fixed reference coordinates Corr(P) RN the rotation solver + is -quiivranu with respect to the other input P, which is written as:
i (VG(P), Corr(P)) = (eP((())))())))
(12)
Consequently, the correspondence predictor Corr needs to be designed to ensure SO invaainnnari
Corr(Wa(P)) Corr(P).
(13)
It means that matter how object is rotated, specific point on that object should be mapped to a static coordinate in NOCS.
(3) Feature extractor in correspondence-busell methods should be SO(3)-invrriam.. Since the respondence predictor is typically defined as Corr(P) = MLP((()), the point-wise feature ex- tractor f( ((P) in correspondence-basse approaches also needs to be (0n)a-innriiant, formulated as:
f(vo f(P).
(14)
which is consistent with Equation 1.
(0))))invariane of DINOv2. To illustrate the SO(3) (3)-invariaance of DINOv2 (Oquab et al., 2024), Figure 5 presents PCA visualization of DINOv2 features from the same instance under different rotations. The visualization shows that the DINOv2 features of the camera lens remain relatively consistent across various rotations, in- dicating their robustness to rotation variations.
However, since DINOv2 does not strictly guar- antee such semantic consistency, we conclude the features as approximately (03))-invriiant, in line with the terminology stated in Second- Pose (Chen et al., 2024).
Figure 5: PCA visualization of DINOv2 features.
(On of ColorPointNet++. To further illustrate the -inv--inae of the proposed ColorPointNer++ network, we show the pipeline comparison between ColorPointNett+ and Point- Net++ (Qi et al., 2017) in Figure 6. Given the input points P Wx where d denotes the coordi- nate dimension and is equal to Euclidean space, and associated point attributes Fattr NxC. (e.g., RGB colors with Co 3), PointNet++ first concatenates them to yield the input features Fin E (xd--Co) and then aggregates the point-wise features Pout NxC by hierarchical grouping and propagation strategy. It should be noted that in each set abstraction level and feature propagation level, PointNet++ concatenates the absolute XYZ coordinates onto the output features. The injection of (0(33))quivarian XYZ coordinates on features results in the output features of PointNet++ lacking 50(33-invariance In contrast, the proposed ColorPointNet++ is structured to
14

---

Published as a conference paper at ICLR 2025
BNG Are7)
INL HCCh
DM (HC)
RR RH
0L.
est.
fockures
sumplag
grouzing
sumplizg pouppe
printost
intapuiate
intepolate
poliizet
set aberrction
set abstmmstonn level
fretare propegation level
fenum propegahon levd
(a) PointNet++
stpp link ouncroaation
BM. CH
NN.CD
(N
01, CHCS
fk O
INL
BCB ingut features
samptng & grouging
poictnet
Samplizg garipteg
peintret
Interpiite
init piame
intepelato
polaizet
abemodtton level
olstrmstion level
feetare pripuyttonn level
featm propegation level
(b) ColerPointNet+
Figure 6: Overview of comparison between the PointNet++ and our ColorPointNet++ network.
achieve (3(3) -iivaannrr feature extraction by avoiding the injection of absolute XYZ coordinates on the features. Specifically. XYZ coordinates are not used for input features and are not concatenated in each level, where the input features are derived exclusively from RGB values. Given that the k Nearest Neighbor (kNN) operation employed in the grouping and interpolation processes is SO(3)- invariant, the output features of our ColorPointNet++ network therefore exhibit (3)33-invariance.
A.2 MORE IMPLEMENTATION DETAILS
Training Details. For deep image feature extraction, we the DINOv2 (Oquab al., 2024) model with a ViT-S/14 (Dosovitskiy et al., 2020) architecture and feature dimension of 384. And for deep point cloud feature extraction, we implement the ColorPointNet++ network incorporating 4 set abstraction levels with multi-scale grouping, and the output feature dimension is 256. In terms of HEALPix grids (Gorski et al., 2005), the resolution expressed by the parameter Nside and the number of grids is equal to N pin 12.2 side. where we set Naide 8 and yield M = =Niia 768. For data augmentation, we follow previous works (Lin et al., 2023b; 2024) to use perturbations with random translation At ~ 0-0.02 0.02), random scale As ~ (0.8, 1.2), and random rotational degree sampled from (0, 20) for each axis. Regarding network optimization, we train SpherePose using the Adam (Kingma & Ba, 2015) optimizer for 200K iterations, with an initial learning rate of 0.001 and a cosine annealing schedule.
Correspondence Loss Functions. The specifie formulas corresponding to the loss functions in Table 6 within the ablation study section (Section 4.3) are provided below:
M 100m
L
Ost ||,,
(15)
M
m=1
1
5 0 m 10 m
1
10m m
Ost || S 0.1
Lsmooth Lt
m
(16)
M
0g 0.05,
10m, m Ost 1 > 0.1
m=1
1 M
LHyperbolic 1,1
arcosh (1 + 0 Ogg I1i)
(17)
m=l
1 M
L1 =
100m O
(18)
M
m=1
1 M
LHypetbolic L
Σ
arcosh (1 + ||,m - -0 l22)
(19)
M
m=l
15

---

Published as a conference paper at ICLR 2025
Table 8: Performance comparison with state-of-the-art methods on CAMERA25 dataset.
Method
Representation
IaUso
1oUgs
5
5"5em
10"2cm
10"5em
GPV-Pose (Di et al., 2022)
Point-based
93.4
88.3
72.1
79.1
89.0
Pis- (Zheng et al,, 2023)
Point-based
93.3
89.4
73.3
80.5
80.4
89.4
Direct Regression
DualPoseNet (Lin et al., 2021)
Spherical Spherical
92.4
86.4
64.7
70.7
77.2
84.7
VI-Net (Lin et al., 2023b)
74.1
81.4
79.3
87.3
NOCS (Wang et al., 2019h)
Point-based
83.9
69.5
32.3
40.9
48.2
64.4
SPD (Tian al., 2020)
Point-based
93.2
83.1
54.3
59.0
73.3
81.5
SGPA (Chen & Dou, 2)21)
Point-based
93.2
88.1
70.7
74.5
82.7
88.4
Correspondence
IST-Net (Liu et al., 2023b)
Point-based
93.7
90.8
71.3
79.9
79.4
89.9
Query6DoF (Wang et al., 2023)
Point-based
91,9
88.1
78.0
83.1
83,9
90.0
AG-Pose (Lin et al., 2024)
Point-based
93.8
91.3
77.8
82.8
85.5
91.6
SpherePose
Spherical
94.8
92.4
78.3
84.3
84.8
92.3
Table 9: IoU metric comparison with state-of-the-art methods on HouseCat6D dataset.
Medied
540g rialls
Kkp
Rowae
Tepa
Glark
Tean
Nant end, 3001
653:450
183715
183/775
5551168
(69/993
99.1733.4
56.7/66
T147554
Poweth eall 2007
Prad lurd
714
668:466
31 1111
646/753
6531
75-7416
94.160.1
674/900
Dusti Rependis
ViNellia 30330)
Sdaiol
6031554
866:186
4637133
965/331
543113
536125
983176.
9517553
949/360
3533724
NocS/Wung 20190
Foashaard
5081213
419750
03165
B:5/054
088120
8137393
343751
187760
354/415
218/46
2547163
Cercrpastenc
A.3 ADDITIONAL EXPERIMENTAL RESULTS
Results on CAMERA25 Dataset. In Table 8, we compare our method with the existing ones for category-level object pose estimation on CAMERA25 (Wang et al., 2019b) dataset. From the results we can see that SpherePose achieves the best performance. state-of-the-art correspondence- based method AG-Pose (Lin et al., 2024) by 1.0% on IoUse. 1.1% on IoU75. 0.5% on 5 and 1.5% on 5 5cm, respectively.
detail, SpherePose outperforms the
Results of IoU Metric on HouseCat6D Dataset. Table 9 presents the quantitative comparison of loU metrics between our method and existing methods on HouseCat6l) (Jung et al., 2024) dataset. Once again, our SpherePose attains the state-of-the-ard performance, surpassing the second-best method SecondPose (Chen et al., 2024) by 5.1% 1% on IoU25 and 6. % on loUso-
Ablation Studies on Point-wise
Table 10: Ablation studies on wise features.
Features. Table 10 presents
the results of detailed ablation studies on point-wise features. For low-level image features, RGB values provide the low- level texture information, bring- ing a performance gain of 1.4% on 592cm. For high-level image features, the frozen pretrained DINOv2 (Oquab et al.. 2024) achieves 4.4% higher perfor- mance on 5 compared to the end-to-end trained ResNet18 (He et al., 2016). Due to the large-scale pretraining, DINOv2 produces semantically consistent patch-wise features that are robust to rotations, which facilitates correspondence prediction. For low-level point cloud features, removing the radius information results in a 5.9% drop on 592em, which highlights the importance of spatial structure awareness. Since the proposed ColorPointNet++ eliminates the injection of raw XYZ coordinates. its features lack implicit encoding of the underlying spatial structure, making explicit SO(3) radius in- formation essential. For high-level point cloud features, ColorPointNet++ outperforms the original PoimtNet+ (Qi et al., 2017)(C by 3.8% on 50m5cm, emphasizing the significance of (0(3)-invarin)) features. Additionally, We have tried HP-PPF (Chen et al., 2024), which also SO(3)-invarinn but the performance is suboptimal compared to not using it. This might stem from the fact that HP- PPF samples 300 points from 2048 observed points estimate point-wis, wise normal vectors, making it prone to noise due to the sparse sampling. Moreover, as a parameter-fres approach, HP-PPF lacks the deep feature extraction capabilities compared to the learnable architecture ColorPointNet++.
Point-wise Features
5-2cm
5
10"2cm
10"5em
RGB
58.2
67.4
76.2
88.2
Low-level Image
None
56.8
65.9
75.9
87.7
DINOv2
58.2
67.4
76.2
88.2
High-level Image
None
48.7
$7.1
69.2
80.3
ResNet18
53.8
62.5
83.8
Radius
58.2
67.4
76.2
88.2
Low level Point Cloud
None
52.3
58.9
72.3
83.6
ColorPointNet++
58.2
67.4
High-level Point Cloud
None
55.9
64.6
74.3
86.7
PointNet++
56.3
64.7
73.2
84.4
HP-PPF
55.1
64.2
74.2
85.9
16

---

Published as a conference paper at ICLR 2025
Table 11: Per-category results of our Sphere- Pose on REAL2275 dataset.
Table 12: Per-category results of AG-Pose (Lin
et al., 2024) REAL2275 dataset.
Category
lotbe
loUn;
5emmm
10"2cm
10"Gem
Category
IoUso
loUn;
5"2cm
5"5cm
10m5cm
bottle
57.6
51.3
71.7
76.8
$1.4
88.7
bottle
57.7
50.3
62.0
64.9
83.4
88.0
howl
100.0
100.0
87.7
96.9
90.8
100.0
bowd
100.0
100.0
88.7
94.3
94.1
99.7
camera
90.9
77.5
5.7
43.9
50.1
camera
90.8
82.9
1.3
24.8
27.3
can
71.4
70.6
76.5
81.6
91.9
98.4
can
71.3
71.2
83.4
85.3
96.3
98.6
laplop
84.8
75.5
59.5
92.7
60.0
97.0
laptop
83.3
74.1
91.1
61.2
95.6
mag
99.6
99.3
48.5
30.3
89.2
94.9
mag
99.4
98.5
32.9
33.4
88.3
89.3
84.0
79.0
58.2
67.4
76.2
88.2
average
83,7
79.5
54.7
61.7
74.7
83.1
Table 13: Per-category results of our Sphere- Pose on CAMERA25 dataset.
Table 14: Per-category results of AG-Pose (Lin
al., 2024) on CAMERA25 dataset.
Category
lotba
1oU1
8'2cm
5-5cm
10m
10"Gem
Category
10Ug0
1o1/s
8"2cm
0"Gem
10"22cm
10"
bottle
93.8
90.6
78.0
93.3
79.5
96.4
bottle
93.7
91.4
80.9
96.4
82.3
99.0
bowl
95.6
bowl
camera
94.5
91.6
69.2
72.8
78.9
86.9
camera
89.2
84.3
57.0
60.9
73.6
81.1
92.2
91.7
96.7
97.9
97.2
98.6
92.1
92.0
99.7
99.8
99.7
99.9
laptop
97.9
87.4
laptop
92.4
mag
93.6
92.3
57.6
58.9
78.7
80.4
mug
93.6
92.7
54.5
54.6
77.1
77.3
sverage
94.8
92.4
78.3
84.3
84.8
92.3
average
93.8
91.3
77.8
82.8
85,5
91.6
Per-category Results. The REAL275 and CAM- ERA25 datasets share the same categories, in- cluding bottle, bowl, camera, can, laptop and mug. In order to conduct in-depth analysis of the relative merits of our method, we show the per-category and average results of our Sphere- Pose and AG-Pose (Lin et al., 2024) on REAL2275 dataset in Table 11 and Table 12, and CAM- ERA dataset in Table 13 and Table 14, respec- tively. It can be observed that SpherePose outper- forms AG-Pose mainly in the camera and mug categories, which have large intra-class shape variations. This result demonstrates the effective-
Table 15: Per-category results of our Sphere-
Pose HouseCat6D dataset.
Category
loUgs
IoUsu
5"2cm
5"5cm
10m2cm
10"5cm
bottle
98.6
87.4
51.0
55.7
67.9
79.1
box
58.9
5.6
0.0
0.0
0.2
0.2
97.4
33.9
41.4
63.0
cup
100.0
97.7
2.8
3.0
60.6
68.4
79.0
63.5
15.7
16.3
46.9
50.1
scapot
94.8
87.3
25.2
44.4
53.6
86.8
cuilery
85.5
74.5
1.4
1.6
8.2
13.0
glass
99.6
98.0
54.8
66.3
78.5
96.4
tube
74.5
30.0
1.0
1.2
5.3
6.7
shoe
99.8
90.8
7.3
29.3
24.5
73.2
average
72.2
19.3
25.9
40.9
55.3
ness of our approach to learn shape-independent transformation via spherical representations, which can resolve the issue of intra-class semantic incoherence arising from point-based representations. We also provide the per-category results on HouseCat6D dataset in Table 15, which contains 10 household categories: bottle, box, can, cup, remote, teapot, cutlery, glass, tube and shoe. Note that we deal with all categories by a single model as in Lin et al. (2022b; 2023b; 2024).
Convergence Speed of the Hyperbolic Corre- spondence Loss Function. Figure visual- izes the training convergence speed and accu- racy comparison between our hyperbolic corre- spondence loss function and the traditional loss
Hyperbolic
Tmmnnin NOCS Angle Froor
function. As discussed in Section 3.3, the hy- perbolic 2 loss function is designed to improve the gradient of the traditional L loss function around zero, facilitating more fine-grained cor- respondence prediction. As shown in the figure,
the hyperbolic 7,2 loss not only accelerates con- vergence during training but also achieves higher accuracy, demonstrating its effectiveness in im- proving correspondence prediction precision.
50%
Training Iteration Step
100K
150K
2005
Figure Training comparison between the hy- perbolic L and traditional L loss functions.
Inference Speed and Overhead. Table 16 shows the comparison of the pose estimation perfor- mance, total number of parameters, and inference speed on REAL275 dataset. As demonstrated by the results, the increase the number L of encoder layers leads to enhanced performance, reaching saturation at six layers. A further increase in the number of layers does not result in further perfor-
17

---

Published as a conference paper at ICLR 2025
Table 16: Comparison different methods terms of performance on REAL275, inference speed and overhead. All the experiments for inference speed are conducted single RTX3090Ti GPU.
Method
2cm
5 5c5
2cmm
10"5cm
Parameters (M)
Speed (FPS)
VI-Net (Lin ct al., 2023b)
50.0
57.6
70.8
82.1
28.9
29.7
SecondPose (Chen et al., 2024)
56.2
63.6
74.7
86.0
60.2
14.3
DPDN (Lin et al., 2022b)
46.0
50.7
70.4
78.4
24.6
27.4
AG-Pose (Lin et al., 2024)
54.7
61.7
74.7
83.1
28.9
27.5
SpherePose with L =
53.6
61.3
74.0
85.8
25.7
26.4
SpherePose with L
55.7
64.5
75.2
86.5
26.1
25.9
SpherePose with L
58.2
67.4
76.2
88.2
26.5
25.3
SpherePose with L
66.7
74.9
86.2
26.9
24.5
mance gains, but rather increases the number of parameters and slows down the inference. There- fore, we set L, in method. Moreover, our approach demonstrates superior performance over existing methods, retaining a comparable number of parameters and inference speed. Specifically, in comparison to the leading correspondence-
method AG-Pose (Lin al., 2024), our Sphere- Pose achieves a 3.5% improvement on 5c2mm, with 2.4 million fewer parameters and only slight decrease FPS. When compared the direct regression baaed methods, SpherePose outperforms the advanced SecondPose (Chen et al., 2024) by 2% on 5cmmm with substantially fewer parameters (26.5 million and 60.2 million, respectively) and higher FPS (25.3 and 14.3, respectively).
A.4 CONCURRENT RELATED WORKS
In this section, we discuss the distinctions between approach and two concurrent related works.
Correspondence- work. Mariotti et al. (2024) propose semantic correspondence estimation approach, which deal with the challenges of object symmetries and repeated parts by incorporating a weak geometric spherical prior to supplement leading self-supervised features with 3D perception capabilities. They project image features guide correspondence prediction. While both this work and ours utilize spherical representations, the primary focus and application are different. Mariotti et al. (2024) aim improve semantic cor- respondence estimation, which involves finding local regions that correspond to the same semantic entities across images. contrast, our SpherePose focuses on correspondence-batt object pose estimation, which involves establishing correspondences between observed points and normalized object coordinates, and then fitting the object pose. Mariotti et (2024) use spher- ical representations to learn 3D perception-driven semantic features, while we leverage spherical representations to learn caaee-independent transformation, which is beneficial for handling large intra-class shape variations.
spherical map and viewpoint information to
category level
Regression- based work. Lee & Cho (2024) propose a 3D rotation regression approach that directly predicts Wigner-D coefficients in the frequency domain, aligning with the operations of spherical pose harmonics predictor to ensure consistent pose estima-
CNNs. They uses an (0(3)-equivarian tion under arbitrary rotations, overcoming limitations of spatial parameterizations such discon- tinuities and singularities. While both this work and ours aim to improve 3D rotation estimation, the operating domains and the way estimate rotation different. Lee & Cho (2024) focus on predicting Wigner-D coefficients in the frequency domain to capture 3D rotations, which is useful for ensuring consistent pose estimation under different rotations. Instead, our method operates in the spatial domain, leveraging spherical representations to learn haaee-nndependet transformation rather than directly predicting rotations through the network.
A.5 VISUALIZATION
Qualitative Comparison. In Figure 8, provide the qualitative comparison of existing correspondence- based methods, i.e., DPDN (Lin et al., 2022b), AG-Pose (Lin et al., 2024) and our SpherePose REAL275 dataset. The visualization results indicate that previous methods adopting point-based representations do not perform well when dealing with novel objects with large shape variations, such as the length variation in camera lenses and the curvature distinctions in mug han-
18

---

Published as a conference paper at ICLR 2025
DPDN
AG- Pose
PM
58
SpharePose
CLC
Figure 8: Qualitative comparison of DPDN (Lin et al., 2022b), AG-Pose (Lin al., 2024) and our SpherePose on REAL275 dataset. Red/Green indicates the predicted/GT results.
dles. By learning shape-independent transformation on the sphere, the proposed method allows for enhanced pose estimation.
More Visualization. We provide more visualization results of pose estimation from our SpherePose, including successful cases in Figure 9 and failed cases in Figure 10. We visualize all six previously unseen scenes within the REAL275 testing dataset, with three images for each scene. can be observed that failures predominantly occur due to the omission of objects in the detection process and significant viewpoint variations that result in incomplete capture of some objects.
19

---

Published as a conference paper at ICLR 2025
Scene
Soene 2
Scene 3
Soene
Scene 5
FIC
Scene 6
Figure 9: Visualization of successful cases from our SpherePose on the 6 testing scenes of REAL275 dataset. Red/Green indicates the predicted/GT results.
20

---

Published as a conference paper at ICLR 2025
Scene
Scene 2
Scene 3
Soene
Scene 5
S
Scene 6
H 5
Figure 10: Visualization of failed cases from our SpherePose on the 6 testing scenes of REAL275 dataset. Red/Green indicates the predicted/GT results.
21


Reviews:
[{"review_id": "23xjAGbHnp", "reviewer": "Reviewer_LyLo", "summary": "The paper addresses the challenge of object pose estimation in 3D space, specifically overcoming the limitations of existing methods that rely heavily on 3D model shapes. It proposes an approach called SpherePose, which utilizes spherical representations to create shape-independent transformations, thereby improving correspondence prediction accuracy. The method incorporates three core innovations: SO(3)-invariant feature extraction, spherical feature interaction using attention mechanisms, and a hyperbolic correspondence loss function for precise supervision. This paper mainly introduces a new proxy shape for objects and a robust architecture for category-level pose estimation. Empirical validation shows superior performance against state-of-the-art methods.", "questions": "- Given that the method relies on SO(3)-invariant feature extraction and RGB-based features, how robust is the approach to diverse lighting conditions and texture variations across different objects?\n- Can the spherical feature interaction using attention mechanisms generalize well to objects with high self-occlusion or cluttered environments, and how is this tested?\n- The hyperbolic correspondence loss function is designed to improve gradient behavior near zero. How does this compare quantitatively with traditional loss functions in terms of convergence speed and accuracy?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper uses a sphere as a proxy to implement category-level object pose estimation. It transforms the 3D shape to a uniform sphere via HEALPix spherical representations, which leads the network to focus on the semantic consistency between different objects instead of shape deviation.\n- The architecture for category-level object pose estimation that achieves precise different objects in one category correspondence prediction.\n- The experiments show that the proposed methods achieve the SOTA results on 6D pose estimation tasks in several datasets.", "weaknesses": "- The point cloud patches are occluded. The center of a point cloud patch may not be the corresponding object's center. The projection results changed a lot when the center moved. Besides, different object point cloud patches have various occlusions. The correspondence by the spherical proxy, whether a real semantic correspondence of objects, could have more evidence or qualitative results.\n- The positional encoding of the anchors needs more explanation. The way to describe the anchor position is vague, spherical coordinates or something else. The position of an anchor could influence the rotation invariance, making the results sensitive to rotation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JfeJYYrc2F", "reviewer": "Reviewer_rB6i", "summary": "This paper addresses the challenge of category-level object pose estimation by proposing a shape-independent representation using spherical features. Starting with an RGB-D image and instance masks, the method processes \"N\" partial point clouds to estimate object pose parameters: rotation R \\in SO(3), translation t \\in R^3, and size s \\in R^3 of the observed instance. The spherical feature construction on the SO(3) HEALPix grid, combined with a hyperbolic correspondence loss, significantly improves pose estimation performance. The paper categorizes pose estimation into correspondence-based vs. regression-based and point-based vs. spherical-based approaches. The proposed method, correspondence-based/spherical-based SpherePose, demonstrates state-of-the-art results on REAL275 and HouseCat6D benchmarks.", "questions": "1. Rotational Invariance/Equivariance in HEALPix Projection:\nHow does the HEALPix spherical projection in Section 3.1 maintain rotational invariance or equivariance between input and output?\n\n2. Related Work and Citations: There are two concurrent works leveraging spherical grids for pose estimation—correspondence-based [B] and regression-based [C]. Please cite and discuss these works to position your method within the broader context.\n\n[B] Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps (Mariotti et al., CVPR 2024 ): https://arxiv.org/abs/2312.13216\n\n[C] 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction (Lee et al., NeurIPS 2024 ): https://arxiv.org/abs/2411.00543", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Strong motivation: The proposed shape-independent proxy representation addresses a crucial need in category-level pose estimation, enhancing generalizability across different object instances.\n\n2. Good analysis of correspondence errors: The method effectively identifies correspondences, as shown in Table 3, where accurate correspondence significantly enhances pose estimation accuracy. \nHowever, consistency of correspondence errors should be validated on another dataset, such as HouseCat6D.\n\n3. Comprehensive Ablation Studies: \tThe authors provide a detailed analysis of different feature extractors (DINOv2, ColorPointNet++) and loss functions (L2 vs. hyperbolic L2). While not the primary contribution, Table 4 highlights the role of backbone networks in performance improvement. Table 6 effectively isolates the impact of the hyperbolic L2 loss, validating its importance. \nHowever, the authors should clearly differentiate the contribution of the backbone networks from the main innovations of the paper (spherical-based proxy and correspondence-based loss), by comparing with existing methods under the same configurations.", "weaknesses": "1. Justification for Spherical Proxy: \nWhy is the 2-sphere an optimal proxy shape for this task? While the paper adopts the spherical representation, a more thorough explanation of the choice of spherical shapes would be helpful.\n\n2. Size Estimation Despite Normalization: Section 3.1 mentions that the point cloud utilizes normalized coordinates, yet the method predicts the size parameter “s”.  How is accurate size estimation achieved despite normalization? Clarification on this aspect is necessary.\n\n3. Focus on SO(3) Over SE(3): The method predicts rotation “R” and translation “t”, focusing on SO(3) invariance. Why is roto-translation-invariance (SE(3)) not considered? (and scale-invariance) This oversight could limit the method’s robustness in varying spatial contexts.\n\n4. SO(3)-Invariant Features and Rotation Discrimination:\nIn Section 3.1, if point-wise features are SO(3)-invariant, how does the method differentiate between different rotations? Wouldn't rotation-“invariant” features cause the model to lose rotational information? An explanation on how SO(3)-equivariant features are maintained would be insightful.\n\n5. Shape-Independence and HEALPix Projection:\nIn section 3.1. HEALPix Spherical projection, does that anchor-based projection guarantee to preserve shape-independence? Even if the spherical projection is already proposed in VI-Net (Lin et al., 2023), ensuring its effectiveness in preserving shape-independence is critical in this framework.\n\n6. Correspondence-Based Loss in Linear Space:\nIn lines 357-358, the paper claims that learning correspondence-based loss is easier in linear space. Why is this the easier case? Would direct regression provide a better supervisory signal? Providing supporting evidence would strengthen this claim.\n\n7. HEALPix vs. Other Grids: It is convinced that the choice of the HEALPix grid over random rectangular SO(3) grids is explained by the drawbacks of equirectangular grids near the poles. \nHowever, could a random SO(3) grid serve as a viable alternative? \nAdditionally, in Table 7, evaluating other SO(3) spherical grids, such as SuperFibonacci spirals [A], would provide a comprehensive comparison, given its fast construction properties.\n\n[A] Super-Fibonacci Spirals: Fast, Low-Discrepancy Sampling of SO(3) (Marc Alexa, CVPR 2022)\n\n8. Clarification of this paper’s contribution: Please organize Sections 3 and 4 to focus on the main contribution of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nwE3FwOAwo", "reviewer": "Reviewer_v4xt", "summary": "This paper addresses the task of category-level object pose and size estimation, introducing a novel method called SpherePose. This method uses a sphere as a shared proxy shape for objects, enabling the learning of shape-independent transformations from spherical representations. To enhance the precision of correspondences on the sphere, SpherePose incorporates three core components, including  SO(3)-invariant point-wise feature extraction, spherical feature interaction, and a hyperbolic correspondence loss function. Experiments conducted on the CAMERA25, REAL275, and HouseCat6D datasets validate the effectiveness of SpherePose.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Unlike point-based representations, SpherePose uses a sphere as a shared proxy shape for objects and employs spherical representations to learn shape-independent transformations.\n\n- Three core components are introduced based on spherical representations to enhance the precision of correspondences.\n\n- SpherePose achieves state-of-the-art results on the CAMERA25, REAL275, and HouseCat6D datasets.", "weaknesses": "- Are the spherical NOCS coordinates derived by normalizing the original NOCS coordinates to unit vectors? If so, it would be beneficial to provide results for regressing the original NOCS coordinates.\n\n- Are the resulting poses obtained from the observed anchors or all sampled anchors on the sphere? If it’s the former, how can we verify that the spherical feature interaction aids in reasoning about the features of occluded anchors?\n\n- Unlike most existing methods that use the Umeyama algorithm, SpherePose incorporates RANSAC for solving rotations. For a fair comparison, results without RANSAC for SpherePose should be included in Table 1.\n\n- It is recommended to include results that do not use RGB or radius values as inputs in Table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2FvfVpjyly", "reviewer": "Reviewer_p4nq", "summary": "This paper introduces a new method for category-level object pose estimation. The authors argue that the shape-dependent representations in previous approaches lead to semantic inconsistencies across different objects within the same category. To handle this issue,  a shape-independent transformation is presented, transforming features to a sphere. The authors also introduce the importance of SO(3)-invariant features for category-level object pose estimation. Multiple SO(3)-invariant features extracted from RGB images and point clouds are combined to learn the NOCS coordinates on a sphere. Experimental results show SpherePose outperforms state-of-the-art methods on benchmarks like CAMERA25, REAL275, and HouseCat6D, showing its effectiveness in handling category-level pose estimation challenges.", "questions": "-\tI am a bit confused about the SO(3)-invariant features in the field of object rotation estimation. We could formulate the rotation estimation problem as R = f(g(x)|w). g(x) means the feature extractor. If the extracted features are rotation-invariant, it would be g(R(x))=g(x). If this is the case, it means f(.) would be unaware of the rotation information. Why is the network able to make the prediction related to rotations? To me, SO(3)-equivariant features seem more reasonable. \n-\tThe proposed spherical representation is designed to be object-agnostic across different categories. What factors limit the method’s ability to generalize to objects from novel categories?\n-\tWhy is the IoU75 worse than that of a point-based method AG-Pose on REAL275?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of leveraging an object-agnostic representation to handle the problem of semantic inconsistencies is technically sound and interesting. The authors propose to combine multiple deep feature extractors, and its effectiveness is demonstrated in the ablation studies. The paper conducts extensive experiments on multiple benchmarks, showing better performance compared with existing methods. The paper is well-written and easy to understand.", "weaknesses": "-\tThe presented ColorPointNet++ is not impressive. There are many alternatives [1,2,3] in the literature which can be used to extract SO(3)-invariant features.\n\n-\tThe effectiveness of DINOv2 towards rotation invariance is questionable. The authors mentioned that it is “approximately” SO(3)-invariant, which is quite vague. It would be more convincing if the authors could provide some experimental results about the invariance of DINOv2 towards 3D rotations.\n\n-\tThe difference compared with some previous methods that also use sphere representations, such as SpherePose, is unclear.  What is the major superiority compared with these methods?\n\n-\tSome important ablation studies are missing. For instance, the authors present a spherical feature interaction module to handle the challenges of self-occlusion. However, an ablation study on the effectiveness of this module is missing. Moreover, an experiment regarding the importance of SO(3)-invariant features for category-level object pose estimation is missing.\n\n[1] Deng, Congyue, et al. \"Vector neurons: A general framework for so (3)-equivariant networks.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n[2] Zhao, Chen, et al. \"Rotation invariant point cloud analysis: Where local geometry meets global topology.\" Pattern Recognition 127 (2022): 108626.\n\n[3] Fei, Jiajun, and Zhidong Deng. \"Rotation invariance and equivariance in 3D deep learning: a survey.\" Artificial Intelligence Review 57.7 (2024): 168.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)