EfficientCAPER: An End-to-End Framework for Fast and Robust Category-Level Articulated Object Pose Estimation
Authors: Xinyi Yu
Haonan Jiang
Li Zhang
Lin Yuanbo Wu
Linlin Ou
Liu Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Articulated Object;Pose Estimation;Joint-centric Pose Modeling
Primary area: robotics

Abstract: Human life is populated with articulated objects. Pose estimation for category-level articulated objects is a significant challenge due to their inherent complexity and diverse kinematic structures. Current methods for this task usually meet the problems of insufficient consideration of kinematic constraints, self-occlusion, and optimization requirements. In this paper, we propose EfficientCAPER, an end-to-end Category-level Articulated object Pose EstimatoR, eliminating the need for optimization functions as post-processing and utilizing the kinematic structure for joint-centric pose modeling, thus enhancing the efficiency and applicability. Given a partial point cloud as input, the EfficientCAPER firstly estimates the pose for the free part of an articulated object using decoupled rotation representation. Next, we canonicalize the input point cloud to estimate constrained parts' poses by predicting the joint parameters and states as replacements. Evaluations on three diverse datasets, ArtImage, ReArtMix, and RobotArm, show EfficientCAPER's effectiveness and generalization ability to real-world scenarios. The framework exhibits excellent static pose estimation performance for articulated objects, contributing to the advancement of category-level pose estimation. Codes will be made publicly available.

EfficienCCPER: An End-to-End Framework for Fast and Robust Category-Level Articulated Object Pose Estimation
Xinyi Yu², Haonan Jiang', Li Zhang", Lin Yuanbo Wu Linin Ou² Liu Liu Zhejiang University of Technology. Zhejiang, China
2 University of Science and Technology of China, Hefei, China
3 Swansea University. Swansea, UK 4 Hefei University of Technology, Hefei, China 211122030177zzjut. enu liuliuGhfut. edu. en
Abstract
Human life is populated with articulated objects. Pose estimation for category- level articulated objects a significant challenge due to their inherent complexity and diverse kinematic structures. Current methods for this task usually meet the problems of insufficient consideration of kinematic constraints, self-occlusion, and optimization requirements. In this paper, propose EfficientCAPER. end-to-end Category-level Articulated object Pose EstimatoR, eliminating the need for optimization functions post-processing and utilizing the kinematic structure for joint-centric pose modeling, thus enhancing the efficiency and applicability. Given partial point cloud input, the EfficienCCAPER firstly estimates the pose for the free part of an articulated object using decoupled rotation representation. Next, canonicalize the input point cloud estimate constrained parts' poses by predicting the joint parameters and states as replacements. Evaluations on three diverse datasets, ArtImage, ReArtMix, and RobotArm, show EfficientCAPER effectiveness and generalization ability to real-world scenarios. The framework exhibits excellent static pose estimation performance for articulated objects, con- tributing to the advancement of category-level pose estimation. Code is available at httpg////ttta..oooo//////////s/t
Introduction
Articulated objects everywhere in our daily lives, from small items like eyeglasses larger entities like dishwashers. Unlike rigid objects, which are treated as single entities in three-dimensional space, articulated objects are made up of several movable rigid parts connected by joints, following specific kinematic structure. Estimating the 6D pose-meaning the 3D rotation and translation for each rigid part from visual observation of an articulated object a fundamental challenge in computer vision. This task is crucial for various applications in robotics augmented reality [2] virtual reality (3] and 3D scene understanding
Generally, compared to instance-level 6D pose estimation 6 7. category-level pose estimation requires machines understand the 3D rotation, 3D translation, and 3D scale of unseen objects, using only semantic category prior.
In recent years, there has been a growing interest in the task of category-level articulated object pose estimation [8.9 [0] Representative methods such as Normalized Object Coordinate Space (NOCS) and its variants 12 have emerged as leading approaches. These methods introduce
"ooresponning author: Linlin Ou and Liu Liu.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

intra-class or intra-pat representations for each rigid part in canonical space and recover the object poses by estimating these representations. This approach has become the mainstream paradigm for investigating category-level articulated objects. However, this solution faces several challenges:
(1) Most of them adopt a part-centric approach for pose modeling, where each part's pose is estimated independently, disregarding the constraints imposed by the kinematic structure. (2) They often struggle with self-occlusion cases, especially where smaller parts are obscured by larger parts from certain camera perspectives. (3) The introduced intra-class or intra-part representations are often intermediate, requiring additional optimization to obtain the final poses, which can be time- consuming.
In fact, all the rigid parts of an articulated object can be categorized into two groups: free parts, which can move into any rotational and translational target (e.g., the main body of a cabinet) without kinematic constraints (with an articulated object containing only one free part), and constrained parts, which can rotate or translate along a pre-defined joint (e.g.. cabinet doors or drawers) following physically reasonable kinematic laws (with any number of constrained parts in an articulated ob- ject). Building on this insight, we propose joint-centric articulation pose modeling mechanism that establishes part-joint pairs. This approach allows learning each part's pose as joint state representation, which is easier for a neural network to learn compared to direct 6D pose regression or intra-class/intra- -art representation. Furthermore, this joint state representation significantly improves visually occluded part pose estimation, as even a few visible points can robustly contribute to joint state prediction.
In this paper, within the joint-centric pose modeling mechanism, we propose EfficientCAPERR a novel end-to-end Efficient Category-level Articulated object Pose EstimatoR. EfficientCAPER breaks down pose estimation into two stages: estimating the pose of the free part and the constrained parts, without the need for intermediate variable estimation or optimization procedures. This departure from traditional approaches, such as A-NCSH [8] and OMAD [1], eliminates the reliance on intermediate steps that recover pose using optimistic equations or retrieval methods.
Specifically, in the first stage, we utilize a decomposition strategy to represent the rotation matrix, as described in [15]. This strategy enables to estimate confidence-awant rotation vectors using hybrid feature extraction encoder (HS-encoder [66). which effectively extracts essential geometric features and relationships. In the second stage, we predict joint states, joint parameters, and the 3D scale of each part instead of using intermediate representations. By integrating these stages, EfficientCAPER achieves comprehensive category-level pose estimation for articulated objects, providing streamlined and efficient approach. We evaluate our EfficientCAPER on three datasets including the synthetic dataset ArtImage [14), and the semi-synthetic dataset ReArtMix [13. We also infer the model
RobotArm dataset (13) that contains much more diverse and complex scenes, which demonstrates that EfficientCAPER has the generalization ability to real-world scenarios. Through these extensive experiments, we provide evidence of the EfficientCAPER superior performance in static pose estimation of articulated objects. In summary. our main contributions are follows:
We proposed EfficienCCAPER, an end-to-end framework for estimating category- articulated object 6D pose efficiently and robustly without any optimization or other post- processing.
Our EfficientCAPER solves articulation pose estimation by two stages that firstly estimates the pose of the free part with decoupled rotation representation and secondly predicts joint states as pose replacement. We name this process as joint-centric pose modeling.
The efficiency and robustness of the EfficientCAPER are demonstrated through the eval- uation of either synthetic or real-world articulated object benchmarks. The sufficient experiments show the dramatic performance improvement and generalization capacity of our method.
2 Related Work
2.1 Rigid Objects Pose Estimation
Rigid object pose estimation has been significant research area in computer vision, with numerous approaches proposed to tackle this challenging problem. Traditional methods often rely template-

---

based methods [7 18 or feature- based techniques such as SIFT [9] and SURF [20] However, these methods are sensitive to occlusions and variations in illumination. Recent advancements in deep learning have revolutionized rigid object pose esti Techniques employing convolutional neural networks (CNNs) have shown remarkable performance improvements. For instance, [2T] proposed a method based on a CNN architecture that directly regresses 6-DoF poses from RGB images. While considerable progress has been made in rigid object pose estimation, the extension to category-level pose estimation remains relatively unexplored. Category-level pose estimation aims to estimate the pose of objects belonging to specific categories, despite the intra-category variations. NOCS [12 is the first method for category-level pose estimation by matching per-pixel correspondences between the observed point cloud and the estimated normalized canonical coordinates. [22] utilize variational auto-encoder to capture both pose- indepeedenddnt and pose-dependent features to directly predict the 6D pose. Given a single scene image, SAR-Net [23] proposed a method that relies on the shape information from the depth (D) channel, without using external real annotatdd training data. GPV-Pose (24) presents geometric-pane consistency terms and point-wise bounding box voting with backbone 3D-GC [25]_ Based on GPV-Pose, HS-Pose [16] proposed network structure named HS-layer that extends 3D-GC to extract hybrid scope latent features from point cloud data. Due to HS-layer's ability to perceive both local and global geometric information and its robustness noise,
we have chosen the HS-Encodes as our backbone.
2.2 Category-level Articulation Pose Estimation
Articulation pose estimation has been a topic of significant interest in computer vision and robotics. In addition to the definition of rigid object pose estimation, articulated objects consist of a finite number of rigid parts that are interconnected through various types of joints. For articulation pose estimation, we need per-part 6D pose as estimated results. Recent advancements in deep learning have led to significant progress in articulation pose estimation. Convolutional neural networks (CNNs) have been particularly successful in learning complex spatial relationships and patterns from raw data. For instance, (26) proposed a method based on a multi-stage CNN architecture for human pose estimation, achieving of-th-of- accuracy benchmark datasets such as MPII Human Pose and COCO. Similarly, (27) introduced pose estimation framework using graph convolutional networks (GCNs) model the spatial dependencies between body joints, demonstrating robust performance under occlusions and challenging poses. While existing research has focused on articulation pose estimation for a specific instance and lacks practicality on articulation.
Category-Ieve articulation pose estimation aims at predicting the pose of previously unseen articulated objects. Li et al. first propose A-NCSH (8) which extends the notation of normalized coordinates [12] into articulation. [13 introduce the concept of part pairs to investigate previously unseen instances by extending the setting of A-NCSH real-world scenarios while studying articulated objects. In addition, Xue et al. first [4] presents utilizing key points as a method for articulation modeling, aiming to enhance the speed of inference while maintaining accurate pose estimation. Recently, [II] proposed a novel category-level pose estimation framework based on reinforcement learning. However, these methods are limited by the accuracy of the kinematic model and may struggle with real-world variability and occlusions.
3 Problem Statement
To achieve end-to-end category-level articulated object pose estimation framework, our core idea to introduce a two-stage end-to-end pose estimator EfficientCAPER without any post-processing or optimization operation. Given 3D observed object point cloud P R with K rigid parts, our EfficientCAPER network performs predictions under unknown CAD models for (1) per-point part segmentation 8, (2) 3D rotation Rrree and 3D translation tree for free part, in where the 3D rotation is encoded as a decoupled rotation representation (3) 3D rotation R(Ah) and per-part 3D translation (k) for each constrained part, which are represented as joint state cons: (4) 3D scale for free part
Stree and constrained parts 8cons. In total, the rotation, translation and scale together constitute an articulation pose estimation result (R(A), ((k) s (
(k)
K 1== where {R(K)) lk=1 = (Rtree, (R( ) k }. (K( K = (Itree: (b K k and (*())) k= = (Atrce, (s(t), ) where we assume that there is only one free part in an articulated object and the corresponding index is 1.
3

---

Ingut
Canonical PC
HS-Encoder
[ag]
Part Sxppetatiiniotiiinnnl
HS-Enc
Canmicllizttnn
Input
1004
Rai Decodit
Just Ducoder
312
X-33
C.
Free Part Pose
[ny 30 Scale
lay Joint Parametes
(e) Jatnt
Stage1
HS-layer
Global frature
Extended Rodrigues Formula Procession
Puint-wise addition
Convolation layer
Stage2
Zlement-wuii addition
3D Graph max pooling
The EfficientCAPER architecture. Taking partial point cloud of an articulated object as input. our network estimates the pose of the free part using HS- Encoder as the backbone and decoupled rotation as representation. In the second stage, we canonicalize the input point cloud and predict per- part scale, segmentation, joint parameters, and joint states, where the latter two are used to recover the constrained parts" poses.
Figure 1:
Method
4.1
Overview
We present the overall pipeline of EfficientCAPER in Figure Given the point cloud of an articulated object, our approach estimates the pose of its parts through two stages. First, estimate the pose of free part, and then recover the poses of constrained parts based on joint state estimation and kinematic constraints.
initial stage, we extract hybrid-scope features from the input point cloud using an HS-Encoder [16]. These features, along with the point cloud, are inputted into one trans- decoder and two rot-decoders estimate the free part's translation and confidence-awar rotation vectors. Moving to the second stage, canonicalize the input point cloud using the estimated pose the free part, which improves the robustness of the estimation process. then use another HS-Encoder to extract hybrid-scope features, which are fed into four subbranches for joint stage estimation, joint parameters estimation, scale estimation, and part segmentation respectively. Finally, the poses of the constrained parts caleulated based the joint state and the free part's pose. Note that the input point cloud can be obtained from an RGB image and a depth image through hack-projection.
4.2 Joint-Centric Articulation Pose Modeling
As discussed above, most of the articulation pose estimation methods adopt part-centric pose modeling strategy that might ignore the effect of kinematic structure and also suffer from the self- occlusion problem. In this paper, we take joint-centric perspective to investigate articulated objects, which regards the pose estimation task as a joint state prediction task. this way, the motion cach constrained part corresponds the joint state changing.
We consider two groups of rigid parts in an articulated object: the free part and the constrained part, where the former can move arbitrarily in the corresponding joint constrained by the kinematic structure. The constrained part's motion is determined by the joint type. Following the articulation setting in A-NCSH [8] and OMAD [14) we consider two types of joints: (1) Revolute joint that allows the rotational motion the articulated object. The joint state 0, is denoted by the relative rotation angle compared to the rest state, and the joint parameters can be defined as &. (u, 9,.), where u, represents the direction of the joint axis and g, represents the pivot point position of joint axis in the canonical coordinate space. (2)
space and the latter can move along

---

Prismatic joint that allows the constrained part to move along the joint direction. The joint state 0g is denoted by the relative distance compared to the rest state, and the joint parameters can be defined as Φ (Up). where up represents the direction of the joint axis. By defining the joint-centric pose modeling method, we can correspond part to joint one by one, and the total articulated object poses for all the parts can be represented by a sequence of joint states O = {0 RR=2 of constrained parts and the pose Rfree: (tree) J of free part. The definitions of free part and constrained part are illustrated in Figure
Free past
Constrained part
Z
patiaz aane frnd 1ga0e
Coostatted cut sarve deoog Da comppentrng your by
Figure 2: Free part and Constrained part in uigure 3: Rotation decoupled strategies. We select ticulated objects. These two kinds of parts are the corresponding two axes according to the sym- connected by joints.
metry priors.
4.3 Free Part Pose Estimation with Decoupled Rotation
In our EfficientCAPER, the input of the network partial point cloud of an articulated object P E RNN33, In the first stage, we aim to estimate the pose (Riee, (rre) J for free part, and here we describe the rotation Riee and translation frree estimation separately.
Rotation Estimation. The GPV-Pose 24 was proposed to estimate the 6D pose with the rigid object point cloud input. When transferring into the free part pose estimation, we input the point cloud from all the rigid parts rather than only the free part's 's point cloud, since (1) the part segmentation is not required as a prerequisite task that further improves the efficiency of our method. (2) other constrained parts can provide auxiliary information for estimating the pose of the free part with strong symmetry. Specifically, given the point cloud P E R N3 we build an encoder-decoder architecture which includes an HS HS-Encoder with four Ha-layers and rot decoder with four 1-D convolution layers. We can extract the feature of each HS layer after encoding, and we performed a max pooling operation on the point cloud dimension of the last layer's output get a global feature that contains global information. We concatenate the global feature with the output of each HS layer as the ultimate output of the encoder. The Firee be decomposed into three direction vectors: x, y, and 7, select two of them for different objects to help the network learn shape prior better. As shown in Figure 3 we choose y-axis and z-axis direction vectors since the laptop is symmetric about the yOz plane. Similarly, we choose the x-axis and z-axis direction for the box. Inspired by [24]. an attempt made estimate confidence value for each vector enhance the robustness during the recovery of the final 3D rotation. For the decoding process, we decode the ultimate feature to get two vectors, then we normalize them to get predicted plane normals and implement a softmax operation to get confidence for each predicted normal.
Translation Estimation. Regarding the translation we firstly zero-center the input point cloud P by reducing the mean of point cloud P. Next, we build a similar decoder consisting of 1-D convolution layers. Inspired by [15), estimate the residual between the mean value of P and the ground truth of translation. The translation feature formed by concatenating the output of every HS-layer and zero-center point cloud, and can finally get residual translation th through the translation decoder. The final translation free part free can be obtained by frree tiree P.. Finally, the 6D pose of the free part is the constitution of rotation and translation Tiree [Btree, )
5

---

4.4 Articulation Pose Canonicalization
Before the second stage of estimating poses for constrained parts, we canonicalize the input point cloud with the predicted free part's pose tree. This operation provides the following benefits: (1) the joint state estimation task in camera space can be transformed into a "pseudo-canonical" space, which is more suitable for neural network learning. (2) the canonicalization process applied to the point cloud can effectively eliminate the effects of different joint configurations in the movable rigid parts, it allows the canonicalized point cloud to provide both shape prior and kinematic prior that make a significant contribution to regress joint states.
In articulated pose canonicalization module, the canonicalized point cloud P is computed as product of the inverse transformation Tiree and the point cloud p(k);
p(k)= (Trree) - (Rtree) (P(() treee)
(1)
By canonicalizing the imput point cloud, the regressing model in stage two will take the canonicalized point cloud as input to estimate joint states, the neural network would increase sensitivity to parameters and states of joint and could considerably enhance regression performance.
4.5 Constrained Part Pose Estimation with Joint State Replacement
At the second stage of our EfficientCAPER, we predict the pose for constrained parts { (R } k== as k=2- labels 5 and joint parameters (u(*), .q (*)) )= To this end, we also build the encoder-decoder architecture network whose parameters are partly shared with that the first stage, Specifically, we employ the HS-Encoder same as the first stage, and design four decoders. Each layer in the decoder consists of a 1-D convolution, batch normalization, and dropout operation. We use the ReLU activation function for the final output of the layer. At the end of the network, we build four parallel branches for estimating joint parameters, joint states, per-part scales, and part labels respectively.
well as scales (s )
Joint Parameter Branch. In order to recover per-part pose for the constrained part, it is required
to estimate the joint parameters (u. q). Refer to A-NCSH [8] that predicts the joint parameters in normalized coordinate space, our EfficientCAPER predicts them the articulation canonicalized space. To be specific, the joint parameter branch involves three heads: (1) joint direction head that regresses the joint direction u for both revolute and prismatic joints. (2) offset head that predicts the projected distance from each canonicalized point Pt to the joint (u, q)- (3) heatmap head that
predicts the score h; of the possibility of p; voting into the joint. this way, the joint parameters (u. q) can be obtained by voting scheme:
u
u
N
1
q
h (q: + 0,)
N
Note that when estimating the parameters for prismatic joint, we only use joint direction head since is not defined in prismatic motion.
Joint State Branch. For estimating the poses of the constrained parts, build prediction branch that predicts the joint state (*) pose replacement. The joint state branch ends with 1-D convolution layer that outputs the final result containing K - 1 channels corresponding to K - constrained parts in articulated object. During inference for revolute joint, given the predicted
joint parameters e( = ( qr (R)) and joint states (*)) we utilize the extended Rodrigues rotation
formula to convert the joint state into matrix [[A)] for k-th part:
R' cos ((+) + +(1 cos (4 Q. 2!!)) sin ((
(4)

---

where O,"² represents skew-symmetric matrix composed of pivot point position q
(k) ( repre-
sents the normalized direction of the joint axis u, (k) Here the Rimn is a matrix size 4 x and we
stack it with the row [0, 0, 0, 1] appended at the end to get relative homogeneous matrix T Teuns as k-th constrained part's pose.
(A)
In terms of prismatic joint, given joint parameters (up (*) and predicted joint state 0 we can also get relative homogeneous matrix as pose for k-th part:
I
8 (N)
(1)
P
up
(5)
comA
0
where I indicates the identity matrix. Finally, we can calculate the poses of constrained parts by T Trree cons (k)
Part Scale Branch. To predict the scale (*x) for all the parts of articulated object, employ the
regression scheme in the part scale branch that predicts the residual size s*(k) g(k) g()) where
((()) defined as the predefined mean size of k-th part. Therefore, the final scales are determined by (kk) = s + (k)
4.6 Multi-Taan Task Loss Function
We utilize multi-task loss functions to train our EfficientCAPER network. the free part pose estimation stage, we first design the L1 loss function to supervise the (Tm: Fa) vector and translation t regression:
Fro
|lrm mmll + ||r, fulli Lirans
[
f|]
where m.n represent the possible selected directions of normals. To train the confidence score that used for (rm rn) voting, the loss function Loom is defined as:
Loons
Σ
|0, - erplp (ko(-kiirr--- - +11 -
iE am).)
(E(1.))
where k1 is factor -13.7 which follows the same setting in GPV-Pose 24]. Regarding the constrained part pose estimation stage, we utilize four loss functions to supervise the joint parameter, joint state, part scale, and part segmentation branches. Firstly for joint parameter branch, the loss function Cpara aims to optimize the projection distance from predicted joint location g* to joint (t))), (1)))::
K Σ
Lioe
(roj u (K))
(8)
K
and we use the LI loss to supervise the joint direction Las-- terms of the joint state branch, since the joint states in the annotation can be positive negative, directly regressing these joint states is quite difficult. So we design the joint state loss function Lstat with positive penalty factor a = 20:
(0)))
(K)
l1.
i 1000)00 (kh) 0
K
Estat
(9)
a (*())
(
(8)
<0
In this loss function, when joint state the k-th joint is opposite to ground truth annotation, we apply penalty factor to the loss of that particular joint which encourages the neural network to learn more correctly. Next, we employ L. loss for part scale branch Lsen and Cross-Entropy loss for part segmentation branch Loeg. Finally, the total training losses Li for stage and Ce for stage
7

---

two are both the weighted sum of the losses from free part pose estimation and constrained part pose estimation modules:
L1 Arothrat + Mrranshatrans + Xeomf Lean
(10) (11)
L2 Atoc Lair + Astathstat + Asca Lsea + Aseghseg
where the weight multiplication factors Arat, Atrons, Aeom Atocs Adir, Natat, Ascay Aseg are set to be 8.0, 10.0, 1.0, 1.0, 1.0, 6.0, 1.0, 0.1. The optimal nyper-parameteen are mined by rrossvvaiidttion method.
5
Experiments
5.1 Experimental Settings
Datasets. We evaluate Efficien(CAPER on three datasets: ArtImage [1), ReArtMix [13, and RobotArm 13.
First, Artimage is a synthetic articulated object estimating dataset with the objects from PartNetMobility. This dataset contains five categories of articulated objects: Laptop. Eyeglasses, Dishwasher, Scissors, and Drawer. Next, our method has also been evaluated on semi-synthetic dataset ReArtMix for articulated object estimating tasks. To validate the generality of our method in real-world scenarios, we finally train and test on the 7-part RobotArm dataset.
Metrics and Implementation Details. To validate the performance of the EfficientCAPER, adopt degree error(") for 3D rotation, distance error(m) for 3D translation, 3D IOU(%) for scale, and per image inference time for estimating speed. During the data pre-processing, input point clouds sampled into 1,024 points as the network inputs. We have adopted the Ranger optimizer and the "flat strategy. The learning rate remains 10-4 in stage "flat". 'he total training epoch is 250 for stage one and 200 for stage two. We put two trained models in the same script for inference directly obtain per-part pose. All the experiments are implemented on NVIDIA GeForce RTX 3090 GPU with 24GB memory.
and anneal" learning
Table I: Comparison with offt-of-thc-arts on the Artimage dataset. We validate our EfficientCAPER on categories Laptop. Eyeglasses, Dishwasher, Scissors and Drawer that contains 2, 2, 4 parts respectively. means the lower the better and means the upper the better.
Category
Roiation Error (*)
Trans ation Esror (m)
Por-part Pose
3D (S6
Inference Time per Image
A-NCSH 8
5.3,
0.054, 0.043
$6.7,40.2
9.0
43.5,
Laptop
ArPERL 11
4.9. 4.7
0.053, 0.066
64.6, 50.4
0.9
EfficentCAPER
3.5, 4.0
0.065, 0.071
88.2, 87.9
0.02
N-
3.7, 22.3, 23.2
0.449, 0.313, 0.324
$2.5, 40.2, 39.6
11.9
Eyoglasses
OMAD [14
7.5.
0,062, 0,103, 0.324
22.8. 20.5, 21.4
2.5
AnPERL 111
6.2.
0.047, 0.095, 0.091
58.6, 46.5, 51.7
EliiientCAPER
7.1,7,255
0.038, 0.109, 0.089
91.8. 82.3, 84.3
0.02
A-NCSH
0.0599
Dishwasher
OMAD [14)
AnPERI ||
600, 6.2
3,9,4.3
0.104, 0.142
0.055. 0.079
66.5, 38.9
69.3,67.6 67.6
1.6
EMiceentCAER
2.5.3.1
0.052, 0.085
91.2,
A. A-NCSH 8
2,0,
0.035, 0.025
46.5, 44.8
6.5
Scissors
AmPERL DT
2.2.
0.031, 0.042
35.6.
40.9. 46.3
0.8
Efficient
5.0,5.1
0.048. 0.099
74.2. 69.2
0.02
NCSH
3.3, 3.3. 2.9
0.145, 0.135, 0.157, 0.075
81.2, 78.4, 82.7
Drawer
OMAD [4
4.4,
4.4.4.4
0.111. 0.143, 0,144, 0.115
75.8. 73.4. 70.2, 71.3
AnPERL 1
3.5. 3.5. 3,5
0.061, 0.112, 0.121,
84.8,
18.6, 79.0.
EMcicentCAER
2.0. 2.0. 2.0, 2.0
0,041, 0,086, 0,090, 0,080
93.2, 87.1. 17.1.877..11111 88.4
0.02
5.2 Experiments on ArtImage Dataset
We report the results of EfficinnCAPPE evaluated on the synthetic dataset containing the articulated objects from PartNet-Mobilits in Table D As it can be seen, EfficientCAPER achieves optimal or suboptimal results on rotation етrог in most categories. We can see that the high estimating performance with only 2.0". 2.6 on rotation error for category Dishwasher. This can be explained by that our joint-centric representation for articulation indicates significant weakening of the impact of self-occlusion. For category Drawer, our method achieves excellent performance with only 0.04 m, 0.086m, 0.090m, and 0.080m translation error. This can be attributed to the framework's 's accurate

---

jOs
Figure 4: Qualitative results ArtImage(top), RcAmtMix(bottom left) and RobotArm(bottoom right).
estimation for state and orientation of the prismatic joint. Obviously, our proposed method can predict 3D scale very well. In terms of inference time, many previous methods typically predict the NOCS coordinate for each observed point and rely on the Umeyama algorithm and the RANSAC method for pose recovery [12 [8, our method can regress the pose without any algorithmic optimization or retrieval method. EfficientCAPER achieves real-time prediction at speed of approximately 50 FPS. Qualitative Results are shown in Figure 4
5.3
Ablation Study
Effect of Articulation Pose Canonicalization. We investigate the effect of canonical point cloud for estimating joint states and 3D scale where experiment ArtImage and Table 2 illustraaees the results. Note that we show joint state error and 3D scale score here since the articulation pose canonicalization module is only employed in the second stage our EfficientCAPER Here our 3D IoU calculation method is as follows: After predicting the length, width, and height of each part, we construct the predicted 31) bounding box and the ground truth 3D bounding box centered at [0,0,0]. We then calculate the 3D scale score between the two bounding boxes. It is obvious that canonicalization of point cloud proposed by us can improve estimating performance a since the network can be more sensitive to kinematic prior and shape prior in the canonical coordinate system.
Table 2: Experiments on the effect articulation pose canonicalization (indicated as APC here) module.
Category
APC
Joint State Erver
3D Scale Score(%)
Table 3: Pose errors for 'Drawer' category with vary- ing occlusion levels.
Laptop
1.8"
86.2, 86.8
Rygglasses
7.8
8.T"
61.3. 51,8,
Occlusion Level
Rotation Error(")
Translation Error(m)
0%%40%
1.9
0.037
Dishwasher
1.8"
2.4"
26-2, 59.6
30.2, 66.2
80%-800
2.0
0.062
80%-1000%
2.0
0.095
Scissors
2,2
56.5, 49.7
Drawer
0.098m, O. 101m, 0.070m
58,, 57.5, 641.7, 65.2
0.075mm, 0.076mm, 0.050m
59.4, 68.4
Self-Occlusion Analysis. To further investigate the robustness under self-occlusion of our Efficient- CAPER, we split the test samples of category Drawer into three subsets based on occlusion level since self-occlusion problems are more common in drawers. The occlusion level is defined as the visible ratio of the points compared to the whole part points, which is similar to A-NCSH [8]. In detail, there are three subsets with various visibility in this experiment: (0%, 40%), (40%, 80%) and (80%, 100%). Here we use mean rotation error and mean translation error that uniformly evaluate the pose estimation performance. Table 3lblesss
shows the experimental result. With an increasing occlusion level, rotation eтror remains essentially unchanged thanks to our joint-centric method. Our method achieves a small translation error for occlusion level rates up to 80%, even in extreme cases with occlusion rates ranging from 80% to 100%, our method still yields favorable result with translation error of 0.095m. Therefore, can be concluded that our method works well in solving the self-occlusion problem. More ablation study can be found in Section A.5.I

---

5.4 Generalization Capacity
Experiments on Semi-Synthetiic Scenarios. First, we assess our method for the estimation of articulated object poses using the dataset known as ReArtMix, which incorporates semi-synthetic scenarios. The detailed results are shown in Table Compared the baseline, our method shows excellent performance on ReArtMix, obtaining better rotation estimation on all categories. We can see that the accurate translation estimation with only 0.003m, 0.005m on translation error for category Cutter. The qualitative results are shown in Figure
Experiments on Real-world Scenarios. To investigate the performance in real-worke scenarios, we also conduct training and evaluation of EfficientCAPER using the 7-part RobotArm dataset. It is important to acknowledge that the robot arm accumulates errois from the first part the end effector due to the multi-depth structure of the robot arm instance, the results in Table 3 show that our method performs well with objects with diverse structures. More qualitative results are shown in Section A.6
Table 4: Pose estimation results on ReArtMix dataset.
Per-part Pose
Category
Method
Rotation Error
Translation Error (m)
3D IDIOU(%)
ReArtNet 13
3,4, 3.6
0.025, 0.034
52.1, 49.3
Box
EfficientCAPER
0.9. 1.7
0,004, 0.015
96,4, 88.9
ReArtNet [13
4.9,
0.032, 0.030
40.1,
Stapler
EFficientCAPER
0.9, 1.6
0.006, 0.019
93.8, 84.5
ReArtNet
3.0,
0.016, 0.017
35.5,
Cutter
ElficienICAPER
1.6,
0.003, 0.004
96.2, 95.7
ReArtNet [3
5.6, 5.3
0.011, 0.013
26.3, 24.3
Scissors
EfficientCAPER
2.3. 4.8
0.006, 0.015
93.1, 87.1
ReArtNet [33
3.1,
0.021, 0.019
51.5, 50.2
Drawer
EfficientCAAPER
0.7, 0.7
0.006, 0.007
95.3, 94.2
Table 5: Pose estimation results on 7-part RobotArm dataset
Per-part Rotation Error (9)
Part ID
5
A-NCSH 8
7.8
7.9
10.3
10.5
11.2
16.4
23.5
EfficienCCAPER
0.04
5.9
7.6
7.3
7.7
11.6
14.6
Per-part Translation Error (m)
Part ID
2
4
5
7
A-NCSH 8
0.012
0.044
0.067
0.066
0.079
0.236
0.403
EfficientCAPER
0.014
0.020
0.048
0.049
0.126
0.124
0.304
6 Conclusion
In this paper, we propose end-to-end approach, namely EfficientCAPER, for solving fast and robust category-level articulated object pose estimation problems. We formulate this task as two-stage framework for the free part and constrained part pose estimation separately that utilizes joint-centric pose modeling to split a whole articulated object into two groups of parts. Experiments demonstrate that EfficientCAPER can obtain of-te-of- estimating performance on various datasets and scenarios, with strong robustness in severe self-occlusion scenes. Future works will focus on investigating the articulated objects with various kinematic structures in the same category or cross-category setting.
Acknowledgements
This research was supported by the Baima Lake Laboratory Joint Funds of the Zhejiang Provincial Natural Science Foundation of China under Grant No. LBMHD24F030002 the National Natural Science Foundation of China under Grant 62373329 and National Natural Science Foundation of China under Grant 62302143 and Anhui Provincial Natural Science Foundation under Grant 2308085QF207 and National Natural Science Foundation of China under Grant 62372150, and Beijing Xiao Yu Intelligence Manufacturing Ltd Inc, Beijing China.
10

---

References
[1] Xinke Deng, Yu Xiang, Arsalan Mousavian, Clemens Eppner, Timothy Bretl, and Dieter Fox. Self-supervised 6d object pose estimation for robot manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), May 2020.
[2] Eric Marchand, Hideaki Uchiyama, and Fabien Spindler. Pose estimation for augmented reality: A hands-on survey. IEEE Transactions on Visualization and Computer Graphics, page 2633-2651, Dec 2016.
[3] Yongzhi Su, Jason Rambach, Nareg Minaskan, Paul Lesur, Alain Pagani, and Didier Stricker. Deep multi-stase object pose estimation for augmented reality assembly. 2019 JEEE Interna- tional Symposium on Mixed and Augmented Reality Adjuncr (ISMAR- Adjjunt),, Oct 2019.
[4] Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Todatddundesstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2020.
[5] Yongming Wen, Yiquan Fang, Junhao Cai, Kimwa Tung, and Hui Cheng. Geen: Geometric constraint co-attention network for 6d object pose estimation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 2671 2671-2679, 2021.
[6] Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, and Silvio Savarese. Densefusion: 6d object pose estimation by iterative dense fusion. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 3343-3352, 2019.
[7] Yifei Shi, Junwen Huang, Xin Xu, Yifan Zhang, and Kai Xu. Stablepose: Learning 6d object poses from geometrically stable patches. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 15222-15231, 2021.
[8] Xiaolong Li, He Wang. Li Yi, Leonidas J Guibas, A Lynn Abbott, and Shuran Song. Category- level articulated object pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3706-3715, 2020.
[9] Liu Liu, Wenqiang Xu, Haoyuan Fu, Sucheng Qian, Qiaojun Yu, Yang Han, and Cewu Lu. Akb-48: real-world articulated object knowledge base. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14809-14818, 2022.
[10] Liu Liu, Han Xue, Wenqiang Xu, Haoyuan Fu, and Cewu Lu. Towards real-world category-level articulation pose estimation. IEEE Transactions on Image Processing, page 1072-1083, Jan 2022.
[11] Liu Liu, Jianming Du, Hao Wu, Xun Yang, Zhenguang Liu, Richang Hong, and Meng Wang. Category-level articulated object 9d pose estimation via reinforcement learning. In Proceedings of the 31 ACM International Conference on Multimedia, pages 728-736, 2023.
[12] He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019.
[13] Liu Liu, Han Xue, Wenqiang Xu, Haoyuan Fu, and Cewu Lu. Toward real-world category- articulation pose estimation. IEEE Transactions on Image Processing, 31:1072-100 1083, 2022.
[14] Han Xue, Liu Liu, Wenqiang Xu, Haoyuan Fu, and Cewu Lu. Omad: Object model with articulated deformations for pose estimation and retrieval. arXiv preprint 1720342 2021.
[15] Wei Chen, Jia, Hyung Jin Chang, Jinming Duan, Linlin Shen, and Ales Leonardis. Fs-net: Fast shape-based network for category-level 6d object pose estimation with decoupled rotation mechanism. In Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pages 1581-1590, 2021.

---

[16] Linfang Zheng, Chen Wang, Yinghan Sun, Esha Dasgupta, Hua Chen, Aleš Leonardis, Wei Zhang, and Hyung Jin Chang. Hs-pose: Hybrid scope feature extraction for category-level object pose estimation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 17163-17173, 2023.
[17] Eric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, and Carsten Rother. Learning 6d object pose estimation using 3d object coordinates. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceed- ings, Part II 13, pages 536-53 1516-551.55 Springer, 2014.
[18] Zhe Cao, Yaser Sheikh, and Natasha Kholgade Banerjee. Real-time scalable 6dof pose estima- tion for textureless objects. In 2016 IEEE Internalional conference on Robotics and Automation (ICRA), pages 2441-2448 TEEE, 2016.
[19] Guangjun Shi, Xiangyang Xu, and Yaping Dai. Sift feature point matching based on improved ransac algorithm. In 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics, volume 1, pages 474-477. IEEE, 2013.
[20] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In Computer Vision-ECCV 2006: 9th European Conference Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings. Part 9, pages
Springer, 2006.
[21] Bugra Tekin, Sudipta N Sinha, and Pascal Fua. Real-time seamless single shot 6d object pose prediction. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 292-301, 2018.
[22] Dengsheng Chen, Jun Li, Zheng Wang. and Kai Xu. Learning canonical shape space for category-level 6d object pose and size estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1 11973-119823 2020.
[23] Haitao Lin, Zichang Liu, Chilam Cheang, Yanwei Fu, Guodong Guo, and Xiangyang Xue. Sar-net: Shape alignment and recovery network for category-level 6d object pose and size estimation. Jun 2021.
[24] Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, and Federico Tombari. Gpv-pose: Category-level object pose estimation via somnetr--guided point- wise voting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6781-6791, 2022.
[25] Zhi-Hao Lin, Sheng- Yu Huang, and Yu-Chiang Frank Wang. Convolution in the cloud: Learn- ing deformable kernels in 3d graph convolution networks for point cloud analysis. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2020.
[26] Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. Convolutional pose machines. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 4724 4732, 2016.
[27] Sijin Li and Antoni B Chan. 3d human pose estimation from monocular images with deep convolutional neural network. In Computer Vision-ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Revised Selected Papers, Part Il 12, pages 332-347. Springer, 2015.
12

---

A Appendix supplemental material
A.1 Overview
In the appendix, present additional information and experiments of EfficientCAPER. An overview of our supplementary material goes as follows: Firstly, we discuss the limitations of the work in Section A.2 Then, explain how choose baseline and introduce more details of the network in
Section A.3 In addition, we describe our training and inference strategies detailedly in Section AA.4
To demonstrate the benefits of our approach, we have also provided some additional experimental results in Section
A.5 Lastly, qualitative results on three datasets are shown in Section A.6
A.2 Limitations
Through analysis of the experimental results, our method heavily relies on free part pose estimation accuracy. Considering limited by the small size of the dataset and the relatively simple structures current dataset, our method's superiority may not fully realized. However, EfficientCAPER robust objects that have diverse structures such robot-arm in the RobotArm dataset.
A.3
The Choice of Baseline
A.3.1
GPV-Pose
For free part pose estimation, we can treat it as a category-level rigid pose estimation task. We follow GPV-Pose (24) since its good utilization of geometric information, showing good performance and robustness to this task. Apart from rotation matrix decomposition and residual translation prediction, there are two other branches to supervise the pose: symmetric reconstruction and bounding box voting. All classes in ArtImage symmetric about the yo2 plane, so that the symmetric reconstruction module can supervise the estimation the plane normals in these two directions. Unlike GPV-Pose, only supervise the rotation in bounding box voting. Apart from the aforementioned aspects, the design of the loss function and the selection of hyper-parameter for these two modules consistent with GPV-Pose.
choose the y-axis and z-axis
A.3.2 HS-Pose
HS-Pose [16] proposed the HS-layer which extends from 3D-GC, can better perceive local and global geometric structure, encode translation and scale information. Due to the superiority of HS- HS-layer, we have replaced 3D-GC with the -S-Encoderr We need to choose the essential support number for the graph convolution network is 7 and the neighbor number for RF-F and ORL 20.
The
A.4 Training and Inference Protocol
Training. In our approach, we train on each object category separately, and the entire training process is divided into two stages. the first stage, we feed a partial point cloud to the HS-Econder to predict the pose of the free part. In the second stage, we need to utilize the pose of the free part to canonicalize the input point cloud, i.e. transforming the point cloud from the camera view to a "pseudo-canonical" view. "Pseudo" means the pose contains errors, the canonical space in this case is not standard. We have designed a special noise addition strategy to implement the training of the second stage. First, we can separately calculate the average rotation error and average translation error by evaluating the pose results of stage one. Then, we add noise to the gt pose to make the noisy pose have approximately the same rotation error and translation error compared to the ground truth. We use the noisy pose to canonicalize the point cloud and we feed the canonical point cloud to the other HS Encoder for training the model of stage two.
Inference. After training, we can separately obtain the best- best-performing models from stage one and stage two respectively. We integrate the two stages into a single inference script. First, we feed the point cloud to the model of stage one to get the pose of the free part. Then we use the pose to canonicalize the point cloud to get the canonical point cloud. stage two, we input the canonical point cloud to the trained model, the model will output joint state estimation, joint parameters
13

---

estimation, 3D scale estimation, and part segmentation. Finally, the poses of the constrained parts are calculated based on the joint state, joint parameters, and the free part's pose. As discussed above, we can directly output the pose and 3D scale of every part.
A.5
Experiments
A.5.1 Ablation Study
Single Free Part Input v.s. Whole Object Input. We have conducted additional set of experiments to validate the assistance of constrained part information in estimating the free part pose. Results are shown in Table 6 We see that rotation error decreases when we use constrained parts because the neural network can learn better through a more comprehensive point cloud with particularly significant increase of 90% in the category Dishwasher. However, result on increased translation error since the original position of the free part may move after adding the constrained part. Overall, it is necessary to feed point clouds of all parts of articulation because allows our two-stage network to simultaneously output the poses of all parts with just single input.
get passable
Table 6: Experiments comparing single free part as imput between both free part and constrained parts as input. Note that we only investigate this analysis in the first stage of our Efficien(CAPER.
Category
Input Parts
Rotation Error())
Translation Error(m)
free
3.8
0.059
Laptop
freetconstrained
3.5
0.061
free
3.4
0.053
Eyeglasses
freetconstrained
3.1
0.037
Free
20.4
0.057
Dishwasher
free+constrained
2.5
0.053
Free
5.0
0.043
Scissors
freetconstrained
5.0
0.048
Free
3.4
0.049
Drawer
fressconttrained
2.0
0.043
A.5.2 Detiled Results
Joint Parameters Estimation Results. In stage two, we also design a branch to estimate the parameters of the axes. All the joint parameters are evaluated in canonical space. For each revolute joint, we use mean angle eтror to evaluate the orientation of the joint axis, and use mean distance error to evaluate the distance between the predicted axis position and the gt axis position. For each prismatic joint, we compute the orientation error of the translation axis. The results are shown in Table 7 Table and Table For orientation, our EfficientCAPER can obtain good performance on all classes. For location, it is obvious that our method can still achieve satisfactory results on three datasets.
Table 7: Experiments estimating joint parameters in the canonical space on Artlmage.
Table 8: Experiments estimating joint parameters in the canonical space on ReArtMix.
Category
Angle Error (")
Joint Parameter
Distance Error (m)
Category
Angle Error (") Distance Error (m)
Joint Parameter
Laptop
0,00
0,037
Box
0.43
0.011
Eyeglasses
0.00, 0.67
0.031, 0.024
Stapler
0.63
0.0T2
Dishwasher
0.00
0.033
Cutter
0.48
Scissors
0.00
0.097
Scissors
0.80
0.005
Drawer
0.04, 0.09, 0.00
Drawer 0.11
Part Segmentation Results. stage two, our model can output part segmentation results which is an optional branch. We conduct two sets of experiments on part segmentation in both camera view and canonical view, and we calculate the mean IOU of all parts. The results shown in Table demonstrate that our framework can achieve performance with an IOU greater than 90% on four categories of objects. is proved that articulation pose canonicalization can also improve the part segmentation performance.
14

---

Table Experiments estimating joint parameters canonical space on 7-part RobotArm dataset
Joint Parameter
Joint ID
4
5
Angle Error (*)
0.005
0.008
0.019
0.114
0.018
0.012
Distance Error (m)
0.012
0.013
0.014
0.028
0.023
0.038
Table 10: Part Segmentation results. APC means articulation pose canonicalization module.
Category
APC
Mean IOU(
94.3
Laptop
95.0
88.3
Eyeglasses
90.1
96.7
Dishwasher
97.7
88.47
Scissors
V
77.5
79.4
Drawer
92.3
A.6 Qualitative Results
In this section, we show more qualitative results on ArtImage 14]. ReArtMix I3 and RobotArm 1K datasets. In Figure we show the visualizations of Ground Truth, A-NCSH and our method on ArtImage. It can be observed that compared to A-NCSH, our method is capable of better predicting the 3D bounding boxes of each part of the object camera space and gets very close results to the Ground Truth on Laptop, Dishwasher and Drawer. Figure 66 and Figure / are the results from ReArtMix and RobotArm. From these results, it can be seen that our method can accurately estimate the pose and 3D scale even in some severe self-occlusion views.
A.7 Ethics Statement and Boarder Impact
Our method has the potential develop the home-assisting robot, thus contributing social welfare. We evaluate our method in synthesized or numan-collectee datasets, which may introduce data bias. However, similar studies also have such general concerns. We do not see any possible major harm in our study.
A-NOSH
EmfidntCA PER
Figure 5: Qualitative results on the ArtImage datasct. We show five categories from ArtImage. The bascline is A-NCSH [].
15

---

Figure 6: Qualitative results on the ReArtMix dataset.
Wey
Freso
Figure 7: Qualitative results the RobotArm datasct.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Section A.2
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We will open-source the checkpoints and all implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification:
We will open-source the dataset and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ Caleaes//ooSSSuubiiseeee wwt.Suubmisiioooiioooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section 5.1 we provide the setting of experiments. We describe the training and inference protocols Section A.4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The effect of random seed could almost be negligible since we set the same initiation seed during experiments. Reproducibility can guaranteed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
19

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In Section. 5.1 we describe the computer resources we used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://ppp.....oooooooooooo//o/e
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics, the discussion is presented in Section
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: Our work has no negative societal impacts, the discussion is presented in Section A.7
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the datasets we used are publicly available datasets.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "YP1ZznHdL8", "reviewer": "Reviewer_UxV6", "summary": "This paper introduces an end-to-end framework designed for category-level articulated object pose estimation. Specifically, it addresses the complexity of articulated objects by using a joint-centric approach that divides the task into two stages: estimating the pose of free parts and then the constrained parts. The method is evaluated on three datasets, demonstrating its effectiveness.", "questions": "Please refer to the weakness part.", "limitations": "The limitations are discussed in the supplementary material.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper is well-organized and easy to follow.\n\n2.It is interesting to estimate the category-level articulated object pose from the joint-centric perspective. \n\n3.The proposed method is shown to be effective on different datasets.", "weaknesses": "1.It is not clear to me how the free part is defined. Actually,  the “free” and “constrained” are defined relatively and can be changed in some sense. Will this cause the ambiguity in the estimation?\n\n2.In the paper, it is claimed that the part-centric can fix the self-occlusion issues, but I didn’t fully understand how the proposed “Joint-Centric Articulation Pose Modeling“ can handle this effectively. \n\n3.The framework takes a cascaded architecture, and I am wondering how the free part pose estimation in the first stage will affect the following constrained part estimation. \n\n4.For the ArtImage dataset, the translation estimation of the proposed method is not stable, and for the “Scissors” case, its performance is much worse than others in terms of rotation and translation errors. More expatiation is expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and evaluation on our submission. Please find our response below.  \n\n***W1: Unclear definition of the free part with relative change against the constrained part***.  \n\nThe \"free\" and \"constrained\" parts are indeed interchangeable, however, this does not cause ambiguity in the estimation process. Concretely, for the pose estimation of the free part, all parts of the articulated object will be input together, treating them as a rigid object without distinguishing between the free and constrained parts. Since all the parts can rotate around each other interchangeably, for simplicity, we pre-define the free part according to the joint location(this configuration is interchangeable). \n\nIn the second stage, we first perform pose canonicalization via the prediction from stage 1 to align the free part. Then the network tries to learn the joint state, which is distinguished from canonical space and rest state. The Parts with obvious movement are considered as constrained parts, whose poses are optimized by our joint-centric modeling way (**please see Figure 1 in the attached PDF**).  \n\n***W2: It is claimed that the part-centric can fix the self-occlusion issues, but I didn’t fully understand how the proposed “Joint-Centric Articulation Pose Modeling“ can handle this effectively***.  \n\nIn the proposed joint-centric modeling, the degrees of freedom for the constrained part are reduced from 3D to 1D (due to kinematic constraints from the joint), and this 1D constraint information is well reflected in the joint state. Therefore, even in the cases of severe self-occlusion, the network can still learn the joint state from few point clouds, regressing 1D directional information naturally performs better than regressing 3D direction information, achieving superior performance.\n\n***W3: The framework takes a cascaded architecture, and I am wondering how the free part pose estimation in the first stage will affect the following constrained part estimation***.  \n\nAs mentioned in Section 4.4 in the main paper, this strategy can be conducive to the following aspects: (1) We use the predicted pose of free part to align the free part, transforming the joint state estimation task from camera space to canonical space, making sure the network can effectively learn the joint state in canonical space. This is a simple and efficient way but still under-studied in existing works (see reviewer Gf45); (2) The canonicalization process applied to the point cloud eliminates the effects of varying joint configurations in movable rigid parts. This process reduces uncertainty, enhances stability and accuracy, and provides both shape and kinematic priors, significantly aiding in the regression of joint states. The ablation study in Table 2 validates these advantages.  \n\n***W4: For the ArtImage dataset, the translation estimation of the proposed method is not stable, and for the “Scissors” case, its performance is much worse than others in terms of rotation and translation errors. More explanation is expected***.  \n\nEstimating translation in category-level pose estimation is challenging. As shown in Table 1, the translation error of constrained part is higher than free part, since it relies on the estimation of free part and joint parameters. However, compared to baselines achieving state-of-the-art performance only for specific categories, our method can achieve comprehensive performance in each category.   \nFor category scissors, typically exhibit complex shapes and partially symmetrical features. This complexity makes it challenging to accurately capture the details of scissors, while symmetry may lead to multiple possible poses.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WNYjNvSSgw", "reviewer": "Reviewer_UsR2", "summary": "This paper introduces a new approach for estimating the pose of category-level articulated objects. The proposed framework, EfficientCAPER, aims to address the challenges of kinematic constraints, self-occlusion, and optimization requirements. The method eliminates the need for optimization post-processing by utilizing a joint-centric pose modeling mechanism. The approach is evaluated on three diverse datasets: ArtImage, ReArtMix, and RobotArm, demonstrating its effectiveness and generalization ability.", "questions": "Please check the weakness w.r.t free part.", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  EfficientCAPER introduces a joint-centric articulation pose modeling mechanism, which differs from traditional part-centric approaches. This mechanism allows the learning of each part’s pose as a joint state representation, improving the network's ability to handle self-occlusion and kinematic constraints.\n2. The method splits the pose estimation process into two stages. First, it estimates the pose of the free part using a decoupled rotation representation. Then, it canonicalizes the input point cloud to estimate the poses of constrained parts by predicting joint parameters and states.\n3. EfficientCAPER eliminates the need for intermediate variable estimation or optimization procedures, making the pose estimation process more efficient and straightforward.", "weaknesses": "1. The method heavily relies on the accuracy of the free part pose estimation. Any error in this stage can propagate and affect the estimation of the constrained parts.\n2.  Although the method is tested on three datasets, the datasets might not cover all possible variations and complexities of real-world articulated objects. This might limit the generalizability of the method to new, unseen categories.\n3. There is no clear definition of what constitutes a free part. The concept appears to be relative, and each part could potentially be considered a free part depending on the context (like the two parts of scissors). This ambiguity might complicate the learning process.\n4.  The difference between a free part and a root part is not well-defined, and they seem to be essentially the same.  \n5.  it is not clear in the method section if the free part is pre-defined. As in the ablations, there are inputs with free-part only.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate that the reviewer understands and recognizes the contributions of this work. We address the main concerns as follows.  \n\n***W1: The method heavily relies on the accuracy of the free part pose estimation. Any error in this stage can propagate and affect the estimation of the constrained parts***.\n\nAs shown in Table 1, our method has earned satisfactory results in estimating poses both the free part and the constrained part. While our method does exhibit cumulative errors, even in datasets of high complexity such as RobotArm, we have also achieved notably superior results compared to the baseline approaches.\n\n***W2: Although the method is tested on three datasets, the datasets might not cover all possible variations and complexities of real-world articulated objects. This might limit the generalizability of the method to new, unseen categories***.\n\nWe follow the suggestion from A-NCSH that the number of rigid parts and kinematic structures is constant for all the objects in the same category, and the goal is to estimate the pose of unseen objects within the category. We consider objects in the physical world to generally fall into two categories: rigid objects and articulated objects. Articulated objects can be further classified into two types—those with rotational joints and those with prismatic joints. Our datasets already include both types of articulated objects, including highly complex ones like robot arms. Therefore, our method can generalize to both kinds of articulated objects within the same category.\n\n***W3: There is no clear definition of what constitutes a free part. The concept appears to be relative, and each part could potentially be considered a free part depending on the context (like the two parts of scissors). This ambiguity might complicate the learning process***.\n\nFor the pose estimation of the free part in the first stage,  we input all parts of the articulated object together, treating them as a rigid object without distinguishing between the free and constrained parts. Since all the parts can rotate around each other interchangeably, for simplicity, we pre-define the free part according to the joint location (this configuration is interchangeable), all objects share the same setting within the intra-category.\n\nFor the pose estimation of the constrained part in the second stage, we first canonicalize the point clouds via the prediction from stage 1 to align the free part. Then the network tries to learn the joint state, which is distinguished from canonical space and rest state(**please see Figure 1 in the attached PDF**). The Parts with obvious movement are considered constrained parts, whose poses are optimized by our joint-centric modeling way.\n\n***W4: The difference between a free part and a root part is not well-defined, and they seem to be essentially the same***.\n\n**Please check Figure 2 in the attached PDF, we show the main difference between a free part and a root part**.\n\nIt predominantly suggests two ways in defining root part for articulation structure:\n\n**Definition I**: the root part is defined as the world, and there is a fixed joint connecting the world to the base part (this definition is known in PartNet-Mobility dataset from SAPIEN[1]). In this definition, the root part is a virtual part that contains no physical parts. \n\n**Definition II**: the root part is directly defined as the base part known in AKB-48 dataset [2]. \n\nIn this work, the free part is more similar to definition II of AKB-48 in terms of the physical base part in the articulation structure, but the differences are: (1) we name the part type following the kinematic motion type, i.e. we assume there is a free joint accompanying with the free part. So every part's pose in the articulated object can be converted to the task that how to estimate the joint parameters and state. (2) \"free\" means it can move at arbitrary position and rotation in the 3D space, while \"constrained\" means the part's motion is limited to the free part's motion.\n\n***W5: It is not clear in the method section if the free part is pre-defined. As in the ablations, there are inputs with free-part only***.\n\nFree part and constrained part is our proposed articulation modeling way in this work, where the free part is pre-defined for each category. As it is stated in our work, there are two stages to solving the articulation pose estimation. The first one is to canonicalize the object by predicting the free part's pose and the second one is to predict the joint states of constrained parts (since it is obviously easy to predict joint state in canonical space). In this case, we find that only free part input in the first stage might cause symmetry problem that affects the canonicalization performance. Thus, we provide an ablation study in A.5.1, that shows when the whole object is inputted, the free part pose estimation can be achieved better so the canonicalization is more accurate. Note that in this ablation study, we only investigate the effect of input on the first stage.\n\n[1] SAPIEN: A SimulAted Part-based Interactive ENvironment. CVPR. 2020.\n\n[2] AKB-48: A Large-scale Real-world Articulated Object Knowledge Base. CVPR. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u6EskaOl9j", "reviewer": "Reviewer_cXQd", "summary": "The paper investigates category-level articulated object pose estimation with a new framework. The framework consists of two stages, including the first for estimation the pose of the free parts using decoupled rotation representation and the second for estimation the pose of the constrained parts by predicting the joint parameters and states as replacements. Experimental results on ArtImage, ReArtMix, and RobotArm demonstrate the outstanding pose estimation performance for articulated objects.", "questions": "1.What is the main difference and challenge of category-level articulated object pose estimation in compared with category-level object pose estimation?\n\n2.It is interesting that what is the performance of the category-level object pose estimation methods (like GPV-Pose) on ArtImage, ReArtMix, and RobotArm?", "limitations": "The authors have adequately addressed the limitation.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.An end-to-end Category-level Articulated object Pose EstimatoR composed of two stages for free parts and constrained parts.\n\n2.Free part pose estimation with decoupled rotation and constrained part pose estimation with joint state replacement. \n\n3.Experimental results demonstrate that the proposed method achieves the less object pose estimation error in comparison with other methods on ArtImage, ReArtMix, and RobotArm.", "weaknesses": "1.The proposed method is called EfficientCAPER, which indicates that the running efficiency is a key factor. However, the expression of this is lacked. For one thing, the reason that why the proposed method can achieve higher FPS than other methods should be explained more clearly. For another, the detailed discussion of running time is also suggested to add. \n\n2.In Table 2, experiments on the effect of articulation pose canonicalization are conducted. However, different metrics (Joint State Error and 3D Scale Score) are used. It is suggested to use the same metrics (Rotation Error, Translation Error (m) and 3D IOU) as Table 1, which can prove the effect of articulation pose canonicalization more clearly. \n\n3.The experimental results demonstrate obvious improvements in Tables 1,3,4. What is the key contribution to the improvements? It is suggested to add more ablation experiments to show this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, and for your thoughtful comments and questions. They will certainly help improve the revised paper. Below we respond in detail to your questions.\n\n***W1: The proposed method is called EfficientCAPER, which indicates that the running efficiency is a key factor. However, the expression of this is lacked. For one thing, the reason that why the proposed method can achieve higher FPS than other methods should be explained more clearly. For another, the detailed discussion of running time is also suggested to add***.\n\nThe analysis of inference time is reported in Section 5.2. More discussions are in below: (1) Compared to some dense prediction methods like NOCS, we do not perform point-by-point predictions, achieving lower computational costs and faster running speed; (2) Our method is end-to-end without any time-consuming post-procession like OMAD (It estimates coarse pose and uses a retrieval-based method to get refined results).\n\n***W2: In Table 2, experiments on the effect of articulation pose canonicalization are conducted. However, different metrics (Joint State Error and 3D Scale Score) are used. It is suggested to use the same metrics (Rotation Error, Translation Error (m) and 3D IOU) as Table 1, which can prove the effect of articulation pose canonicalization more clearly***.\n\nThe joint state is the key target after canonicalization, which is a more intuitive metric in our proposed joint-centric perspective. We additionally provide estimated results of the constrained part with the same metrics as Table 1 in the main paper, it shows that the canonicalization module employed in the second stage is beneficial for estimating the pose of constrained part.\n\n| Category     | APC | Rotation Error (°) | Translation Error (m) | 3D IOU (%) |\n| ------------ | --- | ------------------- | --------------------- | ---------- |\n| **Laptop**   |     | 5.1   | 0.073   | 84.8   |\n|              | ✔   | 3.3      | 0.071      | 87.9  |\n| **Eyeglasses** |   | 8.8, 9.7    | 0.113, 0.094      | 78.4, 79.2 |\n|              | ✔   | 7.2, 5.8            | 0.109, 0.089          | 82.3, 84.3 |\n| **Dishwasher** |   | 3.3        | 0.096            | 79.3 |\n|              | ✔   | 2.6                 | 0.085                 | 83.9  |\n| **Scissors** |     | 5.9           | 0.112                 | 69.5 |\n|              | ✔   | 5.1                 | 0.099                 | 69.2 |\n| **Drawer**   |     | 1.2, 1.2, 1.2       | 0.102, 0.124, 0.100   | 81.8, 80.5, 83.6 |\n|              | ✔   | 1.2, 1.2, 1.2       | 0.086, 0.090, 0.080   | 87.1, 87.1, 88.4 |\n|\n\n\n***W3: The experimental results demonstrate obvious improvements in Tables 1,3,4. What is the key contribution to the improvements? It is suggested to add more ablation experiments to show this***.\n\nAs shown in Table 2 in the main paper, it is justified that canonicalization contributes greatly to the framework, so we conducted the ablation study for this module. Our pose canonicalization operation facilitates better learning of joint parameters and states. Additionally, we also conducted an ablation study in articulation pose modeling. We use GPV-Pose to directly output the poses of all parts of the category laptop under part-centric(i.e., treating each part of the articulated object as an independent rigid object), the result shows that our joint-centric modeling method is beneficial to the estimation of the constrained part. \n\n| Category | Pose Modeling Method | Rotation Error ($^\\circ$) | Translation Error (m) | 3D IOU(%) |\n|----------|-----------------------|--------------------------|-----------------------|-----------|\n| **Laptop**   | Part-centric          | 10.5, 12.7              | 0.205, 0.275         | 49.1, 46.3|\n|          | Joint-centric         | 2.3, 3.3                | 0.065, 0.071         | 88.2, 87.9 |\n|\n\nFurthermore, we explore the effect of the backbone. We have conducted an ablation study on category laptop to estimate the pose of free part , using the MLP, 3D-GC, HS-Encoder as backbones respectively. The results in the table below show that the choice of HS-Encoder is effective.  \n\n| Category | Backbone   | Rotation Error ($^\\circ$) | Translation Error (m) | 3D IOU(%) |\n|----------|------------|---------------------------|-----------------------|-----------|\n| **Laptop**   | MLP    | 20.5       | 0.203         | 23.8|\n|          | 3D-GC      | 7.5     | 0.099  | 76.3|\n|          | HS-Encoder | 4.8     | 0.073   | 85.4|\n|\n\n\n***Q1: What is the main difference and challenge of category-level articulated object pose estimation in compared with category-level object pose estimation?***\n\n**Estimation complexity:** Compared to rigid objects can be represented by a single pose, describing their position and orientation in 3D space. Category-level articulated object pose estimation involves objects with articulated parts that can move relative to each other while obeying kinematic constraints. This adds complexity as the pose estimation not only involves determining the overall object pose but also capturing the poses of its articulated parts and joint parameters.   \n**Severe self-occlusion:** Category-level articulated object pose estimation will increase dimensionality and variability which may cause self-occlusion. Severe self-occlusion will lead to poor performance, especially where smaller parts are obscured by larger parts from certain camera views. As discussed in section 4.2 in the main paper, the joint-centric pose modeling method can effectively alleviate the issue.\n\n***Q2: It is interesting that what is the performance of the category-level object pose estimation methods (like GPV-Pose) on ArtImage, ReArtMix, and RobotArm?***\n\nWe have conducted this experiment(shown in Table in **W3**), directly regressing pose in part-centric modeling method has inferior performance, since the part-centric pose modeling method does not take count into kinematic constraints.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NSCOJf5fkL", "reviewer": "Reviewer_Gf45", "summary": "This paper proposes EfficientCAPER, a novel method for category-level articulated object pose estimation from input point cloud. The proposed method first estimates the 6D pose of the free part (or static part), then uses this estimated 6D pose to transform the input point cloud into canonical space, and finally segments and regresses the pose of each part. The authors experiment on both synthetic and real-world datasets, showing that the proposed method outperforms existing methods.", "questions": "- Please show some qualitative samples of object being self-occluded at 80-100%.", "limitations": "No, the authors did not mention explicitly the limitation of the proposed method.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The proposed method is simple but efficient as shown in Table 2.\n\n- The idea of transforming the input point cloud into its canonical space seems to be straightforward but it appears not to have been done in previous works.\n\n- The proposed method outperforms existing works on both synthetic and real-world datasets.", "weaknesses": "- The paper misses many important related works such as CAPTRA, Ditto. While these methods do not work in the exact same setting, these methods are relevant and it is worthy to discuss the difference with the proposed method.\n\n- I found the paper is not well-written, making it difficult to understand the true contribution of the paper.\n\n- In Table 3, the authors show the results to prove the robustness of the proposed method against self-occlusion. The self-occlusions are sometimes quite very high such as 80-100%. It would be helpful to know in which cases the object can be occluded at this high rate, along with qualitative samples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read this paper and for asking these meaningful questions. Below we respond in detail to your questions.\n\n***W1: The paper misses many important related works such as CAPTRA, Ditto. While these methods do not work in the exact same setting, these methods are relevant and it is worthy to discuss the difference with the proposed method***.\n\nOur work involves estimating articulated objects, while CAPTRA[1] focuses on the tracking task. In terms of translation, CAPTRA utilizes CoordinateNet to predict normalized coordinates, whereas our approach directly regresses it, which enhances efficiency. Ditto[2] is a model based on implicit neural representations, capable of learning and reconstructing the 3D geometry of articulated objects from visual observations through interactive perception, but it does not perform pose estimation. For the joint parameter estimation module, Ditto uses implicit decoding of axis parameters and states from camera point clouds. Still, our work implicitly decodes axis parameters and explicitly decodes the state in the canonical space. We will incorporate more significant related work in the revised version.\n\n***W2: I found the paper is not well-written, making it difficult to understand the true contribution of the paper***. \n\nWe have summarized our contributions in the introduction from **line 71 to line 80**. In short, our key contributions are **pose canonicalization and joint-centric articulation pose modeling method**. Additionally, we will add a concluding statement at the end of the method section to highlight the contributions of our approach in the revised version.\n\n***W3&Q1: In Table 3, the authors show the results to prove the robustness of the proposed method against self-occlusion. The self-occlusions are sometimes quite very high such as 80-100%. It would be helpful to know in which cases the object can be occluded at this high rate, along with qualitative samples***.\n\nWe presented some cases with severe self-occlusion and qualitative results in **Figure 3 in the attached PDF**. The category drawer suffers from severe self-occlusion due to its complex structure and constrained parts with small size. Thanks to our joint-centric pose modeling, our method can effectively address self-occlusion issues and demonstrate robustness to varying levels of self-occlusion. Therefore, both qualitative and quantitative results show the potential that our method can achieve superior performance on objects with more complex structures.\n\n[1] CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds. 2021. ICCV.  \n[2] Ditto: Building Digital Twins of Articulated Objects from Interaction. 2022. CVPR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,  

**Please see the attached PDF for a one-page PDF with a summary of added experimental figures**.  

We are appreciated by the positive comments of the reviewers on the novelty and significance of our method (Reviewers UxV6, UsR2, cXQd, Gf45), readability (Reviewer UxV6), effectiveness (Reviewers UxV6, UsR2, Gf45), and notable improvements compared with state-of-the-art methods (Reviewers cXQd, Gf45).

We provide point-to-point responses to each reviewer. We look forward to engaging in further discussion with the reviewers, answering questions, and discussing improvements.

Meta-review: This paper is about articulated object pose estimation. After considering all reviews and author comments, the paper received three ratings of "borderline accept" and one of "borderline reject". However, the reviewer recommending reject states that he/she is comfortable with an "accept" if the other reviewers recommend so. The authors provided detailed responses to the reviewer queries; unfortunately, except for one reviewer, the others did not provide updated ratings. In AC's view, the responses seem very specific and address the questions raised. The AC recommends accepting this paper.
Decision: Accept (poster)