Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance
Authors: Xueyi Liu
Ji Zhang
Ruizhen Hu
Haibin Huang
He Wang
Li Yi
Venue: ICLR 2023 poster (2023)

Abstract: Category-level articulated object pose estimation aims to estimate a hierarchy of articulation-aware object poses of an unseen articulated object from a known category. To reduce the heavy annotations needed for supervised learning methods, we present a novel self-supervised strategy that solves this problem without any human labels. Our key idea is to factorize canonical shapes and articulated object poses from input articulated shapes through part-level equivariant shape analysis. Specifically, we first introduce the concept of part-level SE(3) equivariance and devise a network to learn features of such property. Then, through a carefully designed fine-grained pose-shape disentanglement strategy, we expect that canonical spaces to support pose estimation could be induced automatically. Thus, we could further predict articulated object poses as per-part rigid transformations describing how parts transform from their canonical part spaces to the camera space. Extensive experiments demonstrate the effectiveness of our method on both complete and partial point clouds from synthetic and real articulated object datasets.

Published as a conference paper at ICLR 2023
SELF-SUPERVISED CATEGORY-LEVEL ARTICULATED OBJECT POSE ESTIMATION WITH PART-LEVEL SE(3) EQUIVARIANCE
Xueyi Liu Ji Zhang", Ruizhen Hu², Haibin Huang", He Wang", Li Yi
TTsinghua University FFudan University Shennzen University 4 Kuaishou Technology Peking University Shanghai Artificial Intelligence Laboratory
"Shanghai Qi Zhi Institute
ABSTRACT
Category-level articulated object pose estimation aims to estimate hierarchy of articulation-aware object poses of unseen articulated object from a known cate- gory. To reduce the heavy annotations needed for supervised learning methods, we present novel self-supervised strategy that solves this problem without any human labels. Our key idea is to factorize canonical shapes and articulated object poses from input articulated shapes through part-level equivariant shape analysis. Specifi- cally, we first introduce the concept of part-level SE(3) equivariance and devise a network to learn features of such property. Then, through a carefully designed fine-grained pose-shape disentanglement strategy, expect that canonical spaces to support pose estimation could be induced automatically. Thus, we could further prediet articulated object poses as per-part rigid transformations describing how parts transform from their canonical part spaces to the camera space. Extensive ex- periments demonstrate the effectiveness of our method both complete and partial point clouds from synthetic and real articulated object datasets. The project page with code and more information can be found at: gggui-ariiculate--os...ee
INTRODUCTION
Articulated object pose estimation is crucial and fundamental computer vision problem with wide range of applications in robotics, human-object interaction, and augmented reality Katz Brock (000): Mu et al. [202T) Labbé et al. (202T): Jiang et al. (2022): Goyal et al. y0 Li et al. (2020b). Different from 6D pose estimation for rigid objects Tremblay et al. (2018); 8); Xiang et al.
(2017); Sundermeyer al. (2018); Wang et al. 2019a) articulated object pose estimation requires hierarchical pose understanding on both the oeject-level and part-level [i et al. (2000)). This problem has been long studied the instance level where exact CAD model is required understand the pose of a specific instance. Recently, there is trend in estimating category-level object pose such generalize novel instances. Despite such merits, supervised category-level approaches always assume rich annotations that are extremely expensive to acquire Li et al. (2020a); Chi & Song (202I); Liu et al. (2022a). To get rid of such restrictions, we tackle this problem under self-supervised setting instead.
that the algorithm
Given a collection of unsegmented articulated objects in various articulation states with different object poses, our goal is to design a network that can acquire a category-level articulated object pose understanding in self-supervised manner without any human labels such as pose annotations, segmentation labels, or reference frames for pose definition.
The self-supervised category-level articulated object pose estimation problem highly ill-posed since it requires the knowledge of object structure and per-pan poses, which are usually entangled with part shapes. Very few previous works try to solve such a problem or even similar ones. The most related attempt is the work of oi et al. 2021). It tackles the unsupervised category-level pose estimation problem but just for rigid objects. They leverages SE(3) equivariant shape analysis to disentangle the global object pose and shape information so that category aaeegory-aligned canonical object space can emerge. This way the category-leve object poses could be automatically learned by predicting a transformation from the canonical space the camera space. Going beyond rigid objects, estimating

---

Published as a conference paper at ICLR 2023
articulated object poses demands more than just global pose and shape disentanglement. It requires more fine-grained disentanglement of part shape, object structure such as part adjacency relationship. joint states, part poses, and
To achieve such fine-grained disentanglement, we propose to leverage part-level SB(3) equivariant shape analysis. Especially, we introduce the concept of part-level SE(3) equivariant features to equip equivariance with a spatial support. The part-level SE(3) equivariant feature of a local region should only change as its parent part transforms but should not be influenced by the transformation of other parts. This is in contrast to the object-level SE(3) equivariant feature for a local region, which influenced by both the region's 's parent part and other parts. To densely extract part-level SE(3) equivariant features from an articulated shape, we propose a novel pose-awart equivariant point convolution operator. Based on such features, we are able to achieve fine-grained disentanglement which learns three types of information from input shapes: 1) Canonical part shapes, which are invariant to input pose or articulation changes and are category-alignes to provide consistent reference frame for part poses; 2) Object structure, which is also invariant to input pose or articulation changes and contains structural information about the part adjacency relationships, part transformation order, and joint parameters such as pivot points; 3) Articulated object pose, which is composed of series of estimated transformations. Such transformations include per-part rigid transformations which assembles canonical part shapes into canonical object shape, per-part articulated transformation which articulates the canonical object shape to match the input articulation state, and a base part rigid transformation transforming the articulated canonical object to the camera space. To allow such disentangiement, we guide the network learning through self-supervised part-by-part shape reconstruction task that combines the disentangled information recover the input shapes.
With the above self-supervised disentangiement strategy, our method demonstrates the possibility of estimating articulated object poses prove its effectiveness on both complete point clouds and partial point clouds from various categories covering both synthetic and real datasets. On the Part-Mobility Dataset Wang et al. (2019b), our method without the need for any human annotations can already outperform the iterative pose estimation strategy with ground a large margin, e.g. reduce the rotation estimation error by around 30 degrees complete shapes and by 40 degrees on partial shapes. Besides, our method can perform on par with to or even better than supervised methods like NPCS Li et al. (2020a). For instance, of 7.9 rotation estimation error on complete shapes, comparable to NPCS's 5.8 error. We can even outperform NPCS on some specific categories such as partial Eyeglasses. Finally, we prove the effectiveness of our part-level SE(3) equivariance design and the fine-grained disentanglement strategy the ablation study. Our main contributions are summarized follows:
superviedd way for the first time. Extensive experiments
segmentation masks on both complete and partial settings by
can achieve average
To our best knowledge, we are the first that tackles the self-supervised articulated object pose estimation problem.
We design pose-aware equivariant point convolution operator to learn part-level SE(3)- equivariant features.
We propose self-supervised framework to achieve the disentanglement of canonical shape, object structure, and articulated object poses.
2 RELATED WORKS
Unsupervised Part Decomposition for 3D Objects. Decomposing an observed 3D object shape into parts in an unsupervised manner is a recent interest in shape representation learning. Previous works always tend to adopt a generative shape reconstruction task to self-supervise the shape decomposition. They often choose to represent parts via learnable primitive shapes Tulsiani et al. (2017); Kawana al. (2020); Yang & Chen (021)) Paschalidou et al. (2021): DDngg et al. (2020); Zhu et al. (2020): Chen et al. (2020) or non pprmmitiv-bbsse implicit field representation Chen et al. (2019) Kawana jet al. (2021). Shape alignment is a common assumption of such methods to achieve consistent decomposition across different shapes.
Articulated Object Pose Estimation. Pose estimation for articulated objects aims to acquire fine-grained understanding of target articulated objects from both the object level and the part level. The prior work Li et al. (2020a) proposes to estimate object orientations, joint parameters, and
2

---

Published as a conference paper at ICLR 2023
per-part poses in sully-sup setting. They define Articulation-awar Normalized Coordinate Space Hierarchy (ANCSH), composed of the canonical object space and a set of canonical part spaces, consistent representation for articulated objects to support pose estimation. In this work, we also want to estimate hierarchy of articulation-awan object poses but in a totally unsupervised setting. Instead of hand-crafting normalized coordinate spaces, wish to let them be automatically induced during learning.
SE(3) Equivariant Networks. Recently, there is trend of pursuing SE(3)-equivaria and invariant
features through network design Weiler al. (2018) Thomas al. (2018): Fuchs et al. (2020): Zhao
et al. (2020); Chen et al. (2021) Equivariance is achieved by designing kemnlls Thomas al. (2018):
Fuchs et al. (2020) or designing feature convolution strategies Chen et al. (2021) Zhao et al. (022))
In this work, design
part-level SE(3) equivariant feature network based Equivariant Point
Network Chen et al. (2021) for articulated object pose estimation. Common SE(3) equivariant feature
of local region would be affected by both its parent part's and other parts" rigid transformations. By
contrast, its part-level SE(3) equivariant feature would only be affected by its parent part.
3 METHOD
Equlvoriant Festure Extraction
Disnrbonglement
Caoveae Obgec Stepo 2
Tus Propsoa
Casexc Stmps Faztesoaatonn
Fat-ked Fastas Boractiee
Object Smuctaar
Artexaee Faderiatice
June Satee
Bisc Par Rigal Tranderaik
[M
Overview of the proposed supe-super articulated object pose estimation strategy. The method takes a complete or partial point cloud of an articulated object as input, factorizes canonical shapes, object structure, and the articulated object pose from it. The network is trained by shape reconstruction task. Left: A high-level abstraction of our pipeline. Right: An illustrate of decomposed information for shape reconstruction. Green lines (<) denote the iterative pose estimation process.
Figure 1:
We present our method for self-supervised category-level articulated object pose estimation. We first propose to learn part-level SEE() equivariant features through novel pose-aware equivariant point convolution module (sec. B.I.. Based on such features, then design a disentanglement strategy to factorize an arbitrarily posed 3D point cloud into three types of information. Such information includes canonical shape with category-aligned pose and articulation state, the object structure describing the part adjacency and joints, as well as the articulated object pose (sec. (3.2). We find part-level SE(3) equivariant features are key to achieve the factorization above. Further, we adopt a part-by-part shape reconstruction task that combines the factorized information for shape reconstruction self-supervise the factorization (sec. 3.3 Our method assumes a category-leve setting where input shapes have the same kinematic chain. For notations frequently used in the following text, N, C, and K denote the number of points, feature dimension, and the number of parts per shape respectively.
3.1 PART-LEVEL SE(3)-EQUIVARLAS EQUIVARIANT NETWORK
We first elaborate on our part-level SE(3) equivariant network. The network o(-) operates on a point cloud X fx, 1 < i < N} with per-point pose and outputs part-level SE(3) equivariant features for all points F = {F. { <
< N}. Here the pose of a point refer to the pose of that point's parent part. We introduce the concept of lert-level equivariant feature to differentiate from object-level equivariant features in Chen et al. (202T). where the per-point feature changes cquivariantly with the global transformation applied to the object. Part-level equivariant feature F; of each point I; changes equivariantly with the rigid transformation applied to its parent part, but remains invariant to transformations of other parts. We develop our network based on the Equivariant Point Network (EPN) Chen et al. (2021) with novel pose-aware equivariant point convolution module to support level equivariance. In the following text, we would briefly review EPN, and then continue with our pose-aware equivariant point convolution.
3

---

Published as a conference paper at ICLR 2023
Equivariant Point Network. EPN takes a point cloud X containing N points and rotation group G with [G] elements as imput and extracts C-dimensional per-point per-rotation features, forming a feature matrix F E RNxCX]GI, F is rotational and translational equivariant to a specific rigid transformation group GA induced by G. The rotational equivariant transformation for each rotation element E G in the feature domain is a corresponding matrix permutation of F along the last dimension. The translational equivariance achieved by EPN is essentially translational invariance. Simply using relative point coordinates for convolution allows F to remain the same while translating the input point cloud.
Pose-aware Equivariant Point Convolution. For part-level SE(3) equivariant features, we design a pose-aware point convolution strategy that operates on a point cloud with per-point poses. While conducting convolution within group of points, our core idea is to align point poses to the pose of the group center. Since we use the pose of point refer to the pose of its parent part, such alignment could cancel out the influence of the varying articulation states on the geometric description of each point. Intuitively speaking, if a point comes from the same part as the group center, information will just get aggregated as a normal convolution. While a point comes from a different part from the group center, pose alignment will canonicalize the articulation state so that the convolution outcome remains the same regardless of the articulation state change. Our pose-aware convolution strategy allows aggregating context information from different parts but avoids feature changing the articulation changes. Equipping EPN with this strategy. we are able to achieve part-level equivariance since the feature of each point only changes as its parent part transforms but remains invariant to the transformation of other parts. We then formally define our convolution operator. Taking point cloud X and the per-pon point pose P = [P[[1 < < N) as input, our convolution operator for the point x; feature at the rotation element y is as follows:
(F (F+h))(r,))= = Σ RRR,
az)),
zyENo,
where F( is an input function, hy (x4,9) is kernel function, Ni. is the set of points in the neighbourhood of r;, P. and P3 denote the input pose of point z, and point x, respectively. R, and Rý is their rotation components. We prove that using the above convolution within EPN leads to part-level equivariance the Appendix A.2 We highlight that we adopt an iterative pose estimation strategy (see Appendix [A.3] for details) for the per-point poses and rotations in Eq. [ which are initialized to be identity in the first iteration.
3.2 PART SHAPE, STRUCTURE, AND POSE DISENTANGLEMENT
To obtain a fine- grainer understanding of an articulated object, we disentangle three types of informa- tion from the input: 1) Canonical shape; 2) Object structure; 3) Articulated object pose. To be more specific, we first use the designed pert-level -Buua-equi network to extract per-point features from an input shape. We then leverage self-supervised slot-attention module to group the featured points, forming a set of featured parts for the disentanglement. We predict a canonical shape for cach part to induce the category- llevel canonical part spaces required by part pose definition. Then disentangle structure and pose-related information that gradually transform canonical part shapes to the observed shape. First, we predict part-assembling parameters to transform each canonical part shape to form the canonical object shape. After that, the kinematic chain, joint parameters and joint states are predicted to articulate the canonical object shape into the observed articulation state. Finally, a base part rigid transformation is predicted to further transform the resulting articulated object the observed shape in the camera space. We will elaborate details of the above designs in the following text.
Part Proposal. The part proposal module groups N points in the input shape X into K parts for per-part equivariant features extraction. It learns an invariant grouping function that maps X together with a point feature matrix F to point-part association matrix W E N Specifically, we adopt an attention- poolonngoooogggogoo operation for the per point invariant feature together with a slot attention module Locatello et al.] (2020) for the grouping purpose. Based the proposed parts, we can group points in the input shape X into K point clouds {X, < i < K} and compute the per-part equivariant feature (F.)1 < < K).
4

---

Published as a conference paper at ICLR 2023
Shape: Canonical Part Shape Reconstruction. With per-part equivariant features, aim to predict a canonical shape for each part which should be aligned within a certain category so that the category- level part pose can be defined. The canonical shape for each part should be invariant every parts' rigid transformations. Thus, we adopt an SE(3)-invariant canonical shape reconstruction module constructed based an SO(3)-PointN)) module converts per-part equivariant features F; into per-part invariant features through attention pooling first and then predicts an SE(3)-invariant shape Zi for each part.
module as utilized in Li et al. (202T). The reconstruction
Structure: Kinematic Chain Prediction. In addition to the canonical shape of each part, we also need to understand the kinematic chain of a shape. The kinematic chain defines how different parts are connected and the order they get transformed when a cascaded transformation happens... i.e. from chain leaves to the chain root. To estimate the kinematic chain for a given shape, we first construct an adjacency confidence graph from object parts and then extract its maximum spanning tree consisting of the set of confident adjacency edges. We set the part with the largest degree in the graph to be the root of the tree, which will also serve as the base part of the object. The transformation order is further predicted as the inverse DFS visiting order of the tree. Notice the kinematic chain should not be affected by the articulated input pose, we therefore leverage per-part SE(3) (invariiaai features for estimation.
Structure: Joint Parameters Prediction. For each pair of adjacent parts. , we will then infer their joint parameters, including an invariant pivot point P and joint axis orientation hypothesis u for each rotation element g E Gg. For pivot points, we treat them as invariant properties and still adopt an invariant shape reconstruction module for prediction. Specifically, we predict the pivot point P between every two adjacent parts i.j from their equivariant feature (Fi, Fj) using an invariant shape reconstruction module Li et al. (2021). For joint axis orientations, we regress an axis orientation hypothesis uf for part corresponding to each rotation group element g E Gg from its equivariant feature Fj.
Pose: Part-assembling Parameters Prediction. Part-assembling parameters transform the predicted canonical part shapes to assemble a canonical object shape. As parameters connecting invariant canonical shapes, they should be invariant to every parts' rigid transformations as well. Here, we simply predict translation vector Pf E R for each part i. We predict them through invariant shape reconstruction modules from ver-part equivariant feature {F; S i < K}. We can then assemble
predicted canonical part shapes together to form the canonical object shape: = (Z: + p Sis K).
Pose: Joint States Prediction. Joint states describe the articulation state of an object. For each part i, we predict a joint state hypothesis for each rotation element g E G from its equivariant feature F,, iee. rotation angle f for revolute part or translation scalar sy for prismatic part. We can therefore articulate the canonical object shape based on the predicted kinematic chain and joint states with the base part fixed, so as to match the object articulation from the input observation.
Pose: Base Part Rigid Transformation. The base part rigid transformation needs to transform the articulated canonical object shape to the camera space. Since we have previously predicted joint states hypotheses for all rotation element g. we will also need multiple base transformation hypotheses correspondingly. We simplify the base part transformation to be rotation, which proves to be effective in practice. A straightforward way is to use the rotation matrix corresponding to each rotation element g as the base transformation hypothesis. We follow this idea but also predict an additional residual rotation as a refinement. By transforming the articulated shape of the canonical object shape via the predicted base part rigid transformation, we can align the resulting shape with the observed input object.
Articulated Object Pose. With the above predicted quantities, we can calculate per-rotation articu- lated object pose hypotheses for an input articulated object X, including three parts: 1) translation d of each part which assembles antegor--aligned canonical parts into a canonical object; 2) per- rotation articulated transformation of the canonical object based upon the predicted kinematic chain, joint parameters and per-rotation joint states; 3) per-rotation base part rigid transformation which transforms the articulated canonical object into the camera space. The rigid transformation hypothesis
5

---

Published as a conference paper at ICLR 2023
for each part i corresponding to each rotation element g G' is denoted P." (R², .t). We treat them as part pose hypotheses.
3.3 SHAPE RECONSTRUCTION BASSE SELF-SUPERVISED TASK
Based the reconstructed canonical part shapes and predicted per-rotation part pose hypotheses, can get per-rotation shape reconstruction for each part à: {Y, = R Z; + tilg E G). A part-by-part reconstruction task is adopted self-super the network. Besides, we add a regularization term for each predicted joint so that the joint indeed connects two parts.
Self-superiiee Loss. The per-rotation shape reconstruction for the whole object can be calculated by concatenating all part reconstructions: Ys = [Y," I S K).
Shape Reconstruction-batt
We then adopt min-of-N loss between the input observation X and the reconstructed posed point clouds:
Cree min d(X, Y9), gEG
where d xxxx x3 x x3 IR denotes the distance function between two point clouds and could be unidirectional or bidirectional Chamfer Distance as example.
Regularization for Joint Prediction. Predicted joints should connect adjacent parts and support natural articulations. However, just supervising joint parameters from the reconstruction loss is not sufficient for the needs above. Therefore, predicted joint (u )) where gu = argmin, gegminnecd(XXX dX, Y*) (Eq.2) Specifically, given the predicted pivot point p and joint orientation u we independently randomly sample a set of points from the joint by shifting the pivot point p P [p.)) 10 < K* The joint regularization loss term is follows:
devise point-based joint constraint term for each
Lreg
Σ (KJ)ETT
d(P (2) + d( .. 23) d(P +(P
where Z! and Z are shapes of the part i in the canonical object space before and after its articulated transformation, Er the set of adjacent parts, d(X1, X2) the unidirectional Chamfer Distance function from point cloud X₃ to X2.
Our final self sele-super shape reconstruction loss linear combination of the above two loss terms: C = Cree + Xreeg where À is
Clonks
Wadhing
Loptop
Partil Poinx Clouds
Eycphases
X
Baput
Recon.
Canon.
Inpux
Recon
Canon.
Imput
Rezom.
Canon.
Ingun
Cunoo.
Visualization for qualitative evaluation. For every two lines. the first line draws the results of our method, and the second line draws those of NPCS. Every three shapes from the left side the right side the input point cloud (Input)), reconstruction (Recon.), and the reconstructed canonical object shape (Canon.). not assume input shape alignment but aliga them here when drawing just for better view. Please zoom in for details.
Figure 2:

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
We evaluate our method on the category-level articulated object pose estimation task (sec. 4.2 to demonstrate effectiveness. Besides, we also test its performance on two side tasks that can be completed by our network at the same time, namely part segmentation (sec. 4.3). and shape reconstruction (sec. 4.4).
4.1 DATASETS
Following previous literature Li et al. (020a). we choose seven categories from three datasets for evaluation on both complete shapes and rendered partial point clouds: 1) Four categories from the Part-Mobility Waan et al. (2019b) dataset, namely Oven, Washing Machine, Laptop (denoted as Laptop (S)), and Eyeglasses with revolute parts. 2) One category, Drawer with prismatic parts, from SAPIEN dataset Thomas al. (20IT). 3) Two categories from real dataset HOI4D Liu al. (2022b). namely Safe and Laptop (denoted as Laptop (R) with revolute parts. Please refer to the Appendix B. for data preparation details.
4.2 CATEGORY-LEVEL ARTICULATED OBJECT POSE ESTIMATION
Metries. Following Li et al. (2020a), we use the following metrics to evaluate our method: 1) Part-based pose-related metrics, namely per-part rotation error Rerr(") and per-part translation error Terr. both the form of mean and median values; 2) Joint parameters, namely joint axis orientation errors (err(") in degrees and joint position error dere, both in the form of mean values. Please refer to the Appendix B.8 for details our evaluation strategy.
Baselines. Since there is no previous works that have exactly the same setting with ours, we choose NPCS Li et al. (2020a), supervised pose estimation method for articulated objects, and ICP, traditional pose estimation approach, as our baseline methods. To apply them on our articulated objects with arbitrary global poses. we make the following modifications: 1) We change the backbone of NPCS to EPN Chen et al. (202T) (denoted as "NPCS-EPN") and add supervision on its discrete rotation mode selection process to make it work on our shapes with arbitrary global pose variations. We do observe that the NPCS without EPN will fail to get reasonable results on our data (see Appendix B.5 for details). Beyond part poses, we also add joint prediction branch for joint parameters estimation. 2) We equip ICP with ground-truth segmentation labels (denoted as "Oracle ICP") and register each part individually for part pose estimation. Notice that Oracle ICP cannot estimate joint parameters.
Experimental Results. Table T presents the experimental results of our method and baseline methods on complete point clouds. We defer the results on partial point clouds to Table 7 in the Appendix [B.]] We can make the following observations: 1) As self-supervised strategy, our average and per- category performance are comparable to that of the supervised baseline NPCS-EPN. We can even sometimes outperform NPCS-EPN such as the joint axis orientation estimation on Safe. 2) Without any human label available during training, our method can outperform the Oracle ICP with ground- truth segmentation labels by a large margin in categories. As a further discussion, the poor performance of Oracle ICP may be caused by the part-symmetry related problem. It would add ambiguity on part poses especially when we treat each part individually for estimation. Please refer to Appendix for more discussions. For qualitative evaluation and comparison, visualize the input objects, reconstructions, and the predicted canonical object shapes by our method and NPCS in Figure Our method is able to reconstruct category-level aligned canonical shapes, which serve as good support for estimating category- articulated object poses.
4.3 PART SEGMENTATION
Evaluation Metric and Baselines. The metric used for this task is Segmentation IoU (MloU). We choose three position-bain based segmentation strategies, namely BAE-Net Chen et al. (2019). NSD Kawana et al. (2020), BSP-Net Chen et al. (2020) and one motion- based segmentation method ICP TCPla as baselines for this task. For BAE-NEt and BSP-Net, generate data in their implicit
7

---

Published as a conference paper at ICLR 2023
Table 1: Comparison between the part pose estimation performance of different methods on all categories. "R" denotes rotation errors with the value format "Mean Rarr/Median Rerr". ". "T" denotes translation errors with the value format "Mcan /ecc/Me Ter:" " "I" denotes joint parameters estimation results with the value Tormat "Mean Merr/Mean derr": "Avg." refers to "Average Value". Since Oracle ICP could not predict joint parameters, we only present joint parameter prediction results of our method and supervised NPCS-EPN. For all metrics, the smaller, the better. Best values and Bold, while second best ones are shown blue.
Mirhed
Ovee
Waslog
Fyrglases
Lapep(i)
Safe
L.prep (R)
Thower
SPC3-221 (uparraod]
54T74445 2.AN7 20
64615441
27552.4 V.507.44
6.72%.10 15 15.961550
1,751159
2.772.50
V.2tar 22.
203/2.55
2,15/3255
5.3044770
3.34
Oracke ICP
46,0038
47.11/8.41
12/00/47
54,1123255
35.71/25112
04.5925..53
43 13.560200
66.04433544
531852215
$7,9055
47.5041105
42196/912
5002146.41
54.11447.15,
46.11422445
Cer
7:077 F5
4077399
70 19.3779
184199554
7/56166
(1
$1009.00
5-7044
364/286
SPC3-EPX
0A029AACH ACH
0.9059006
0:01490011
bapa xood)
0.0ONNL119
0.06000015
8/027/9/244
0.837/86923
0.01N00.00
0/12300021
Oradia N7
0.00M50.500
0.185501933
0.39800 EST.
0.07L/N657
0.120V9.030
0.072N0066
0.DKVVY.017
0.1330
0.1200 123
Oura
0L94860942
GHAZ/NLI00
ILSS8/OLIOS
2,1050 100
NPCS-EPN
504/9/776
7,42/0
5.940.129
141550063
8.53//0544
9,27/0.09
Ctrre
115
17. 17.00115 17
3EL 30.21/
12.10M
18,90
MVT 100
representation using the data generation method described in IM-NET Chen & Zhang 2019). We improve the evaluation strategy for NSD and BSP-Net considering the global pose variation of our data (see Appendix B.3 for details).
Experimental Results. E we present experimental results of our method and baselines on complete point clouds. Results partial data are deferred to Table 8 in the Appendix [B.4 Our method can consistently outperform such four part segmentation methods in all categories. BSP- Net, BAE-Net, and NSD assume input data alignment and highly rely position information for segmentation. However, such segmentation cues may not be well preserved in our data with arbitrary global pose variations. By contrast, part motions can serve as a more consistent cue for segmenting articulated objects than positions. We hypothesize that ICP's poor registration performance on some categories such Eyeglasses further lead to its low segmentation IoUs.
4.4 SHAPE RECONSTRUCTION
Evaluation Metric and Baselines, We choose to use Chamfer LI as our evaluation metric for shape reconstruction. To demonstrate the superiority of part-by-part reconstruction for articulated objects over the whole shape reconstruction, we choose EPN which treats them as rigid objects for reconstruction as the baseline.
Experimental Results. As shown Table our method can consistently outperform the EPN-based whole shape reconstruction. We suppose part-by-part reconstruction where only simple parts should be recovered makes the reconstruction an easier problem for networks than recovering the whole shape.
Table 2: Comparison between the part segmentation performance of different methods on all categories. Metric used for this task is Segmentation MIoU, calculated on 4096 points for each shape. Values presented in the table scaled by 100. Larger values indicate better performance. ag* denote cases where the network fails by segmenting input shapes into single parts.
Owen
Wadhing Machine
Fyrplasses
Epptop (5)
Safe
Laptop (R)
Drawer
BAB NedChen al. 2019 NSD
46,07
37.19
39.83*
66.35
22,83*
47.50
$6.43
$3.31
BSP-
8

---

Published as a conference paper at ICLR 2023
Table 3: Comparison hetween the shape reconstruction performance of different methods on all categorics.
Metric used in this task is Chamfer L1. The smaller, the better.
Washing Machine
Method
Oven
Fyeglasses
Laptop (5)
Safe
Tpptop (R)
Drawer
Avg.
EPN Li La al. 2021
0.033
0.051
0.028 0,025
0.029
0.030
0.028
0.057
0.036
Ours
0.025
0.049
0.024
0,026
0.026
0.045
0.031
Table 4: Ablation study w.r.t the effectiveness of joint reguralization for part pose estimation and the design of pose-aware equivariant feature communication (denoted as "Pose."). Reported values are per-category per-part average values. Please refer to the caption of Table for the data format of "Joint".
Method
Seg. Tol
Meun Hurr(")
Mediam Rerr!")
Moan Fure
Median Ver.
Join
Chamier L.
No Zuir
76.40
11.34
16.87
0,020
0.365
0L038
With
74.32
10.40
0.072
0.073
22.01/0.111 19.72/0.003
0.032
Loeg (Pose.)
0.052
ABLATION STUDY
In this section, we try to ablate some crucial designs the method to demonstrate their effectiveness, including part-level feature accumulation, pose-aware point convolution, and joint regularization.
Part-level Feature Accumulation. We grouping module group points into parts for part- level features in our method. To demonstrate the effectiveness of using part-level features for part shape, structure, pose disentanglement, we ablate part-level features and only features from the whole shape for part-level properties prediction, similar to those used in Kawana et al. (202T); )); Chen al. (2019). Table Sablle5commee compares their performance. For each metric, we report its per-category per-part average value. It can be observed that part-level features can help with part-based properties prediction, letting the network achieve better performance on pose-related metries.
Pose-aware Point Convolution. Our method contains a pose-aware equivariant feature convolution design for part-level SE(3) equivariant feature learning. To demonstrate the superiority of part-level equivariance over common global equivariance. we compare the model's performance when using part-level equivariant features (With Creg (Pose.)) with the one using global equivariant features (With Creg) in table For each metric, its per-category per-part average value is reported. The network using part-level equivariant features could consistently outperform the using only global equivariant features on all metrics.
Joint Regularization. Besides reconstruction loss, we additionally add a joint regularization term to predict joints that connect two adjacent parts. Beyond acquiring joint-related parameters, joint regularization could improve the pose estimation performance as well, especially for translation prediction, as shown in Table
6 CONCLUSION
In this work, we propose a self-supervised strategy for category-level articulated object pose esti- mation without any annotations. Leveraging part-level SE(3) equivariant features, we propose part shape, structure, pose disentanglement strategy that successfully accomplish the category-level articulated object pose estimation task. A part-by-part shape reconstruction task adopted to self- supervise the network learning. Experiments prove the effectiveness of our method and our core ideas. This work can reduce the annotation efforts for solving this tasks and would also promote further thinkings designing part-level equivariant networks.
Table 5: Ablation study w.r.t. the effectiveness of accumulating part-level features for part-based properties prediction. Reported values are per-category per- part average values on all categories. "Joint" represents joint parameter estimation errors, with the value in the format of "Mean 0 Mean d
Method
Seg. loU
Mean ("..(")
Median (_..
Mean T... Median Tu..
Joint
Chamfer L1
Without Parts
7158
12.85
11:22
0.068
0.00
27.900.77 172
0.036
19.720.103

---

Published as a conference paper at ICLR 2023
REFERENCES
Haiwei Chen, Shichen Liu, Weikai Chen, Hao Li, and Randall Hill. Equivariant point network for 3d point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14514-14555 14523, 2021.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939-5948, 2019.
Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. Bac-net: branched autoencoder for shape co-segmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 8490-8499, 10--8499, 2019.
Zhiqin Chen, Andrea Tagliasacchi, and Hao Zhang. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 45-54, 2020.
Cheng Chi and Shuran Song. Garmentnets: Category-level pose estimation for garments via canonical space shape completion. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 3324-3333, 2021.
Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, and Andrea Tagliasacchi. Cvxnet: Learnable convex decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 31 31-44, 2020.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-transformers: 3d roto- translation equivariant attention networks. Advances in Neural Information Processing Systems, 13:1970-1981, 2020.
Mohit Goyal, Sahil Modi, Rishabh Goyal, and Saurabh Gupta. Human hands probes for interactive object understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3293-3303, 2022.
ICP. Iterative closest point. In
Zhenyu Jiang, Cheng- Ceeng-Chun Hsu, and Yuke Zhu. Ditto: Building digital twins of articulated objects from interaction. In Proceedings of the IEFE/CVF Conference on Computer Vision and Puttern Recognition, pp. 5616-5626 2022.
Dov Katz and Oliver Brock. Manipulating articulated objects with interactive perception. In 2008 IEEE International Conference on Robotics and Automation, pp. 272-277. IEEE, 2008.
Yuki Kawana, Yusuke Mukuta, and Tatsuya Harada. Neural star domain primitive representation. Advances in Neural Information Processing Systems, 33:7875-7886, 2020.
Yuki Kawana, Yusuke Mukuta, and Tatsuya Harada. Unsupervised pose-aware part decomposition for 3d articulated objects. CoRR, abs/21 10.04411, 2021. URL httpp::/aaxiv..rrg///////////
2110. 04411
Yann Labbé, Justin Carpentier, Mathieu Aubry, and Josef Sivie. Single-view robot pose and joint angle estimation via render & compare. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1654-1663, 2021.
Xiaolong Li, He Wang. Li Yi, Leonidas Guibas, A Lynn Abbott, and Shuran Song. Category- articulated object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3706-3715, 2020a.
Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbotr, Shuran Song, and He Wang. Leveraging se(3) equivariance for self-supervised category-level object pose estimation, 2021.
Yong-Lu Li, Xinpeng Liu, Han Lu, Shiyi Wang, Junqi Liu, Jiefeng Li, and Cewu Lu. Detailed 2d-3d joint representation for human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10166-10175 2020b.
10

---

Published as a conference paper at ICLR 2023
Liu Liu, Han Xue, Wenqiang Xu, Haoyuan Fu, and Cewu Lu. Toward real-world category-level articulation pose estimation. IEEE Transactions on Image Processing, 31:1072-1088,, 2022a. doi: 10.1109/1122022113133888
Yunze Liu, Yun Liu, Che Jiang, Zhoujie Fu, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. 2022b.
arXiv preprint arXiv:2203.014
Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention, 2020.
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhi- wei Jia, and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107.14483, 2021.
Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. Neural parts: Learning expressive 3d shape abstractions with invertible neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 3204-3215, 2021.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space. Advances in neural information processing systems, 30, 2017.
Martin Sundermeyer, Zoltan- -sala Marton, Maximilian Durner, Manuel Brucker, and Rudolph Tricbel. Implicit 3d orientation learning for 6d object detection from rgb images. In Proceedings of the european conference on computer vision (ECCV), pp. 699-715, 2018.
Hugues Thomas, Charles R Qi. Gean-emma Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6411-6420, 2019.
Martyn Thomas, Gerhard Schymik, Thomas Walther, Dominique Himbert, Thierry Lefévre, Hendrik Treede, Holger Eggebrecht, Paolo Rubino, Antonio Colombo, Rüdiger Lange, al. One-year outcomes of cohort in the edwards sapien aortic bioprosthesis european outcome (source) registry: the european registry of transcatheter aortic valve implantation using the edwards sapien valve. Circulation, 424(4):425 433, 2011.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley, Tensor field networks: cnd translation-cquivariann neural networks for 3d point clouds. arXiv preprint arXiv: 08219, 2018
Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, and Stan Birchfield. Deep object pose estimation for semantic robotic grasping of household objects. arXiv preprint arXiv:1809. 2018
Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition, pp. 2635-2643, 2017.
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pp. 2642-2651, 2019a.
Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao and Kai Xu. Shape2motion: Joint analysis of motion parts and attributes from 3d shapes. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8876-8884, 2019b.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3d steerable cnns: Learning rotationally equivariant features in volumetric data. Advances Neural Information Processing Systems, 31, 2018.
11

---

Published as a conference paper at ICLR 2023
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: simulated part-based interactive environment. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11097-11107, 2020.
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes, arXiv preprint arXiv:1711.00199, 2017.
Kaizhi Yang and Xuejin Chen. Unsupervised learning for cuboid shape abstraction via joint segmen- tation from point clouds. ACM Transactions on Gruphics (TOG), 40(4): 2021.
Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, and Federico Tombari. Quaternion equivariant capsule networks for 3d point clouds. In European Conference on Computer Vision, pp. 1-19. Springer, 2020.
Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas J Guibas, and Hao Zhang. Adacoseg: Adaptive shape so-segmentation with group consistency loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 8543 8552, 2020.
12

---

Published as a conference paper at ICLR 2023
The appendix is organized follows:
Proofs and further explanations on the method
Proof of part-Level equivariant property of the pose-aware point convolution module (sec. A.2 Further explanations on some method components (sec. A.3).
Experiments
Data preparation (sec. B.1
Implementation details (sec. B.2
Implementation details for baselines (sec. B.3
Experiments on partial point clouds (sec. B.4
Additional comparisons and applications (sec.
B.5
Robustness to input data noise (sec. [.66].
Visualization of part-level equivariant features (sec. B.7
Evaluation strategy for category-level articulated object poses (sec. B.8
Discussion part symmetry (sec.
A
METHOD DETAILS
Equivariant Feature Extraction
Part Proposal
Shape Reconstruction
Attattonn Paoling (p]
Perport Insarie Fexuro Fu.. (4..00)
Skit Alzzion
Post-pott Acaetitton (M)
Per-tt Recesttacion yo (tf)
Porgginnt Fianeo (0(e x)
Crorping Tor Porpet Festres
Ponikool Eqshasiari Network
/Cinnncal Space
Canoriaa Objact Space all
M
(2) 7,
Tregon Eiriorret Fezure {ri (fOkg)))
Tanslimain Conpenttnn
Esiaaaed Fhem Posce (gg
Kinetati Cleis
Pu-..
Juint Stotenadd Per-nx Bas Parl Rigal
fetie Parametors bx
Part Saructure, Poss Disentangkement
(f
Overview of the proposed supe-super articulated object pose estimation strategy. The method takes a complete partial point cloud of an articulated object as input, factorizes canonical shapes, object structure, and the articulated object pose from it. The network is trained by a shape reconstruction task. Part-level SE(3) equivariant features are learned by iterating between part pose estimation and pose-aware equivariant point convolution. Green lines (+-) denote procedures for feeding the estimated part poses back to the pose-aware point convolution module.
Figure 3:
A.1 OVERVIEW
We provide an detailed diagram of our self-supervised learning strategy Figure
A.2 PROOF OF PART-LEVEL EQUIVARIANT PROPERTY OF THE POSE- FOSE-AWARE POINT CONVOLUTIOON MODULE
In this section, we prove the part-level equivariant property of the designed pose-aware point convo- lution module:
(F+hi)(Eng) =
Σ
R(, (9(00 P.P.ta,)).
(3)
Pf P- P-
where P. and P, are the (estimated) pose of x; and zj from the canonical object space the camera space respectively, R. and R, are the (estimated) rotations of point and point 2, from the canonical object space to the camera space respectively, V denotes the set of point x; neighbours in
the canonical object space. Note that the neighbourhood set N2, in the Equation represents the neighbourhood of x; in the camera space with points in which belong to x, 's neighourhood in the
canonical object space, i.e. N Np.1ze p-11z"" .N., would vary x; 's pose changes, while M Ni. p-11; keeps the
same. {5ylæ E AV,, = (0 2, E N Np-
13

---

Published as a conference paper at ICLR 2023
To prove the part-level equivariance of (F+h1) (x;+)))x g). need to prove 1) (F*hi))(z 9) is invariant to the rigid transformation of point xy 's each neighbouring point zy; 2) (F * h1 (z g) is equivariant to the rigid transformation of 2; itself.
We then prove those properties for the continuous convolution operations, (F * hz)(z.:g) = SMyERS F(zj, R,, " "))[(((;=
Theorem 1. The continuous operation (F (1) = S2,Ex0 F(zg.gR.Ft ( (g(2; p,)) is invariant to each arbitrary rigid transformation AP} = (AR, E SO13), At, E ERR3) (x, E ER,,, 2y * 24) of x neighbouring point 2z.
Proof. To prove the invariance of (F*h1)(Iss 9), we need to prove that Va, E R² 2y F 24, VAP, e SE(3,, R2 - RR we have
P * (1) = =F
Let n/ A.jaj PJ AP,3P,P, then we have,
[ 2
A AP)(F
Flc "))loo p.Pp-1x())
F(AP, RR, AR, (g1 - P.P.5 AP; AP,z,))
JzyER9
F(AR, jJ Jz RR.R, ARRR'-RR111 /h1 (g(2;; P.Pp1a.) x;))
JazERS
FlI Fl ))((((1 P.P x,))
JZJERD
= * (Z::9)
Theorem 2. The continuous operation (F x h1))(11)))) = J2, ((zg.RR, (hrR))))a(oo (g(1s 2.)) is equivariant to the rigid transformation AP; = (AR, E SO13), At, E R of xz.
Proof. To prove that (F=hi)(z, g) is equivariant the rigid transformation of of need to prove that VAP, E SE(3), we have
AP + (1) (AR.F (1;,
It can be proved by AP,(F* (F (Fth)((P,,,)A))
= -f
JZJERS Fl,,,ARR -OOAAP- AP.P,
J2JERS Fle,, (9AR, (M -P5'2,)
(AR,F (11.g.)).
A.3 FURTHER EXPLANATIONS ON SOME METHOD COMPONENTS
Kinematic Chain Prediction. The kinematic chain is predict as an invariant property from per- part invariant features to describe part articulation transformation order. It is predicted through the following four steps: 1) Predict an adjacency confidence value C., for each part pair (i, 2) Construct an fully-coo culle-connected adjacency confidence graph (V, E) based values, with all parts as its nodes and predicted confidence values as edge weights: 3) Find a maximum spanning tree from the constructed graph G: T ET); 4) Calculate the DFS visiting order of 'T
predicted confidence
14

---

Published as a conference paper at ICLR 2023
C1
C1
C2
C3
DFS Order
Kinematic Chain
Figure 4: Kinematic chain prediction procedure (an example of the object containing three parts).
and take the inverse visiting order as the predicted kinematic chain. We draw the prediction procedure in Figure
Invariant/Equivarian Features for Prediction. Given per-part equivariant feature output from the feature backbone P. its equivariant feature for equivariant properties prediction is further calculated from an Poinne,o i.e. F. = (en3))PPinNNe((, F.). Its invariant feature for invariant properties prediction is then computed through a max-pooling operation: Pinv Max-PoolinggF;))
Joint Axis Orientation. We assume that all joints' axis orientations in shape are consistent. Thus, in practice, we set the orientation of all joints to the same predicted orientation, i.e. u² <- u V(i.j) E,, Vgg E Gg. where (fmm Jm.) is set to the part pair connected to the tree root. Saying (i,j) Ey, we mean directional edge from part i to part In the node pair (i,j) Ey, node i is deeper than node j in the tree T. It indicates that node i's subtree should rotate around the joint u passing through the joint between i and j.
Iterative Pose Estimation. Our pose-aware equivariant point convolution module requires per-point pose as input. Due to our self-supervised setting where input poses are not assumed, we adopt an iterative pose estimation strategy. Through this design, we can improve the quality of part-level equivariant features gradually by feeding back estimated poses in the last iteration to the pose-aware point convolution module in the current iteration.
because that more accurate input per- per-point poses would lead to better "part-level" SE(3) equivariant features considering the nature of our pose- aware point convolution. In practice, we set per-point poses to identity values in the first iteration due to the lack of estimated poses, i.e. Po (Ro; to).
Kinematic Chain Joint Parameters
(pf )
Joint States Base Part Rigid Transformation
Canonical Part Space {7, e B MFxa]
Canonical Object Space {Z] Z, pf. P(.)
Camera S Space {Y*)
Figure 5:
The relationship between our three crucial spaces: the canonical part spaces, the canonical object space, and the camera space.
Canonical Part Spaces, Canonical Object Space, Camera Space. For each part, the canonical part space normalizes its pose. For each object, the canonical object space normalizes its object orientation, articulation states. Camera space denotes the observation space. Each part shape in the canonical part space its canonical part shape. Each object shape with articulation states canonicalized is called its canonical object shape. Canonical spaces category-level concepts, while canonical shapes are instance level concepts. Figure 3 draws the relationship between such three spaces mentioned frequently our method.
Partial Point Clouds. The loss function used in Li et al. (2021) for partial point clouds is the unidirectional Chamfer Distance. Using this function can make the network aware of complete shapes by observing partial point clouds from different viewpoints. Such expectation can be achieved for asymmetric objects if the viewpoint could cover the full SO(3) space. However, restrict the range of the viewpoint when rendering partial point clouds for articulated objects to make each part visible. Such restriction would result in relatively homogeneous occlusion patterns. Therefore, choose to use unidirectional Chamfer Distance only for certain categories such as Safe when tested on partial point clouds.
15

---

Published as a conference paper at ICLR 2023
Table 6: Per-category data splitting.
Washing Machine
Oven
Byeglasses
Laptop (S)
Safe
Laptop (R)
Drawer
#Total
32
41
42
82
30
50
30
#Train
28
36
37
73
26
44
24
#Test
4
9
6
Equivariant/Ivvariant Properties of the Designed Modules. The designed method wishes to use part-level -eqiivaaiie reduce the difficulty of factorizing part pose and part shape. Exact part-level equivariant features can make those modules meet our expectations. However, due the approximate squiaaaquiv of the employed feature backbone and the estimated part pose that may not be very accurate, we cannot expect such invariance/equivariavni instance, if we not consider part kinematic constraints, the part shape reconstruction module and the part-assembling parameters prediction module should be invariant to K rigid transformations in if using global equivariant features, while it should be
for them. For
the quotient group (SE())/A))(EE()) invariant to the rigid transformation in the quotient group SE(3)/G, if using part-level equivariant features given correct part pose estimation. Similarly, the pivot point prediction module should be invariant equivariant features. Part-level equivariance design could reduce the difficulty of a network doing factorization, which may count reason for its effectiveness.
rigid transformations in the quotient group (SE(3)/G A) if using part-level
B EXPERIMENTS
B. DATA PREPARATION
Data Collection. We choose seven categories from three different datasets, namely Oven, Washing Machine, Eyeglasses, Laptop (S) with revolute parts from Shape2Motion Wang et al. (2019b), Drawer with prismatic parts from SAPIEN Xiang et (2020), Safe and Laptop (R) with revolute parts from HOI4D Liu et al. 2022b)
The first five datasets are selected according to previous works on articulated object pose estimation or part decomposition| Li et al. (2020a); Kawana et al. (202I). To further test the effectiveness of our method on objects collected from the real world, we choose two more categories (Safe and Laptop (R) from real dataset [iu et al. 2022b).
Data Splitting. We split our data according to the per-category data split approach introduced in Li ct al. (2020a). Note that not all shapes a category are used for training/testing, Incomplete shapes instances whose canonical articulation states inconsistent with other shapes are excluded from experiments. Per-category train/test splits are listed Table
Data Preprocessing. For each shape, we generate 100 posed shapes in different articulation states.
Then for complete point clouds, we randomly generate 10 rotated samples from each articulated posed object. When generating articulated posed objects, we would add restrictions on valid articulation state ranges. For Oven, Safe, and Washing Machine, the valid degree range of their lids is [45", 135%). For Eyeglasses, the range of the degree between two legs and the frame is set to [0", 819). For Laptop (S) and Laptop (R), the range the degree between two parts is set [9 99%).
For partial point clouds, we render depth images of complete object instances using the same rendering method described in [i et al. (2021). The difference that we manually set a viewpoint range for each category to ensure that all parts are visible in the rendered depth images. For each articulated posed shape, we render 10 depth images for it. The dataset will be made public.
Data samples visualization. In Figure 6 we provide samples training and test shapes for some categories for an intuitive understanding w.r.t. intra-category shape variations. Such variations mainly come from part geomtry (e.g. Eyeglasses frames, Oven bodies, Laptop) and part size (e.g. Washing Machine, Laptop).
16

---

Published as a conference paper at ICLR 2023
Trun Troes
Lowg-R
Figure Samples of training and test shapes.
B.2 IMPLEMENTATION DETAILS
Architecture. For point convolution, we use kernel-rotated version kernel point convolution (KPConv) TTPConyTThomas et al. ) proposed in Chen et all (2021). The size of the (one) convolution kernel is determined by the number of anchor points and the feature dimension. In our implementation, we use 24 anchor points. Feature dimensions different convolution blocks are 64, 128, and 512 respectively.
Training Protocol. In the training stage, the learning rate is set to 0.0001, which is decayed by 0.7 every 1000 iterations. The model is trained for 10000 steps with batch size on all datasets. We use the self-supervised reconstruction loss to train the network, with the weight for joint regularization À set to 1.0 empirically. We use Adam optimizer with 3 = (0.9, 0.999), 10"
Software and Hardware Configurations. All models are implemented by PyTorch version 1.9.1, torch_cluster cluster version 1,5.1, and Python 3.8.8. All the experiments are conducted on a Ubuntu 20.04.3 server with 8 NVIDIA GPUs, 504G RAM, CUDA version 11.4.
version 2.0.7, pyrender version 0.1.45, trimesh version 3.2.0,
B.3 IMPLEMENTATION DETAILS FOR BASELINES
NPCS Li et al. 2020a): The NPCS's original version Li ct al. (2020a) trains a network for category- level articulated object pose estimation supervised manner. It utilizes a PointNet++ Qi al.) (2017) to regress three kinds of information and set of pre-defined normalized part coordinate spaces. Then in the evaluation process, RANSAC algorithm is leveraged to calculate the rigid transformation of each part from its predicted normalized part coordinates to the shape in the camera space. To apply NPCS in our experiments, we make the following two modifications: 1) We change the backbone used in NPCS from PointNet++ to EPN. We further add supervision mode selection process for the major rotation matrix prediction as does in Chen et al, (2021). 2) We add joint axis orientation prediction branch and pivot point regression branch for joint parameters estimation. Such two prediction branches act on the global shape feature corresponding to the selected rotation mode and predict residual rotation and translation for estimation. By applying the major rotation matrix of the selected mode, we could then arrive at the joint axis orientations and pivot points in the camera space.
its rotation
Oracle ICP. To apply ICP on the articulated object pose estimation, we introduce Oracle ICP. Oracle ICP registers each ground-truth part from the template shape to the observed shape iteratively. We randomly select segmented shapes from the train set to register on each test shape. For complete point clouds, we first centralize the part shape from both of the template shape and the observed shape, we then iteratively register the template part shape to the observed part shape under 60 initial hypotheses. For partial point clouds, iteratively register the template part shape to the observed part shape under 60 initial hypotheses together with initial translation hypotheses. The one that achieves the smallest inlier RMSE value is selected as the registration result. After each registration,
17

---

Published as a conference paper at ICLR 2023
we assign per-point segmentation label the label of the nearest part. Therefore, we also treat Oracle ICP as one of our segmentation baseline.
BSP-Net Chen et al. 2020). BSP-Net reconstructs an input shape using implicit fields as the representation by learning to partition the shape using three levels of representations, from planes to convexes, and further to the concave. Indices of reconstructed covexes are consistent across different shapes in the same calegory. Thus, we can map from each convex index ground- truth segmentation label. The relationship can then help us get segmentations for test shapes. The mapping can then help us segment each test shape by assigning each convex to its corresponding part segment. The intra-category convex partition consistency further provide cross-instance aligned part segmentations. However, due to the global pose variations of our data, such convex index consistency may not be observed when directly applying BSP-Net on data. Thus, we propose to improve the evaluation process of BSP-Net to mitigate this problem. Specifically, for each test shape, we find a shape from the train set that is the most similar to the current test shape. Then, we directly use its convex- menvx--segmentttinn mapping relationship to get segments for the test shape. The segments are then used to calculate the segmentation IoU for the test shape.
NSD Kawana et al. (2020). Neural Star Domain Kawana al. (020) decomposes shapes into parameterized primitives. To test its part segmentation performance on our data with arbitrary global pose variations, we adopt the evaluation strategy similar that used for BSP-Net.
B.4 EXPERIMENTS ON PARTIAL POINT CLOUDS
In this section, we present the experimental results on rendered partial point clouds of our method and baseline methods.
Articulated Object Pose Estimation. In Table 7 we present the part pose estimation and joint parameter prediction results of our method and baseline methods. Compared to the pose estimation results of different models achieved on complete point clouds, our model can sometimes outperform the supervised NPCS baseline (using EPN as the backbone), such as better part pose estimations and joint parameter prediction results on the Laptop (S) dataset.
In Figure 8 draw some samples for a qualitative evaluation. Moreover, we also provide the visualization of all categories for complete point clouds in Figure 7 In the figure, the point cloud distance function used for Safe is unidirectional Chamfer Distance, while that used for others is still bidirectional Chamfer Distance. Using unidirectional Chamfer Distance can relieve the problem of joint regularization on partial point clouds to some extent. It is because that this way the point cloud completion could be naturally enforced. For instance, reconstruction results for the Safe category are drawn in Figure X However, points that are not mapped to any point in the input shape will also affect the point- based joint regularization. For simple shapes, using bidirectional Chamfer Distance could also sometimes make the decoder decode complete part shapes, e.g. reconstructions for Laptop (R). As for the reconstructed reference shape in the canonical object space, better joint predictions would lead to better global shape alignment. For instance, we can observe that the angle between two parts of Laptop (R) and Laptop (S) is relatively consistent across shapes with different articulation states. Joint regularization enforcing the connectivity between two adjacent parts both before and after the articulated transformation in the canonical object space. It could then help make the joint behave like a real joint, based on which we the "lazy" network tends to decode part shapes with consistent orientations. However, there is degenerated solution where the decoded rotation angle is put near to zero. In that case, the joint regularization term could be satisfied by decoding "twisted" part shapes. Since the decoded angle is near zero, the connectivity between two parts will not be broken when rotating along the decoded joint. Sometimes, decoding angles near to zeros is local minimum that the optimization process gets stuck in. At that time, the regularization loss term is a large one but the decoded joint parameters are not optimized in the correct direction by the network. The reconstructed shapes in the canonical object space do not have consistent angles, e.g. Washing Machine and Safe drawn in Figure g
Part Segmentation. In Table g we evaluate the segmentation performance of our method. BSP-Net is not compared here since it requires mesh data for pre- pre-processing, which is not compatible with the rendered partial point clouds. Oracle ICP uses real segmentation labels to register each part from
18

---

Published as a conference paper at ICLR 2023
Oven
Madhine
Washing
Expphooes
Laptop (S)
Safe
Laptop (R)
Denser
Figure Visualization for experimental results complete point clouds. Shapes drawn for every three shapes from the left side to the right side are the input point cloud, reconstructions, and the predicted canonical object shape. We put drawers in an aligned space just for better visualization. Their global pose may vary when feeding into the network. Please zoom in for details.
the example shape to the observed shape. Despite this, it can still achieve satisfactory estimation results due to shape occlusions and part-symmetry-relan pose ambiguity issues.
we evaluate the shape reconstruction performance of our method. The part-by-part reconstruction strategy used by our method can outperform the EPN-based whole shape reconstruction strategy in most of those categories except for Washing Machine. One possible reason is the poor segmentation performance of our model on shapes the Washing Machine category.
Shape Reconstruction. In Table 9
B.5 ADDITIONAL COMPARISONS AND APPLICATIONS
Comparison with Other Baselines. We compare our method with other two baselines that not discussed in the main body in Table TO
Firstly, we use KPConv Thomas al. (2019) as NPCS's feature backbone (denoted "NPCS- KPConv") and test its performance on our data with arbitrary global pose variation. We can see that NPCS of this version performs terribly compared our unsupervised method. NPCS estimates part poses by estimating the transformation from estimated NPCS coordinates and the observed shape. therefore requires invariant NPCS predictions estimate category-level part poses. However, such prediction consistency may not be casily achieved for input shapes with various global pose variations.
The second one is Oracle EPN, where we assume ground-truth part segmentation labels and use EPN to estimate the pose for each individual part. Despite in such oracle setting, EPN cannot infer joint parameters since it estimates per-part poses individually. Besides, the part symmetry problem will also hinder such strategy from getting good performance to some extent, which will discussed in the next section C
19

---

Published as a conference paper at ICLR 2023
Machine
Washing
Byeplaces
Laptop (8)
Sale
Laptog (R) Timoue
Figure 8:
Visualization experimental results on partial point clouds. Shapes drawn for every three shapes from the left side to the right side are the input point cloud, reconstructions, and the predicted canonical object shape. Please zoom in for details.
Figure 9: Reconstruction for shapes in different articulation states and manipulations to change their states. Shapes (in blue and orange) drawn on the two sides are manipulated shapes from their nearest reconstructions. Others reconstructions (in purple and green). Please zoom in for details.
Shape Reconstruction and Manipulation. The predicted joints can enable us to manipulate the reconstruction by changing the value of predieted rotation angles. We then arrive shapes in new sarticulation states different from input shapes. In Figure 9 we draw some examples for Laptop (S) and Oven.
B.6 ROBUSTNESS TO INPUT DATA NOISE
Figure 10: Visualization for the model performance on input data with random noise, Shapes for each three drawn from left to the right are input data corrupted by random normal noise, segmentation, and reconstruction, respectively. We align shapes here just for better visualization, while they may be into arbitrary poses for input.
20

---

Published as a conference paper at ICLR 2023
Table 7: Comparison hetween the part pose estimation performance of different methods all test categories (partial point clouds). "R" denotes rotation errors with the value format "Mean Rerr/Median Rer "T" denotes translation errors with the value format "Mean Terr/Mcdian Terr" " "J" denotes joint parameters estimation results with the value format "Mean Mern/ derr". " ICP could not predict joint parameters. Therefore, only the results of supervised NPCS and method joint prediction presented. For all metries, the smaller, the better. Bold numbers for best values, while blue values represent second best ones.
Method
Oven
Washing Machine
Eyeglasses
Laptop (S)
Safe
Laptop (R)
Drawer
Avg.
7,2666.08
21.74/10.80
(supervised)
NPCS-EPN
2.51/2.27.
2.93/2.64
4.7113864
8,56/7.46
23.39/17.33,
20,86/18.76
29.90132.71
26,4022..56,
6.6415.76,
5,43/5.19
9,3918 75,
6.75/6.14
22.92/10.18.
25.10/14.16.
7.34/6.83
13.34/10.73
R
73,24/78.73 73.
82,93/82. 64,
Orucle
80,53/10.80,
32,42/17.88
68.74/74.09.
67.48/73.01,
38.72/34,44,
30.78/28.66 674,
61.31/59.51,
48.55/47.29
20.68/20.50
19.39/16.99
(9.23/74.53
63.22/68.23
52,28/42.16
42,06/39.25
54.39/52, 82,
26,88/29.66 66
2.50/1.79,
11.7717877
161/1.52
4.69/3.77.
10.18/5.30.
15.38/ 15.38/44.4
8.5016.85
2.60/1.29 79.
Ours
10.839915
12.81/12.51
9.56/5.36,
11.10/5.22
21.91/19.04
6.92/5.66
2,06/1.79,
8.36/6.47
7.53/6.12
2,00/1.79
0.085/0.075,
0.441/0.365 365,
(supervised)
NPCS-EPN
0.028/00030.
0.028/0.023
0.034/0.030.
0.033/0.028
0.056/0.052.
0.057/0.044
0.26310.533,
0.286/0.236
0.022/0.021.
0.034/0.034
0.048/0.043.
0.047/0.044
0,367/0. 343,
0.549/0.299,
0.081/0.065
0.145/0.117
197.
0.692/0 097,
0.322/0.311, 0.324K0.31,,
0,265/0.278,
0.28
0,28000.289.
0 170,
Orccle ICP
0. 08188 197.
0,218/0.222
0.16910.17
0.136/0.144
0.185/0.193
0.2671U.27
0.246/0.248
0.305/0.306
0.1.5/0..164
0.129/0.132
0:044/0.046,
0.07110.065,
0.17910. 164,
0.219/0,226.
0.04440.034,
0.03300 030,
0.08810.082
0.04770.050,
Ours
0,204/9.220
0.253 /0.254
0. 1169900060 765.
0.031/0.025
10000 104
0.07099066
122/0. 131,
0.119001110
0.177/0.171
0.172/0112
NPCS-EPN
20.11/0.221.
(supervised)
28.62N0.022
8.05/0.194
20.11/0.239
10.91/0.155
11.23/0.084
12.25/0.134
11,21/-
15.31/0.160
26.967700087
Ours
5,24/0.105
22.30/0.212
26.96/0.260
10.83/0.142
16.16/0.170
18.02/0.179
7.43/-
21.61/0.144
Table 8: Comparison between the part segmentation performance of different methods (partial point clouds). The metric used for this task is Segmentation MIoU, calculated on 4096 points for each shape. Values presented in the table are scaled by 100. Larger values indicate better performance.
Washing Machine
Oven
Eyeglasses
Laptop (S)
Safe
Laptop (R)
Drawer
Oracle ICP Ours
75.83
73.07
68.92
54.01
66.90
58.38
87.07
51.73
56.80
84.94
44.64
86.04
45.45
Besides testing the performance of the proposed method on partial point clouds with occlusion patterns caused by viewpoint changes, we also test its effectiveness on noisy data. Specifically, we add noise for each point in the shape by sampling offsets for its x/y/z coordinates from normal distributions, e.g. Ax ~ [O, o²), where we set a 0.02 here. Results on Oven and Laptop (S) are presented in Table [1 From the table, we can see the degenerated segmentation IoU on Oven's noisy data, while still relatively good part pose estimation performance. Another discovery the even better joint axis orientation prediction, but larger offset prediction perhaps due to the poor segmentation. Besides, the shape reconstruction quality also drops a lot, probably due to the randomly shifted point coordinates. We observe similar phenomenon on Laptop (S). In Figure 10] we draw some examples for a qualitative understanding w.r.t. model's performance on noise data.
B.7 VISUALIZATION OF PART-LEVEL EQUIVARIANT FEATURES
Aiming for an intuitive understanding w.r.t. the property output by the designed part-level equivariant network, we draw features output by the global equivariant network and part-level equivariant network for some laptop samples in Figure IT From the figure, we can see that the point features of the non-motion part (base) do not change lot when the moving part (display) rotates angle. That echoes the wish for the pert-level equivariance design to disentangle other parts' rigid transformation from the current part's feature learning.
21

---

Published as a conference paper at ICLR 2023
Table 9: Comparison between the shape reconstruction performance of different methods (partial point clouds). The metric used this task is unidirectional Chamfer LI from the original input shape to the reconstructed shape. The smaller, the better.
Washing Machine
Method
Oven
Eyeglasses
Laptop (S)
Safe
Laptop (R)
Drawer
EPN al.
2021
0.040
0.043
0.044
0.032
0.020
0.026
0.079
Ours
0.035
0.062
0.041
0.025
0.019
0.024
0.061
Table 10: Comparison between the part pose estimation performance of different methods. Backhone used for NPCS is KPConv. "R" denotes rotation with the value format "Mean Ree//n Rarr", "T" denotes translation errors with value format "Mean /Median T Tor.". " "J" denotes joint parameters estimation results with the value format "Mean /ean derr". For all metrics, the smaller, the better. Bold numbers for best values.
Washing Machine
Method
Oven
Eyoglasses
Laptop (S)
Safe
Laptop (R)
Drawer
Avg.
62.73/69 42,
SPCS-KPConv
44 1643693
56 36.20V56.22
$1,99/53.97
55.67/66.44,
11,6//1 1.8/11.10,
49. 49.98/68 43.
56.16/60.33 34.
42,48/38 08.
50.74/53.99
(supervised)
60.58/63.35
50.16/51
55.63/61.33
43.48/42.22 22
73.40/83.55 55
57.23/63
42.29/38.
48.76/46.82
R
54.01/13.09.
47,88/47 II..
Oracle EPN
7.0716.888,
16.339.17
33.56/20.49
7.97/7.60 60,
116.56/119
86. 12/66/ 07.
18.98/12.75
18.33/9.73,
45.85/48. 59,
18.03/27.67
20.46/14.03
21.08/19.00
35.79/37.11 11.
30.84/25 23,
37,81/30.73
43.83/39 A6
3,18/
Ours
3.74/7.35,
4.07/3.97
19.27/19.19
7,49/7.37,
12.29/10.89,
12.53/9.88 888
8.16/8.21.
10.41/9.34
7.34/5.16,
13.83/13.59
9.03/9.09.
5.71/3.61.
3.64/2.84
3.18/2.73,
3.18/2.71.
3,18/2.71
7.90/7.14
0,401/0 326
0.791/0.742.
NPCS-KPComy 0. 133/0.1 121
(supervised)
0.10400.091
0. 146/0, 142,
0,418/0 257
0.396/0.266
0,233/0.203
0.217/0.169
0,055X0052
0.098/0.091
0. 0.1790.222,
0.161/0.174
0,694/0.600 6404,
L.005/0.942
0.316/0.279
0.271/0.240
0.204/0
Oracle EPN
0.031.0.030.
0.05800052
0.046/0.044,
0.05902003
0.128/0.118.
0.199/0
0.334/0.29 292
0.132/0.122
0.117/0.090
0.092/0.086, 1.157
0.158/1.151
0.09490.82
0.161/0. 146.
0.177/0. 166.
0.143/0. 129
0,29000.20
0.09%
0.0154/0.039
0,082/0.083, 0.054/0.052
0.040/0.037.
0.021/0.019. 0.06641.069.
0.097/0.092
Ours
0.067/0.046
0.042/0.0334
0.086/0.088.
0.07000.055
0.036/000422
0.037/0.035
0.027/0.026
0.108/0. 105
0.065/0000
0.109/0.100
NPCS- NPCS-KP KPConv
60,3800 329.
(supervised)
55.62/0.194
55.01/0. 1449
60.59/0.379
41.4000.259
54,07/0.055
57.04/0.070
52.48/-
$4.60/0.205
17.7500.045
Ours
20.30.0.089
28.40/0.118
17.75/0.129
30.31/0.122
4.36/00331
17.17/0.169
38.86/
21.86/0.100
B.8 EVALUATION STRATEGY FOR CATEGORY- ARTICULATED OBJECT POSES
To evaluate the category-level part pose estimation performance of our model, we adopt the evaluation strategy used in Li et al. (202T).
For part-based metrics, we first feed set of train shapes in the canonical articulation states and canonical object pose state to get set of per-part pose predictions (P.). Then can calculate the residual pose P; for each part from the canonical part space defined by human to the canonical part space defined by the network from the pose prediction set (via RANSAC). After that, predicted pose from the canonical part space defined by human can be computed by applying the inverse residual pose estimation on the estimated per-part pose, e.g. P P-- P. When calculating the rotation and translation from part shape X1 to X2, we first centralize their bounding boxes (X₂ and Xz, respectively). Then, the transformation from X1 to X₂ is taken the transformation from X1 to X2-
For joint parameters, we take the angle error between the predicted joint axis orientation and the ground-truth axis orientation as the metric for joint axis orientation prediction. Metric for joint position prediction is set to the minimum line-to-line distance, following Li et al. (2020a). Only joint axis orientation prediction error is computed for prismatic joints.
C DISCUSSION ON PART SYMMETRY
In this section, we discuss the sert-symmeeryy-reee related problem that one would encounter in the part pose estimation problem. For rigid objects, the pose of shape is ambiguous for symmetric shapes.
22

---

Published as a conference paper at ICLR 2023
Table 11: Performance comparison of the proposed method clean data and data corrupted by random normal
noise.
Category
Method
IoU
Mean Rore(" )*)
Median Rave (*)
Terr
Median Terr
Joint
Chamfer
Without
76.22
7.74, 4.07
7.35. 3,97
0.054. 0.067
0.052. 0.046
20.30 30.3690.089
0.025
With noise
55.35
9.84, 11.05
9.54, 9.99
0.073, 0.063
0.073, 0.037
9,28/0.100
0.049
Without noise
82.97
7.34, 10.41
9.36
0.040, 0.046
0.037, 0.042
30.31/0.122
0.024
Laptop (S)
70.04
9.52
32.8400029
Figure 11:
Visualization for an intuitive understanding w.r.t. the difference between the part-level cquivariant feature and globa equivariant feature of specific part. Visualized features are obtained by using the PCA algorithm reduce the feature dimension point features of non-motion part with the moving part in gray. Features drawn on the left global equivariant features while those on the right are from the part-level equivariant network.
which further normalized to the range of We only draw
To say shape X is symmetric, X = [X]. In those cases, the performance of the pose estimation algorithm may degenerate due to ambiguous poses. It is reasonable phenomenon, however. But for articulated objects, we may have symmetric parts even if the whole shape is not symmetric one. For those shapes, we still expect for accurate part pose estimation. indicates that estimating part poses for each part individually not reasonable due to part pose ambiguity. That's why choose to model the relationship between parts, or specifically, the kinematic chain, joint parameters. Without such object-level inter-part modeling, cannot get accurate part poses by estimating their pose individually, segmentation. The comparison between Oracle EPN and our method in Table I0 can demonstrate this point some extent.
that there is non-trivial SE(3) transformation SAo such that
using fround-truth
23


Reviews:
[{"review_id": "vZdp5ANR9XJ", "reviewer": "Reviewer_gB2X", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles the problem of category-level articulated object pose estimation, which requires articulated pose estimation of an unseen object from a known category. The authors propose a self-supervised method to reduce the need for annotations. This method factorizes the input point clouds into canonical shapes and part-level poses. By designing a network that learns a part-level equivariance property, the object shape could be reconstructed from the canonical parts. Experiments show that the proposed method can perform on par with or even better than existing methods that are supervised.\n", "summary_of_the_review": "Overall I am leaning towards accepting the paper, and I look forward to the authors' response to the above questions.", "strength_and_weaknesses": "Strengths:\n- The problem being studied is important and general enough to be applied in other scenarios.\n- The proposed method is intuitive in the sense that we as humans would perceive the canonical shapes of the object parts, not just the entire object.\n\nWeakness:\n- The presentation could be slightly improved. For example, some information in Figure 3 could be put into Figure 1 to help readers get a clearer picture.\n- From my understanding of the paper, the canonical shapes Z_i are instance-level concepts. Should we have a loss to make sure the shapes are aligned between different instance objects of the same category?\n- How are the number of parts controlled in the algorithm?\n- What is the stopping criterion for the iterations?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MBFQv22LHk", "reviewer": "Reviewer_QLf2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors present a novel self-supervised strategy to reduce the heavy annotations needed for supervised learning methods for Category-level articulated object pose estimation. Specifically, they design a pose-aware equivariant point convolution operator to learn part-level SE(3)-equivariant features. Then, they propose a self-supervised framework to achieve the disentanglement of canonical shape, object structure, and articulated object poses. After that, the authors further predict articulated object poses as per-part rigid transformations describing how parts transform from their canonical part spaces to the camera space. Extensive experiments demonstrate the effectiveness of their method.", "summary_of_the_review": "The authors study a challenging problem and propose a novel approach. This paper is well-organized and easy to follow. In addition, the experimental results are promising.", "strength_and_weaknesses": "Strength:\n1. The paper considers a challenging setting, where the human labels of the articulated object pose estimation task is not given. Category-level articulated object pose estimation is a very popular task. Getting more insight about each different method can be valuable to many people.\n2. The proposed method is novel and effective.\n3. The paper is well written and organized.\n4. Comprehensive experiments are conducted to support the effectiveness of the proposed method.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IvrwNLvRolM", "reviewer": "Reviewer_GUmv", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to estimate articulated object pose on the category level in a self-supervised manner, by applying SE(3) equivariance on the part level. SE(3) equivariant features are extracted for each point from the input point cloud. The point cloud segmentation is conducted using slot attention, which divides the point cloud into articulated parts. Then the equivariant features of each segmented part are aggregated to estimate the part-level and object-level canonical space. Meanwhile, the kinematic chain and the joint parameters are regressed. The entire pipeline is trained using a shape reconstruction loss and a joint regularization loss, without manual annotation on object/part-level pose, kinematic structure, or joint parameters. Experiments show that the articulation structure and joint status emerge in such a self-supervised learning process and the estimation results are reported to be close to or better than the supervised counterpart.  ", "summary_of_the_review": "The overall quality of the paper is very high. I’m excited to see decent results given the complex system and simple self-supervised loss. The description of the method still needs some improvement, but I would still argue for acceptance. ", "strength_and_weaknesses": "Strength\n1. It is very exciting to see that articulation emerges through the proposed self-supervised learning pipeline without any supervision. The system is complex, yet the results look very nice. Great work!\n2. The paper proposed to integrate pose and part awareness in the equivariant point convolution, which naturally handles different articulated poses and unifies the point cloud feature in a canonical coordinate frame. \n3. The results show that the proposed method achieves great performance, despite the system complexity and the simple self-supervised reconstruction loss. \n\nWeakness\n1. The paper is not well written, and the description of the proposed method is unclear. Different components are enumerated but a good overview of the method section is missing, for which I think Figure 3 in the appendix would be helpful. Specifically, what does “per-rotation articulated pose hypotheses” mean? What is the intuition behind the joint regularization loss? \n2. It is not clear what role SE(3)-equivariant operations play in the proposed method. A baseline replacing all equivariant operations with non-equivariant operations is missing, or an argument on why equivariant is necessary is needed. \n3. Experiments on real-world datasets are missing. (Figure 4 of ANCSH)\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "8qb1y7Ml7p", "reviewer": "Reviewer_sxak", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is working on self-supervised articulated object pose estimation at a category level. The objective is to estimate the canonical shape, articulated structure, and transformation that assembles all canonical shapes together. The supervision comes from the overall reconstruction. Experiments are carried out on 7 categories from 3 datasets. Result shows that the proposed method outperforms or on par with baselines, some of which have external supervision.", "summary_of_the_review": "The proposed method alleviates the need of data annotation for articulated object pose estimation. The result is fair given that it's self-supervised. ", "strength_and_weaknesses": "### Strengths\n- Self-supervision is a good idea when it comes to articulated and part-level tasks, where the annotation effort is high.\n- Many biases are carefully engineered to make sure the correct feature (invariant/equivariant) is used.\n- Based on the experimental results, the proposed self-supervision pipeline gets reasonable preformance.\n\n### Weaknesses / Questions\n- Section 3 is wordy and it would be helpful to re-organize.\n- In Part-assembling parameters prediction, only the translation is predicted. Is translation alone sufficient? It seems that this will make the reconstruction of canonical shape harder.\n- 'we first construct an adjacency confidence graph from object parts and then extract its maximum spanning tree consisting of the set of confident adjacency edges' How are the edge weights determined? How do you define the subset that is ‘the set of confident adjacency edges’?\n- 'The part proposal module groups N points in the input shape X into K parts' How is K determined? Does that affect the final performance?\n- Since the part segmentation learned by the proposed method comes from self-supervision, how can it be directly compared to the ICP and NPCS where there is ground truth part-segmentation? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster