Rethinking the compositionality of point clouds through regularization in the hyperbolic space
Authors: Antonio Montanaro
Diego Valsesia
Enrico Magli
Venue: NeurIPS 2022 Accept (2022)
Keywords: Hyperbolic space
Point cloud classification

Abstract: Point clouds of 3D objects exhibit an inherent compositional nature where simple parts can be assembled into progressively more complex shapes to form whole objects. Explicitly capturing such part-whole hierarchy is a long-sought objective in order to build effective models, but its tree-like nature has made the task elusive. In this paper, we propose to embed the features of a point cloud classifier into the hyperbolic space and explicitly regularize the space to account for the part-whole hierarchy. The hyperbolic space is the only space that can successfully embed the tree-like nature of the hierarchy. This leads to substantial improvements in the performance of state-of-art supervised models for point cloud classification.

Rethinking the compositionality of point clouds through regularization in the hyperbolic space
Antonio Montanaro' Politecnico di Torino, Italy antonio. montanaro@polito......
Diego Valsesia Politecnico di Torino, Italy diego. www..eaaaalaallioooooo.oo
Enrico Magli Politecnico di Torino, Italy
enrico wttoommlll@@ee pagli@polito.it
Abstract
Point clouds of 3D objects exhibit an inherent compositional nature where simple parts be assembled into progressively more complex shapes form whole objects. Explicitly capturing such part-whole hierarchy is long-sought objective in order to build effective models, but its tree-like nature has made the task elusive. In this paper, we propose to embed the features of a point cloud classifier into the hyperbolic space and explicitly regularize the space account for the part-war whole hierarchy. The hyperbolic space is the only space that can successfully embed the rike nature of the hierarchy. This leads substantial improvements the performance of state-of-art supervised models for point cloud classification.
1 Introduction
Is the whole more than the sum of its parts? While philosophers have been debating such deep question since the time of Aristotle, we can certainly say that understanding and capturing the relationship between parts constituents of whole complex structures is of paramount importance building models of reality, In this paper, we turn our attention to the compositional nature of 3D objects, represented point clouds, where simple parts can be assembled to form progressively more complex shapes. Indeed, the complex geometry of an object can be better understood by unraveling the implicit hierarchy of its parts. Such hierarchy can be intuitively captured by tree where nodes close to the root represent basic universal shapes, which become progressively more complex as we approach the whole-objec leaves. Transforming object into another requires swapping parts by traversing the tree up to a common ancestor part. is thus clear that a model extracting features, that claim capture the nature of 3D objects, needs to incorporate such hierarchy.
In the last years, point cloud processing methods have tried to devise methods to extract complex geometric information from points and neighborhoods. Architectures like graph neural networks 1 compose the features extracted by local receptive fields, with sophisticated geometric priors exploiting locality and self-similarity, while different school of thought argues that simple archi- tectures, such as PointMLP [3] and SimpleView View 44, with limited geometric priors are nevertherless very effective. It thus raises question whether prior knowledge about the data is being exploited effectively.
Info3D 6 and DCGLR 7 recognized the need to reason about local and global interactions in the feature extraction process. In particular, their claim that
In this sense, works such as PointGLR 5
"Code of the project: http:///gthhooooo///ggaaasssss/s
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

maximizing the mutual information between parts and whole objects leads to understanding of local and global relations. Although these methods present compelling results for unsupervised feature extraction, they still fall short of providing significant improvements when finetuned with supervision.
In our work, we argue that those methods do not fulfill their promise of capturing the part-whole relationship because they are unable to represent the tree-like nature of the compositional hierarchy. Indeed, their fundamental weakness lies in the use of spaces that are either flat (Euclidean) or with positive curvature (spherical). However, it is known that only spaces with negative curvature (hyperbolic) are able to embed tree structures with low distortion volume of the Euclidean space grows only as power of its radius rather than exponentially, limiting the representation capacity of tree-like data with an exponential number of leaves. This unique characteristic has inspired many researchers to represent hierarchical relations in many domains, from natural language processing principles for point clouds and 3D data is still unexplored.
This is due the fact that the
9
10 to computer vision 11 _12 However, the use of such
The main contributions of this paper lie in the following aspects:
we propose novel regularizer to supervised training of point cloud classification models that promotes the part-whole hierarchy of compositionality in the hyperbolic space;
this regularizer can be applied to any ofate-of-art architecture with a simple modification of its head to perform classification with hyperbolic layers in the regularized space, coupled with Riemannian optimization [13;
we observe a significant improvement in the performance a number of popular architec- tures, including offe-offthhearr techniques, surpassing the currently known best results on two different datasets;
we are the first to experimentally observe the desired phrt-whole hierarchy, by noticing that the geodesics in hyperbolic space between whole objects pass through common part ancestors.
2 Related work
Point Cloud Analysis Point cloud data are sets of multiple points and, in recent years, several deep neural networks have been studied process them. Early works adapted models for images through 2D projections
[1]. 15]. Later, PointNet [16] established new models working directly on the raw set of 3D coordinates by exploiting shared architectures invariant to points permutation. Originally, PointNet independently processed individual points through a shared MLP. To improve performance, PointNet++ [17] exploited spatial correlation by using a hierarchical feature learning paradigm. Other methods 18 19). [20), treat point clouds as a graph and exploit operators defined over irregular sets to capture relations among points and their neighbors at different resolutions. This is the case of DGCNN 21). where the EdgeConv graph convolution operation aggregates features supported on neighborhoods as defined by nearest neighbor graph dynamically computed in the feature space. Recently, PointMLP 3 revisits PointNet++ to include the concept of residual connections. Through this simple model, the authors show that sophisticated geometric models are not essential to obtain -tate-of-the-ae arr performance.
Part Compositionality Successfully capturing the semantics of 3D objects represented as point clouds requires to learn interactions between local and global information, and, in particular, the compositional nature of 3D objects constructed from local parts. Indeed, some works have focused on capturing global-local reasoning in point cloud processing. One of the first and most representative works PointGLR (5) In this work, the authors map local features different levels within the network to common hypersphere where the global features embedding is made close to such local embeddings. This is the first approach towards modeling the similarities of parts (local features) and whole objects (global features). The use of a hypersphere as embedding space for similarity promotion traces its roots in metrie learning works for face recognition 22]. In addition to the global-locs embedding, PointGLR added two other pretext tasks, namely normal estimation and self-reconstructton. to further promote learning of highly discriminative features. Our work significantly differs from PointGLR in multiple ways: i) positive curvature manifold such as the hypersphere unable to accurately embed hierarchies (tree-like structures), hence our adoption of the hyperbolic space; ii) we actively promote continuous embedding of part- whrle hierarchies by

---

penalizing the hyperbolic norm of parts proportionally to their number of points (a proxy for part complexity); iii) we move the classification head of the model to the hyperbolic space to exploit our regularized geometry. A further limitation of PointGLR is the implicit assumption of model generating progressive hierarchies (e.g. via expanding receptive fields) in the intermediate layers. In contrast, our work can be readily adopted by any state-of-the-ar model with just replacement of the final layers. Other works revisit the global-local relations using maximization of mutual information between different views [6]. clustering and contrastive learning [23], distillation with constrast [7]. self-similarity and contrastive learning with hard negative samples (44) Although most of these works include the contrastive strategy, they differ in the way they contrast the positive and negative samples and in the details of the self-supervision procedures, e.g., contrastive loss and point cloud augmentations. We also notice that most these works focus unsupervised learning, and, while they show that the features learned in this manner are highly discriminative, they are also mostly unable to improve upon state-of-the-an supervised methods when finetuned with full supervision. These approaches differ from the one followed in this paper, where we focus on regularization of a fully supervised method, and show improvements upon the supervised baselines that do not adopt our regularizer.
Hyperbolic Learning The intuition that the hyperbolic space is crucial to embed hierarchical structures comes from the work of Sarkar [8] who proved that trees can be embedded in the hyperbolic space with arbitrarily low distortion. This inspired several works which investigated how various frameworks of representation learning can be reformulated in non-Euciee -on-Euclidean manifolds. In particular,
9 [3 and [1O] were some of the first works to explore hyperbolic representation learning by
introducing Riemannian adaptive optimization, Poincarè embeddings and hyperbolic neural networks for natural language processing. The new mathematical formalism introduced by Ganea et al. [TO] was decisive to demonstrate the effectiveness of hyperbolic variants of neural network layers compared to the Euclidean counterparts. Generalizations to other data, such as images (25 and graphs (26) with the corresponding hyperbolic variants of the main operations like graph convolution (26] and gyroplane convolution [12] have also been studied. In the context of unsupervised learning, new objectives in the hyperbolic space force the models to include the implicit hierarchical structure of the data leading to a better clustering in the embedding space [2]. [1]. To the best of our knowledge, no work has yet focused on hyperbolic representations for point clouds. Indeed, 3D objects present an intrinsic hierarchy where whole objects are made by parts of different size. While the smallest parts may be shared across different object classes, the larger the parts the more class-specific they become. This consistently fits with the structure of tree where simple fundamental parts are shared ancestors of complex objects and hence we show how the hyperbolic space can fruitfully capture this data prior.
3 Method
In this section we present our proposed method, named HyCoRe (Hyperbolic Compositional Regu- larizer). An overview is presented in Fig. [ At a high level, HyCoRe enhances any state-of-the-a neural network model for point cloud classification by 1) replacing its last layers with layers perform- ing transformations in the hyperbolic space (see Sec. [32).. and 2) regularizing the classification loss to induce a desirable configuration of the hyperbolic feature space where embeddings of parts both follow a hierarchy and cluster according to class labels.
3.1 Compositional Hierarchy in 3D Point Clouds
The objective of HyCoRe is to regularize the feature space produced by a neural network so that captures the compositional structure of the 3D point cloud at different levels. In particular, we notice that there exists hierarchy where small parts (e.g., simple structures like disks, squares, triangles) composed of few points are universal ancestors to more complex shapes included in many different objects. As these structures are composed into more complex parts with more points, they progressively become more specific to an object or class. This hierarchy can be mathematically represented by tree, as depicted in Fig 2 where simple cylinder can be the ancestor of both pieces of chair or table. While the leaves in the tree are whole objects, thus belonging to a specific class, their ancestors are progressively more universal the higher up in the hierarchy they sit.
3

---

HYBBRBBLLI
HYPERBOLIC CLASSIFICATION HEAD
BACKBONE
Exp
Mobius layer
Class
Class Loss
Object
cr
or Part
Contrastive Reg. Roore
Hierarchy Reg.
RHor
Small part. close Center
Different class: repel
Whole obgezk
Same class:
close border
attract
Hierarchy Regularizer Ryser
Contrastive Regularizer Raw
Figure 1: HyCoRe overview. point cloud classification model is regularized by promoting the
feature space to include compositional information. Hierarchy regularizer: simple parts should be mapped closer to the center of the Poincarè disk (common ancestors of whole objects). Contrastive regularizer: parts of the same class should be embedded closer than parts of other classes.
At this point, it is important noting that the graph distance between leaves is determined by the shortest path passing through the first common ancestor for objects in the same or similar classes, while objects from significantly dissimilar classes have the shortest path passing through the root of the hierarchy. In order to ensure that we can embed this tree structure in feature space, we need space that preserves the geometrical properties of trees and especially the graph distance. In particular, the embedding space must be able to accommodate the exponential volume growth of tree along its radius. classic result by Sarkar [ showed that flat Euclidean space does not provide this, leading to high errors when embedding trees, even in high dimensions. On the contrary, the hyperbolic space, a Riemannian manifold with negative curvature, does support exponentially increasing volumes and can embed trees with arbitrarily low distortion. Indeed, the geodesic (shortest. path) between two points in this space does pass through points closer to the origin, mimicking the behavior of distance defined over tree.
In particular, we will focus the Poincarè ball model of hyperbolic space. Since hyperbolic space is non- -ucliieen manifold, it cannot benefit from conventional vector representations and linear algebra. As a consequence, classical neural networks cannot operate in such a space. However, we will use extensions [10] of classic layers defined through the concept of gyrovector spaces.
3.2 Hyperbolic Space and Neural Networks
The hyperbolic space is Riemannian manifold with constant negative curvature. The curvature determines the metric of space by the following formula:
gR = (A2) gE =
+ c||x| gE
(1)
where gR is the metric tensor of a generic Riemannian manifold, X is the conformal factor that depends the curvature and the point which is calculated. and gø is the metric tensor of the Euclidean space R", i.e., the identity tensor I, Note how the metric depends on the coordinates (through |x|]) for с * 0, and how Riemannian manifold with zero curvature. Spaces with c > are spherical, and with c < hyperbolic.
0 yields ga 2gæ, i.e., the Euclidean space is flat
The Poincare Ball in n dimensions ah is a hyperbolic space with -1, and is isometric to other models such the Lorentz model. The distance and norm defined as:

---

ENCOORR
ENCODER
065
1074
111
(NCOORS
BsccOss
Figure 2: 3D objects possess inherent hierarchies due to their nature as compositions of small parts. The hyperbolic space embed trees and hierarchical structures with lower distortions than the Euclidean space. The number of points the embedded part point cloud is highlighted in figure. Embeddings shown experimental results projected to 2D Poincarè disk with hyperbolic UMAP.
yl1 dp y) cosh (1+2 (1 1) )
||x||e tanh ([|xl])
(2)
Since the Poincarè Ball is a Riemannian manifold, for each point x E ID" we can define logarithmic map logx DD' + TxD" that maps points from the Poincare Ball the corresponding tangent space TyI"" E IR",, and an exponential map expx TxID" + D" that does the opposite. These operations [TO] are fundamental move from one space to the other and viceversa.
The formalism to generalize tensor operations in the hyperbolic space called the gyrovector space, where addition, scalar multiplication, rector-matris multiplication and other operations redefined as Möbius operations and work in Riemannian manifolds with curvature These become the basic blocks of the hyperbolic neural networks. In particular, will use the hyperbolic feed forward (FF) layer (also known as Möbius layer). Considering the Euclidean case, for a FF layer, we need a matrix M IRn Rn linearly project the input x R" the feature space RT and, additionally, translation made by a bias addition, i.c., + b with y, b E R" and, finally, pointwise non- lineeiiyyy T T
Matrix multiplication, bias and pointwise non-linearity are replaced by Möbius operations in the gyrovector space and become:
|Mx|| |||||
Mx ||x||
y = 0 (x) = Vc
tanh
Lanh ((vel(()[lll)))
(3)
A6
z = y 1. b= exps
logg(b))
(0) = exps (o(log((z))
(4)
where M and b are the same matrix and vector defined above, c is the magnitude of the curvature. Note that when - we recover the Euclidean feed-forward layer. An interesting property of the Möbius layer is that it is highly nonlinear; indeed the bias addition in hyperbolic space becomes a nonlinear mapping since geodesics are curved paths in non-flat manifolds.
3.3 Hyperbolic Compositional Regularization
Armed with the formalism introduced in the previous section, we are ready to formulate our HyCoRe framework, anticipated in Fig. 1 Consider point cloud Px as a set of 3D points p E R with N elements. We use any state-of-the-an point cloud processing network as a feature extraction backbone E I8 x3 I to encode Px in the corresponding feature space. At this point we apply an
5

---

exponential map expo R - 1D" to map the Euclidean feature vector into the hyperbolic space and then a Möbius layer ID + IDV to project the hyperbolic vector in an f-dimensional Poincarè hall. This is the hyperbolic embedding of the whole point cloud PN. i.c., Zwhole H (exp(E(Px)) We repeat the same procedure for a sub-part of PN. which we call Pv with a number of points N' N, create the part embedding Zpart ((exx((((xx))) ID in the same feature space as before.
DV.
We now want to regularize the feature space induce the previously mentioned properties, namely the phrt-whole hierarchy and clustering according to the class labels. This is performed by defining the following triplet regularizers:
Rhier wholle *pout) = max (0, -1|z whole + || 1| 11 + +7/N²)
(5)
Reontr (wholee Zpants Zpan) = max (0, dp (z (Zwhlle: Zpan) - dp (Zwhole+ 2pan) + +6)
(6)
where x whole and ppat are the hyperbolic representation of the whole and a part from the same point cloud, while put is the embedding of part of a different point cloud from different class.
The Ruier regularizer in Eq. (5) induces the com- positional part-whole hierarchy by promoting part embeddings to lie closer to the center of the Poincarè ball and whole embeddings to be closer the edge. In particular, we use a vari- able margin 7/N' that depends on the number of points N' of the part Pv. This means that shapes composed by few points (hence simple universal shapes) will far from the whole ob- ject representation and with lower hyperbolic norm (near the centre). On the other hand, em- beddings of larger parts will be progressively closer to the edge the Poincarè ball, depend- ing on the part size. Since geodesics between two points pass closer to the ball center (Fig. 3 this structure we impose to the space allows to visit common part ancestors while traversing geodesic between two whole objects. This regularization thus mimics continuous version of part-whole tree embedded in the Poincarè ball.
Common part ancestor
Geodesic patn
Object A
Object B
Figure 3: Geodesic path.
The Remmir regularizer in Eq. (6) promotes correct clustering of objects and parts in the hyperbolic space. In particular, parts and whole of the same point cloud are promoted be close while a part from a different class is mapped far apart with respect to the other whole. It ensures that the parts of point cloud of a different class are in terms geodesic distance. a margin hyperparameter to control the degree of separation between positive and negative samples.
The two regularizations are included the final loss in this way:
L = ICE + Or Regnnt + Baier
where LAE is the conventional classification loss (e.g., cross-entropy) evaluated on the whole objects. The classification head is a hyperbolic Möbius layer followed by softmax. In principle, one could argue that LCE could already promote correct clustering according to class labels, rendering Reeatr redundant. However, several works 10) have noticed that the Möbius-softmax hyperbolic head is weaker than its Euclidean counterpart. We thus found it more effective to evaluate LeE the whole objects only, and use Rewnte as a metric penalty that explicitly considers geodesic distances to ensure correct clustering of both parts and whole objects.
At each iteration of training with HyCoRe we sample shapes with random N' varying within a predefined range. A part is defined as the N' nearest neighbors of a random point. In future work, it would interesting to explore alternative definitions for parts, e.g., using part labels if available but, at the moment, only address definition via spatial neighbors avoid extra labeling requirements.

---

Table Classification results on ModelNet40. *: re-implemented. **: re-implemented but did not
exactly reproduce the reference result.
Method
AA(%)
OA(%)
Training
+
90.5
supervised
DDGCNN 21
90.2
92.9
supervised
Point Transformer 30
90.6
93.7
supervised
PA-DGC [31
93.6
supervised
CurveNet 32
93.8
supervised
** *PonntMLP[3
91.2
93.4
supervised
*PonnMMP (voting)
91.4
93.7
supervised
DGCNN+Self-Reoon.
33
92.4
finetuned
DGCNN+STRL 34
93.1
finetuned
DGCNN+DCGLR
93.2
finetuned
90.6
finetuned
Pointett+++Hyooke
91.1
regularized
DGCNN +HyCoRe
91.0
93.7
regularized
PointMLP +HyCoRe
91.7
94.3
regularized
PointMLP +HyCoRe (voting)
91.9
94.5
regularized
Table 2: Classification results on ScanObjectNN.
Method
AA(%) OA(%)
Table 3: Effectiveness of hyperholic space.
DGCNN
77.8
80.3
Average Accuracy (%)
SimpleView 4
80.8
PRANet[35
79.1
82.1
Dim
16
64
256
512 1024
MVTN 361
82.8
DGCNN
76.6 77.5 77.8 76.6 76.3
PointMLP
84.4
86.1
DGCNN+BuCoRe
78.2
78.9
79.0 78.8 79.0
**PointNeX(177
86.4
88.0
Hype-DGCNN
76,8
75.9
76.5
76.0 77.5
DGCNN=HyyCoR
80.2
82.1
DGCNN+HYCORe
79.1 80.0
80.2
80.2 79.7
Poin
85.9
87.2
PointNeXt-HyCoRe
87.0
88.3
4 Experimental results
4.1 Experimental setting
We study the performance of our regularizer HyCoRe the synthetic dataset ModelNet40 (12,331 objects with 1024 points, 40 classes) and on real dataset ScanObjectNN 28 (15,000 objects with 1024 points, 15 classes). We apply our method over multiple classification architectures, namely the widely popular DGCNN and PointNet++ baselines, as well as the recent state-of of-thh--ttt art PointMLP model. We substitute the standard classifier with its hyperbolic version (Möbius+softmax) as shown in Fig. [ We use f = 256 features to be comparable to the official implementations the test the model over different embedding dimensions in the ablation study. Moreover, we set a = B = 0.01, 7 = 1000 and = 4. For the number of points of each part N', we select random number between 200 and 600, and for the whole object random number between 800 and 1024 to ensure better flexibility of the learned to model to part sizes. We train the models using Riemannian SGD optimization. Our implementation is Pytorch and use geoopt 22 for the hyperbolic operations. Models are trained on an Nvidia A6000 GPU.
27
Euclidean space, then
4.2 Main Results
Table shows the results for ModelNer40 classification. In the first part we report well-known and sfate-of-the-art supervised models. We retrained PointNet++, DGCNN and the state-of-the of- art PointMLP as baselines, noting some documented difficulty [38] with exactly reproducing the

---

Figure Embeddings produced by the hyperbolic encoder, projected 2 dimensions with hyperbolic UMAP. Each color represents a class; small points correspond to parts; large points correspond to whole objects. Parts are closer the center, sitting higher in the hierarchy (whole objects at the border may share common part ancestor reachable via the geodesic connecting the objects).
Table 4: Classification results when one of the regularizations omitted.
Table 5: Performance vs.
curvature of the
Poincarè Ball
AA(%)
OA(%)
Average Accuracy (%)
DGCNN
77.8
80.3
Curvature
1
0.5
0.1
0.01
DGCNN+ Phier
77.9
80.5
DGCNN+ Reonntr
79.2
81.6
Hype-DGCNN
76.5
76.9
76.6
76.9
DGCNN=HyCoRe
80.2
82.1
DGCNN+HyCoRe
80.2
79.4
78.7
78.5
official results. addition, the second part of the table reports the performance methods (3
proposing self-supervised pretraining techniques, after supervised finetuning. Concerning PoRntGLR [5), the most similar method HyCoRe, we ensure a fair comparison by using only the L2G embedding loss and not the pretext tasks of normal estimation and reconstruction.
34
Finally, the last part of the table presents the results with HyCoRe applied the selected baselines. We can see that the proposed method achieves substantial gains not only compared to the randomly initialized models, but also compared to the finetuned models. When applied to the PointMLP, HyCoRe exceed the state-of-th thee----thhhrrr art performance on ModelNet40. Moreover, it is interesting to notice that the embedding framework of PointGLR is not particularly effective without the pretext tasks. This is due to the unsuitability of the spherical space to embed hierarchical information, as explained Sec. 2 and indeed not far from results we obtain with our method in Euclidean space.
Table reports the classification results on the ScanObjectNN dataset. Also in this case, HyCoRe significantly improves the baseline DGCNN leading
be comparable with the state-of-the-at methods such as Simple [4], PRANet (35] and MVTN (36]. In addition, PointMLP that holds the state of the art for this dataset, is further improved by our method and reaches impressive overall accuracy of 87.2 %, substantially outperforming all the previous approaches. Although the authors in [ claim that classification performance has reached saturation point, we show that including novel regularizers in the training process can still lead to significant gains. This demonstrates that the proposed method leverages novel ideas, complementary to what is exploited by existing architectures, and it is thus able to boost the performance even of tate-of-the-an methods. It is also remarkable that an older, yet still popular, architecture like DGCNN able outperform complex and sophisticated models such the Point Transformer, when regularized by HyCoRe.

---

Table 6: Hyperbolic Norms of labeled parts from the whole object up to the single parts.
Table
Plane+uprights
Legs+uprights
Plane
Legs
Uprights
5.32
4.56
2.08
4.07
2.05
1.99
Aircraft 4.98
Winnsetail-engines 4.56
Wings+tail 4.45
Wings 4.22
Fuselage 3.37
Tail 2.94
In addition, further prove that enforcing the hierarchy between parts is useful to build better clusters, we show in Fig. 4 2D visualization with UMAP of the hyperbolic representations for the ModelNet40 data. Colors denote classes, big points whole objects and small points parts. Besides the clear clustering according to class labels, it is fascinating to notice the emergence of the part-whole hierarchy with part objects closer to the center of the disk. Importantly, some parts bridge multiple classes, such as the ones in the bottom right zoom, i.e., they are found along geodesic connecting two class clusters, serving as common ancestors. This can happen due to the fact that some simple parts having roughly the same shape appear with multiple class labels during training, and the net effect of Reomer is to position them midway across the classes.
The tree-likeness of the hyperbolic space can be also be seen in the visualization in Fig. (right). There embed shapes with gradually large number of points up to the whole object made by 1024 points. We can notice that the parts are moved towards the disk edge as more points are added. Furthermore, quantitative analysis of the part-whole hierarchy is shown in Table 6 Here we calculated the hyperbolic norms of compositions of labeled parts. We can see that, as the parts are assembled with other parts, their hyperbolic norms grow, up the whole object that is pushed close to the ball edge.
4.3 Ablation study
In the following we show an ablation study focusing on the DGCNN backbone and the ScanObjectNN dataset. The dataset selection is motivated by the fact that it is a real dataset, able to provide more stable and representative results compared to ModelNet40.
We first compare HyCoRe with its Euclidean version (EuCoRe) to investigate the effectiveness of the hyperbolic space. The basic principles and losses the same, but in EuCoRe distances and network layers are defined in the Euclidean space. Table 3] shows the results. With Hype-DGCNN we indicate the hyperbolic version of DGCNN, as represented in Fig. [ but without any regularization, serving as a baseline to assess the individual effect of the regularizer. We also test the models over a different number of embedding dimensions. We can see that FuCoRe only provides a modest improvement, underlining the importance of the hyperbolic space. We also notice that the hyperbolic baseline struggles be on par with its Euclidean counterpart, as observed by many recent works (25). [10]. However, when regularized with HyCoRe, we can observe significant gains, even in low dimensions. This also leaves an open research question, about whether better hyperbolic baselines could be built so that HyCoRe starts from a less disadvantaged point.
In Table 4 we ablate HyCoRe by removing combination of the two provides the overall best gain.
of the two regularizers. We can see that the
In order to study the effect of different space curvatures c, Jevaluuus HyCoRe from the standard curvature down to 0.01. We remark that some works 125).13 39), report significant improvements when is very low (c.g., 0.001), but this is counter-intuitive since the hyperbolic space then resembles an almost flat manifold. On the contrary, we do see improved results at higher curvatures.
Since HyCoRe constrains the network to learn the relations between parts and whole object, we claim that, at the end of the training process, the model should be better able classify coarser objects. In Figs. 5a and 5b we show the test accuracy of DGCNN on ModelNet40, when presented with a uniformly subsampled point cloud and with a small randomly chosen and spatially-contiguone part, respectively. Indeed, we can notice that HyCoRe provides a gain up to 20 percentage points for very sparse point clouds, and also able to successfully detect the object from smaller parts. For fair comparison, we also report the baseline DGCNN with training augmented by random crops

---

80
Test OA (%)
62
60
Test OA
40
DGCNN+HYCORE AGg
DGCNN HyCoRe DGCNN+ Aug
20
20-
DGCNN
DGCNN
Number of input points
Part size (points)
(a) Subsampled input. HyCoRe is more robust when the point cloud has coarser sampling.
(b) Parts with different size. HyCoRe better detects objects from only small part.
Figure 5: Test inference of DGCNN on ModelNet40.
of parts. Even though the augmentation is useful to improve accuracy, HyCoRe more effective demonstrating the importance of compositional reasoning.
5
Conclusions
Although deep learning in the hyperbolic space is in its infancy, in this paper showed how can successfully capture the hierarchical nature of 3D point clouds, boosting the performance of state-of-the-arr models for classification. Reasoning about the relations between objects and the parts that compose them leads not only to better results but also more robust and explainable models. In the future, would be interesting to explore different ways of defining parts, based on spatial nearest neighbors but rather on more semantic constructions. One important extension is to adapt HyCoRe to segmentation. Since segmentation aims to classify single points and the corresponding parts, contrary to classification, the parts embeddings should be placed on the boundary of the Poincarè Ball, where there is more space to correctly cluster them, and the whole objects (made by composition of parts) near the origin. We could exploit the label of the parts to this end or investigate unsupervised settings where the part hierarchy emerges naturally.
Acknowledgments and Disclosure of Funding
Computational resources were provided by HPC@ POLITO, a project of Academic Comput- ing within the Department of Control and Computer Engineering at the Politecnico di Torino (ttp://www.hhc.ppliio..t) This research received no external funding.
References
[1] W. Shi and R. Rajkumar, "Point-gnn: Graph neural network for object detection a point cloud," in Proceedings the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 1711-1719.
[2] H. Chen, Liu, W. Chen, Li, and R. Hill, "Equivariant point network for 3d point cloud anal- ysis," in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021, pp. 14514-14523.
[3] X. Ma, C. Qin, H. You, H. Ran, and Y. Fu, "Rethinking network design and local geometry in point cloud: A simple residual mlp framework," arXiv preprint arXiv:2202.07123. 2022.
[4] A. Goyal, H. Law, B. Liu, A. Newell, and J. Deng, "Revisiting point cloud shape classification with a simple and effective baseline," International Conference on Machine Learning. PMLR, 2021, pp. 3809-3820.
10

---

[5] Y. Rao, J. Lu, and J. Zhou, "Global-loca" bidirectional reasoning for unsupervised representation learning of 3d point clouds," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5276-3365555
[6] A. Sanghi, "Info3d: Representation learning on 3d objects using mutual information maximiza- tion and contrastive learning," in European Conference on Computer Vision. Springer, 2020, pp. 626-642.
[7] K. Fu, P. Gao, R. Zhang, H. Li, Y. Qiao, and M. Wang, "Distillation with contrast is all you need for self-supervised point cloud representation learning," arXiv preprint urXiv:2202.04241, 2022.
[8] R. Sarkar, "Low distortion delaunay embedding of trees in hyperbolic plane," in International Symposium Graph Drawing. Springer, 2011, pp. 355-366.
[9] M. Nickel and D. Kiela, "Poincaré embeddings for learning hierarchical representations." Advances in neural information processing systems, vol. 30, 2017.
[10] O. Ganea, G. Bécigneul, and T. Hofmann, "Hyperbolic neural networks," Advances in neural information processing systems, vol. 31, 2018.
[11] Z. Weng. M. G. Ogut, S. Limonchik, and Yeung, "Unsupervised discovery the long-tail in instance segmentation using hierarchical self-supervision." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2603-2612.
[12] J. Hsu, J. Gu, G. Wu, W. Chiu, and S. Yeung. "Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations," Advances in Neural Information Processing Systems, vol. 34, 2021.
[13] G. Becigneul and O.-E. Ganea, "Riemannian Adaptive Optimization Methods," in International Conference https//ppeenviiv...ee//mom//ti/p
Learning Representations, 2019. [Online]. Available:
[14] H. You, Y. Feng, R. Ji, and Y. Gao, "PVNet: joint convolutional network of point cloud and multi-view for 3D shape recognition," in Proceedings of the 26th ACM international conference on Multimedia, 2018, pp. 1310-1318.
[15] L. Li, S. Zhu, H. Fu, P. Tan, and C.-L. Tai, "End-to-end learning local multi-view descriptors for 3d point clouds," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1919-1928.
[16] R. Qi, H. Su, K. Mo, and L. J. Guibas, "Pointnet: Deep learning on point for 3d classification and segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652-660 660.
[17] C. R. Qi. L. Yi. H. Su, and L. J. Guibas, "Pointnet++: Deep hierarchical feature learning on point sets metric space," " Advances neural information processing systems, vol. 30, 2017.
[18] Y. Liu, B. Fan, S. Xiang, and C. Pan, "Relation-shapy convolutional neural network for point cloud analysis." " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1895-89044
[19] W. Wu, Z. Qi, and L. Fuxin, "Pointconv: Deep convolutional networks on 3d point clouds," in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9631-9630.
[20] Y. R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, "PointCNN: Convolution on X-transformed points," Advances neural information processing systems, vol. 31, 2018.
[21] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, "Dynamic graph cnn for learning on point clouds," ACM Transactions on Graphics (TOG), vol. 38, no. 5, p. 146, 2019.

---

[22] J. Deng, J. Guo, N. Xue, and S. Zafeiriou, "Arcface: Additive angular margin loss for deep face recognition." in Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, 2019, pp. 4690-4699.
[23] G. Mei, L. Yu, Q. Wu, and Zhang, "Unsupervised learning on 3d point clouds by clustering and contrasting," arXiv preprint arXiv:2202.0255.. 2022.
[24] B. Du, X. Gao, W. Hu, and X. Li, "Self-contrastive learning with hard negative sampling for self-supervised point cloud learning." in Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 3133-3142.
[25] V. Khrulkov, L. Mirvakhabova, E. Ustinova, I. Oseledets, and V. Lempitsky, "Hyperbolic image embeddings." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6418-6428.
[26] R. Ying. R. Hc, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, "Graph convolu- tional neural networks for web-scale recommender systems," in Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp. 974 983..
[27] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, "3d shapenets: A deep representation for volumetric shapes," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1912-1920.
[28] M. A. Uy Q.-H. Pham, B.-S. Hua, Nguyen, and S.-K. Yeung, "Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data," in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1588- 1597.
[29] M. Kochurov, R. Karimov, and S. Kozlukov, "Geoopt: Riemannian optimization in pytorch," 2020.
[30] H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V. Koltun, "Point transformer," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 259--11 162..
[31] M. Xu, R. Ding, H. Zhao, and X. Qi, "Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3173-3182.
[32] T. Xiang, C. Zhang, Y. Song. J. Yu, and W. Cai, "Walk in the cloud: Learning curves for point clouds shape analysis," " in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 915-924.
[33] Sauder and B. Sievers, "Self- self-suuerrviee deep learning on point clouds by reconstructing space," Advances in Neural Information Processing Systems, vol. 32, 2019.
[34] S. Huang, Y. Xie, S.-C. Zhu, and Y. Zhu, "Spatio-temporal self-supervised representation learning for 3d point clouds," in Proceedings of the IEEE/CVF International Conference Computer Vision, 2021, pp. 6535-656 6545.
[35] S. Cheng, X. Chen, X. He, Z. Liu, and X. Bai, "Pra-net: Point relation-aware network for 3d point cloud analysis," " IEEE Transactions on Image Processing, vol. 30, pp. 4436-4448, 2021.
[36] A. Hamdi, Giancola, and B. Ghanem, "Mvtn: Multi-view transformation network for 3d shape recognition," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1-11.
[37] G. Qian, Y. Li. H. Peng, J. Mai, H. A. A. K. Hammoud, M. Elhosciny, and B. Ghanem, "Pointnext: Revisiting pointnet++ with improved training and scaling strategies," arXiv preprint M.Xiv:2206004600 2022.
[38] "Reproducing PointMLP on ModelNet40." http://ghthhccoo/////////////l/t accessed: 2022-05-16.
[39] A. Ermolov, L. Mirvakhabova, V. Khrulkov, N. Sebe, and I. Oseledets, "Hyperbolic Vision Transformers: Combining Improvements in Metrie Learning." arXiv preprint arXiv:2203. 10833 2022.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations your work? [Yes] We discussed that the hyperbolic baseline is generally weaker than its Euclidean counterpart, and that it is unclear different definitions for parts could be exploited.
(c) Did you discuss any potential negative societal impacts of your work? [No] Our work foundational research and we do not see a direct path towards malicious use of the specific regularization technique presented in the paper.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] We provide a draft version of the code in the supplementury material. The full version along with detailed instructions to reproduce the experiments will be released upon acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We use standard datasets and splits well known by the literature and report all values of the hyperparameters for our tests. Hyperparameters were either chosen ensure fair comparisons or by rough poossvalidatiin..
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [No] We follow the practice of all existing literature report the best value to ensure fair comparisons with existing works.
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "fY9DiJPFCI", "reviewer": "Reviewer_45jd", "summary": "The paper proposes learning shape representation for the 3D shape classification task, \nby regularizing embeddings in hyperbolic space. The intuition on which this paper is based\nis that complex shapes can be made by combining simpler parts and this composition can\nbe explained by a tree-like hierarchy. The paper proposes regularizing shape embeddings\nsuch that simplest and most basic parts are embedded at the root level and entire shapes\nare embedded at the leaf level, where embeddings are defined using hyperbolic space.\nSpecifically, a shape embedding is first mapped to hyperbolic space and then a mobius layer\nis applied to project it to Poincare ball. The paper proposes two regularization, first is to \nencourage the whole shape embeddings to be close to leaf level and part level embeddings\nare close to the root. The second regularization encourages a shape and its parts\nto be closed to each other in embedding space and far from embeddings of parts from different\nshapes.\nThis approach consistently improve the performance of shape classification task on several\nneural network architectures.", "questions": "1. Is it easy to extend this approach to semantic segmentation task? If yes, why did authors not include those experiments? If not, can authors include some discussion on this matter.", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The paper is clearly written and explains the motivation of the proposed approach well.\n2. The proposed approach is well described\n3. All experiment details are provided ensuring reproducibility.\n4. The proposed approach consistently improve the performance using several neural network architecture, ensuring generalizability.\n5. All ablations are provided that all components are pertinent.\n**Weakness**\n1. The main weakness I see is that only shape classification is chosen to benchmark the approach. Specifically, since the embeddings are better aware of the composition the embeddings can shine in part segmentation task.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PP4emTAJXDC", "reviewer": "Reviewer_p5Dv", "summary": "The authors present a novel representation for latent space of point clouds to capture and take advantage of the hierarchical nature of the underlying shapes. The proposed method encourages point clouds to be projected to a hyperbolic space, where common simple structures are forced to the center of the space, and the complex shapes are moved to the edge.\n\nTo achieve this they propose a module that can be added onto existing frameworks. The module takes the feature vector of a set of points representing a shape and projects it to a Poincarè ball. To achieve the proposed hierarchical representation, they also extract partial point cloud from the same shape and other shape as input and use it in the regularization term. The constraints push the partial point clouds to the center, while keeping the parts of the same point cloud close and pushing back parts from other point clouds. \n\nThe regularization strategy establishes a more effective latent feature space, as seen from the classification accuracies on the benchmark datasets ModelNet40 and ScanObjectNN, as well as from the visualization of the resulting space, which shows features from different classes are kept apart.", "questions": "I wondered if the partial point clouds would contaminate the feature space of the backbone network. Have the authors tried freezing the backbone networks so that the feature calculation backbone network is not affected by the triplet loss?\n\nI also wondered what would happen if the partial point clouds created in each epoch for the constraints were centralized. It would seem to lead to a more robust common part ancestor, but leading to difficulty in establishing hierarchy between whole and partial shapes.\n\nIf segmentation was conducted in the suggested way in the section above, how would the results be? It is a shame that partial information is only utilized to map the point clouds in the hyperbolic space. All the efforts to include such data do not seem to be fully utilized.\n\nHave the authors attempted to use other sets of data for visualization? Although slightly out of context, it may have been better to use a more simple set of targets for the visualization task, such as humans in different poses. Human models have parts that can be used as the partial data, and commonality and transformation are easier to observe.", "limitations": "The authors do mention the procedure of preparing the partial shapes to be the main limitation of the proposal. The method currently takes a random point and collects $N'$ nearest points to define one partial shape. Using segment data could improve the partial representation of the proposed feature space. \n\nAlso, I believe lack of a framework to use the obtained feature for shape segmentation is also a drawback, as the authors go so far as to including parts of shapes as input to the proposed framework.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nThe representation is very novel and interesting, as it attempts to take advantage of the hyperbolic space in order to establish hierarchical relationships that starts from common simple parts to wholistic characteristic shapes. The authors thoroughly explain the geometric background of the proposal, and define a sound and practical constraints that drives data to be in the desired respective positions in the hierarchy.\n\nThe training procedure of preparing partial shapes to establish this hierarchy is also unique. The resulting feature space, as illustrated in Fig. 4, suggests that the constraints on the feature space have effectively mapped the latent features of even partial data to the desired locations.\n\nThe fact that the proposed module can be placed on top of existing point cloud analysis backbone networks is a huge advantage. By employing this representation, the conventional methods gain massive boosts in classification accuracy, which can be observed from Tables 1 and 2. \n\n### Weaknesses\n\nThe training procedure seems to require effort. From the text, the partial shapes have to be prepared at each epoch just to calculate the triplet loss defined in the paper. As the process is rather random, we can easily imagine the training process taking much longer than most existing frameworks. The burden of the training process seems to be excluded in the experimental section. \n\nIt is doubtful whether the claimed hierarchy is actually achieved by the proposal. The colormap in Fig. 4 seems to be well-organized near the edge of the ball, but seems rather random near the middle. The interpolation results in Fig.2 of the appendix, despite the efforts to incorporate various partial shapes in the training phase, also isn't as convincing as desired. They are definitely smaller in size, but the claimed shape commonality is difficult to observe. \n\nDespite the efforts to consider parts of shapes, it is disappointing that the authors do not include any point segmentation task, which is deeply related to local information of shapes, in the evaluation. One can easily imagine using the $\\mathbf{z}_{whole}$ feature as the global feature, which can be concatenated with conventional point-wise feature to conduct such task.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NcdjnjUsW68", "reviewer": "Reviewer_GCkG", "summary": "This paper presents a method for promoting the part-whole hierarchy in the learned feature space. In particular, it proposes to embed the features of a point cloud encoder into hyperbolic space. The part-whole hierarchy is enhanced through an explicit regularizer. Such a key idea is backed by the theory that the hyperbolic space (space with negative curvature) is the only space that can embed tree structures with low distortion. A regularizer layer is proposed for supervised training of point cloud classification models. It can be applied to existing architectures with a simple modification. Performance boost across a number of popular architectures is reported in the experiments. ", "questions": "* It is not clear to me why Equation (5) can promote part embeddings to lie closer to the center while whole embeddings to the edge. The main reason is that Equation (5) only specifies constraints on the distance between the embedding of whole and part. It only encourages the part and the whole feature to stay apart. Why the part feature (instead of the whole feature) is guaranteed to be pushed to the center? What is the technical insight behind it?\n\n* Why hierarchy learning can be beneficial to the task of classification? In the end, only the whole shape is used for classification, and the parts are not used as the input. What is the advantage of the proposed method over a naive method that simply applies contrastive learning on the feature space of the whole shape?", "limitations": "No limitations or potential negative societal impact are provided in the paper. I would recommend the authors provide some failure cases or limitations in the rebuttal (if any). ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n*  The idea of promoting a part-whole hierarchy in the hyperbolic space provides a novel perspective for learning discriminative features of the point cloud, which I believe is beneficial to the community. Though it is only verified in the task of classification, I think it could be valuable for more tasks that require in-depth analysis of part hierarchies, e.g. matching incomplete point cloud, point cloud generation from parts, etc.  \n\n* The experiments have shown steady performance improvement by applying the proposed method to existing mainstream backbones.\n\n* Nice visualization of the formed feature space for a better understanding of the effectiveness of the proposed approach.\n\n* Code is provided for reproduction. \n\n### Weaknesses\n\n* Since hyperbolic learning is new to the point cloud analysis community, more insights on the technical part should be given to assist the audience to comprehend why the proposed method could work as expected. I would point out what should be improved in the questions section.\n\n* The paper claims the proposed method can be applied to any existing architecture. However, the result section only shows it is applied to a small subset of methods that are compared. This makes me wonder if the results are cherry-picked. Can it provide a performance boost to arbitrary architecture?  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "LTPK8g1zaWK", "reviewer": "Reviewer_v4Kn", "summary": "This paper proposes to embed the features of a point cloud classifier into the hyperbolic space and explicitly regularize the space to account for the part-whole hierarchy. To do this, it employs a hyperbolic neural network [10] and introduces losses to regularize the part-whole hierarchy.\n", "questions": "(1) further discussion of the definition of the part-whole hierarchy and its difference to object-part hierarchy usually defined in literature.\n\n(2) comments on if $R_\\text{hier}$ is sufficient to regularize hierchy with multiple levels, and also supply the result of DGCNN+HyCoRe without  $R_\\text{hier}$\n\n(3) improve visualization of figure 4, give more visual example of the pointcloud & embedding pairs. \n\nSee discussions above for explanations.", "limitations": "I could not find a limitation section in the draft.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nIt seems the proposed use of hyperbolic neural network with regularization is able to improve accuracy of different point cloud classification backbones. The experiments demonstrated the benefit of the proposed work. \n\n\nWeakness:\n\nThe intuition of the proposed method looks handwavy to me. \n\nIn literature, when talking about part-whole hierarchy, it almost always refers to a single object consists of multiple subparts, and the subparts further dissolve into smaller parts. The parts need not to be the same and most times they are assumed to be a mixture of different parts. However, in this work, it is in an unusual opposite direction, i.e. different objects share a single “atom” part and the full objects are “grown” piece by piece in sequential order (as shown in figure 2). Such definition of the hierarchy looks suspicious to me, as there shouldn’t be an unique sequential order / hierarchy to define how an object instance is composed in piecewise order, and it makes less sense to require different objects correspond to a single “common part ancestor”. This is in contrast to embedding WordNet (in the original hyperbolic neural network paper) where the word hierarchy is uniquely defined according to categorical relationships. \n\nAlso, the way the author designed the contrastive and hierarchical loss (equation 5) is not fully justified by the author’s definition of part-whole hierarchy. The loss in equation 5 only enforces the embedding of the whole object to differ from the embedding of parts, in other words it only distinguishes the last level of hierarchy against the rest levels, but there is no regularization to distinguish subparts between intermediate levels.  Looking at the ablation in Table 4, it does seem that the hierarchical loss make little difference to baseline. Could the author also supply a version of DGCNN+HyCoRe without $R_\\text{hier}$? The visualization in Figure 4 is not that helpful either, it is hard to tell the size of each points when everything is densely packed, and not straightforward to see the quality of the learned “part-whold hierarchy” other than the clustering.\n\nOverall, I feel “hierarchy” may not be a good explanation to what the authors actually did. Probably just part-whole contrastive learning (without the hierarchical part) is more appropriate. \n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "B_58HVlxahL", "reviewer": "Reviewer_r1Np", "summary": "The paper proposed to utilize hyperbolic space to learn part-whole hierarchy for 3D point clouds. The main idea is regularizing in the hyperbolic space if part or whole are representing same object, and if they are from different objects, the regularization is pushing them to larger distance (similar to triplet loss, or contrastive loss). The paper applied the proposed regularization to many different network architecture (e.g. DGCNN, PointNet++, PointMLP, etc) and improved the baseline across two datasets (ModelNet40 and ScanObjectNN). ", "questions": "See Weakness section above.", "limitations": "See weakness section above. ", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\n1. Utilizing hyperbolic space for regularizing part-whole hierarchy is a new idea. The geodesic distance in the hyperbolic space naturally suitable for the tree structure of part-whole hierarchy and defining the regularization on the part-whole space makes more sense comparing with defining on the Euclidean space. (although I have some questions below)\n\n2. The proposed regularization is agnostic to different network architectures, and the  paper experimented on multiple  different backbone for point cloud classification, each achieved improvement over compared baselines on two dataset. \n\n3. The paper is well written and easy to understand. \n\nWeakness:\n\n1. (minor point) The definition of part-whole hierarchy seems controversial. In the paper, the part is defined as the ancestor of the whole object, and different object can share same ancestor. This is not intuitive, as one object is composed of multiple **different** parts, that means, if looking from the path from part to whole, it's not a tree structure  (one child node is strictly below one parent node). An intuitive way of defining part-whole hierarchy is the whole is the ancestor of the parts and parts can be share with many different objects. This is also how PartNet [a] (A dataset for part-whole hierarchy for 3D object) is created, and also the part-whole hierarchy mentioned in the seminal work [b]. \n\n2. Overclaiming. I don't agree with the author the paper is learning to \"promotes the part-whole hierarchy of compositionality in the hyperbolic space\" Specifically, the only thing the paper is proposing a regularization on the feature space, therefore how the object is composed with different part is not clear, there's no explicit way of representing object as a hierarchical tree using the proposed method, not mentioning the compositionality. Besides, the way the paper is defining \"parts\" is through subsampling a small local region of the object (Line 188), this is not a semantic part and is only a subgroup of the object, and also the definition of part-whole hierarchy is controversial as above\n\n3. The comparison is not a fair comparison. I appreciate the paper compared with multiple network architectures and see the improvement over compared baselines. However, since the main idea is proposing a regularization loss in the hyperbolic space, a fair comparison should be apply the same regularization loss, but in the Euclidean space. The compared results is only using supervised loss for pretraining and fine-tuning with the classification loss, what about using both losses for fine-tuning? It was mentioned in Line 95 that \"they are also mostly unable to improve upon state-of-the-art supervised methods when finetuned with full supervision\", but there's no evidence to support this. \n\n\n4. Since the paper is proposing to learning compositionality with part-whole hierarchy, a more convincing experiment would be running with a dataset that contains part-whole hierarchy (e.g. PartNet [a]), and provide a quantitative analysis on the learnt hierarchy v.s. other baselines that learns part whole hierarchy. \n\n\n5. (minor point) In the Fig. 4 visualization, it seems the clustering is not great? If we only focus on the light green color, they are spread in many regions in the graph, does this mean the clustering of light green class is not great?\n\n[a] PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding \nKaichun Mo, Shilin Zhu, Angel X. Chang3, Li Yi, Subarna Tripathi, Leonidas J. Guibas, Hao Su\n\n[b]How to represent part-whole hierarchies in a neural network\nGeoffrey Hinton\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept