SeLCA: Self-Supervised Learning of Canonical Axis
Editors: Sophia Sanborn, Christian Shewmake, Simone Azeglio, Arianna Di Bernardo, Nina Miolane
Abstract
Robustness to rotation is critical for point cloud understanding tasks, as point cloud features can be affected dramatically with respect to prevalent rotation changes.
In this work, we propose SeLCA, a novel self-supervised learning framework to predict the canonical axis of point clouds in a probabilistic manner.
In essence, we propose to learn rotationalequivariance by predicting the canonical axis of point clouds, and achieve rotationalinvariance by aligning the point clouds using their predicted canonical axis.
When integrated into a rotation-sensitive pipeline, SeLCA achieves competitive performances on the
ModelNet40 classification task under unseen rotations.
Most interestingly, our proposed method also shows high robustness to various real-world point cloud corruptions presented by the ModelNet40-C dataset, compared to the state-of-the-art rotation-invariant method.
1. Introduction
With the recent advances in deep learning, there have been successful attempts to reason about 3D point clouds (Wang et al., 2019; Wu et al., 2019).
However, presenting consistent inferences with respect to varying rotation transformations on point clouds remains a challenging problem.
Existing approaches to tackle this issue can be largely divided into a) methods which rely on rotation-equivariant or invariant networks (Rao et al., 2019; Poulenard et al., 2019) and b) methods which aim to achieve rotation invariance by aligning point clouds to their canonical pose using principal component analysis (Li et al., 2021a).
In this paper, we focus on a major drawback of existing methods that has received little attention - their brittleness to real-world corruptions e . g . occlusion and noise.
Existing methods either have a strong reliance on the plane symmetries or the intrinsic geometries of point clouds (Li et al., 2021a; Xiao and Wachs, 2021), which are largely affected by point cloud corruption, therefore having limited applicability to real-world settings.
Our contribution can be summarized as follows:
• We propose SeLCA, a novel self-supervised pipeline to predict the rotation-equivariant canonical axis of 3D point clouds, • We propose a novel alignment scheme on geodesic spherical tessellations to effectively represent the canonical orientation distribution of a point cloud, • We demonstrate the superior robustness of SeLCA against realistic point cloud corruptions of the ModelNet40-C dataset, while being competitive on the ModelNet dataset for point cloud classification.
[Figure]
2. Self-supervised Learning of Canonical Axis
It is impossible to provide strong supervision of canonical axes due to the absence of standard rules to pre-define the canonical axis of a point cloud.
To this end, we train our SeLCA pipeline to output a canonical axis distribution that satisfies the following property: when a rotation R is applied to the point cloud then the predicted canonical axis distribution should reflect the corresponding rotation by R , i .
e .
the canonical axis distribution should be equivariant to the rotation of point clouds.
Figure 1 illustrates the overall training pipeline.
2.1. Geodesic Spherical Tessellation: Regular Icosahedron
We take inspiration from the axis-angle representation of 3D rotation to represent our canonical axis.
As we want the output of our network to represent the canonical axis distribution, we need a discrete representation which can represent as many equispaced points on a unit sphere i.e., a geodesic spherical tessellation.
We therefore use a regular icosahedron to represent our canonical axis distribution, noting the icosahedral group is the largest finite subgroup of SO(3).
2.2. Self-supervised Training Pipeline
First, we randomly select a rotation matrix from the icosahedral group to transform a point cloud.
We feed the original point cloud and the rotated point cloud into our DGCNN feature extraction network to extract a 12-dimensional feature for each point cloud.
We apply softmax on this feature to yield a 12-bin histogram, where each bin represents a vertex
[Table]
on a regular icosahedron, and each bin value represents the probability of a vector from the origin to the vertex to be the canonical axis of the point cloud.
Note that we can map the selected rotation matrix (from the first step) to a permutation order of the histogram due to the closure of the icosahedral group.
Then, we permute the histogram from the original point cloud using this permutation order, then supervise the two histograms - the permuted histogram from the original point cloud, and the histogram from the rotated histogramto be close to one another, such that the canonical axis distribution of a point cloud is equivariant with respect to rotation.
Formally, our loss is defined as:
[Equation]
where p o and p R denote the output histogram bin of the original and rotated point clouds.
3. Experiments 3.1. Object classification
We evaluate our model on the ModelNet40 dataset (Wu et al., 2015) on the task of classification.
The results are presented in Table 1.
We can see that our method achieves competitive performances compared to most rotation-equivariant methods and rotation-invariant methods despite not using equivariance-guaranteed architectures or representations.
Also, we can see that the accuracy drop of our method from SO(3) / SO(3) to z/ SO(3) is smaller than the majority of rotation-equivariant methods, showing that leveraging our learned rotational equivariance to model rotational invariance is empirically effective.
We can also select up to top- k canonical axis candidates to perform test-time augmentation e .
g ., k =denotes using a soft ensemble of output results from using top- k argmax positions of the output canonical axis distribution.
We guide the readers to the appendix for elaborate implementation details and evaluation settings.
[Figure]
3.2. Classification on noisy real-world datasets
We evaluate our method on the ModelNet40-C dataset (Sun et al., 2022) on the task of classification against DGCNN (Wang et al., 2019) and Li et al.
(2021a), the current stateof-the-art PCA-based rotation-invariant method.
The ModelNet40-C dataset is based on the ModelNet40 validation set, but augmented with 15 corruption types at 5 severity levels.
We report the results for 6 types of corruptions in Figure 2.
We refer the readers to the appendix for the full results.
It can be seen that our method shows the highest performance overall under various types of point cloud corruptions, where our robustness is especially pronounced under background, cutout and density corruptions.
4. Discussion and Future Work
Albeit the rotational equivariance of our canonical axis distribution is not theoretically guaranteed as we do not rely on equivariant or invariant networks, the probabilistic nature of our approach enables our approach to be significantly more robust to various types of point cloud corruptions compared to existing rotation-equivariant and -invariant representations, while yielding competent results on synthetic datasets.
A promising future direction would be to build on our approach to improve the empirical rotational equivariance of the predicted canonical axis distribution.
Another limitation of our approach is that we only predict the canonical axis of a point cloud, but not the canonical angle around the predicted canonical axis.
Correctly predicting the canonical angle together with the canonical angle is expected to achieve a significant performance improvement, which we leave for future endeavors.
This work was supported by Samsung Electronics Co., Ltd. and the Institute of Information & Communications Technology Planning & Evaluation (IITP) grants (No.2021-0-02068: AI Innovation Hub (50%), No.2022-0-00290:
Visual Intelligence for Space-Time Understanding and Generation based on Multi-layered Visual Common Sense (40%), No.2019-0-01906: AI Graduate School Program at POSTECH (10%)) funded by Korea government (MSIT) Appendix A. Implementation Details
We employ DGCNN (Wang et al., 2019) as both our network structure to output the canonical pose distribution of a point cloud, and also as the base model for all downstream task experiments carried out in this paper.
At train time, we first train our network in a self-supervised manner as explained in Section 2, and then train the downstream network while leaving our orientation estimator to be frozen.
We add weight normalization (Salimans and Kingma, 2016) on the last layer during training.
We use RAdam optimizer and cosine annealing learning rate scheduler with the initial learning rate 0.0005.
The dropout ratio of DGCNN is set to 0.5 and the weight decay is set to 0.0001.
We use centering (Caron et al., 2021) on each histogram bin.
We randomly sample 1024 points from 2048 points of point clouds for each iteration.
We follow the original training settings of DGCNN when training for downstream networks except for setting the initial learning rate to 0.01.
For our experiments, we follow existing work (Li et al., 2021a) to compare different methods in 3 different ways: (1) z/z : both the train and test sets are rotated only around the z
-axis, (2) z/ SO(3): the train set is rotated only around the z -axis and the test set is randomly rotated, and (3) SO(3) /
SO(3): both the train and test sets are randomly rotated.
Appendix B. Geometry-aware Icosahedral Permutation
We go over the details of the geometry-aware icosahedral permutation, which is applied on the output of our ATP method in correspondence to the rotation applied on the point cloud, in order to provide self-supervision.
The main idea here is that each histogram bin i . e ., entry in our normalized output, corresponds to a single vertex on the icosahedron.
Therefore, applying a rotation sampled from the icosahedral rotational symmetry group to the icosahedron results in a permutation of the vertices i .
e ., the histogram bins.
This guarantees the rotated coordinates ˆ v are a permutation of v .
For further details, refer to
Algo 1.
Algorithm
Original Histogram Bin: { h i } 12 i =
Coordinates of Icosahedron: { v i } 12 i =
Rotation Matrix R
Output : Permuted Histogram ˆ h for i = { 1 , 2 , · · · , 12 } do ˆ v i
=
Rv i j ∗ =
argmax 12 j =
|| ˆ v
i − v j ||ˆ h i =
h
j ∗ end for
Appendix C. Full Results on the ModelNet40-C Dataset
In this section, we present the full set of results on the ModelNet40-C dataset, which accounts for a total of 3 settings ( z/z , z/ SO(3) and SO(3) / SO(3)), 15 corruption types and 5 severity levels.
[Figure]
We report the classification results of ours (green circle), Li et al.
(2021a)
(blue cross), and DGCNN (Wang et al., 2019) (orange square) with 15 different corruptions and five severity levels increasing from left to right.
Under most corruptions,
Li et al.
(2021a) shows lower performances than other methods.
Background
Cutout
Density_inc
Distortion
Distortion_rbf
Distortion_rbf_inv
Gaussian
Impulse
Lidar
Occlusion
Rotation
Shear
Unifrom
Upsampling
Density
We report the classification results of ours (green circle), Li et al.
(2021a) (blue cross), and DGCNN (Wang et al., 2019) (orange square) withdifferent corruptions and five severity levels increasing from left to right.
In most corruptions, DGCNN shows much lower classification accuracy than the others.
Ours shows the best performance overall.
[-Background 
-]
[-Cutout 
-]
[-Density 
-]
[Figure]
[-Gaussian 
-]
[-Impulse 
-]
[Figure]