Learning Generalizable Part-based Feature Representation for 3D Point Clouds
Authors: Xin Wei
Xiang Gu
Jian Sun
Venue: NeurIPS 2022 Accept (2022)
Keywords: Point cloud classification
domain generalization
generalizable part-based representation

Abstract: Deep networks on 3D point clouds have achieved remarkable success in 3D classification, while they are vulnerable to geometry variations caused by inconsistent data acquisition procedures. This results in a challenging 3D domain generalization (3DDG) problem, that is to generalize a model trained on source domain to an unseen target domain. Based on the observation that local geometric structures are more generalizable than the whole shape, we propose to reduce the geometry shift by a generalizable part-based feature representation and design a novel part-based domain generalization network (PDG) for 3D point cloud classification. Specifically, we build a part-template feature space shared by source and target domains. Shapes from distinct domains are first organized to part-level features and then represented by part-template features. The transformed part-level features, dubbed aligned part-based representations, are then aggregated by a part-based feature aggregation module. To improve the robustness of the part-based representations, we further propose a contrastive learning framework upon part-based shape representation. Experiments and ablation studies on 3DDA and 3DDG benchmarks justify the efficacy of the proposed approach for domain generalization, compared with the previous state-of-the-art methods. Our code will be available on http://github.com/weixmath/PDG.

Learning Generalizable Part-based Feature Representation for 3D Point Clouds
Xin Wei, Xiang Gu, Jian Sun (23)
School of Mathematics and Statistics, Xi' an Jiaotong University, P. R. China {uxmath, xianggu)Ostu. htt.:sssu...tt....... xjtu.edu. edu. jiansunoxjtun eew.jtttedddaa edu.cn cn
Abstract
Deep networks on 3D point clouds have achieved remarkable success in 3D classi- fication, while they are vulnerable geometry variations caused by inconsistent data acquisition procedures. This results in challenging 3D domain generalization (3DDG) problem, that to generalize model trained on source domain to an unseen target domain. Based on the observation that local geometric structures are more generalizable than the whole shape, we propose reduce the geome- try shift by generalizable part-based feature representation and design novel part-based domain generalization network (PDG) for 3D point cloud classification. Specifically, we build part-template feature space shared by source and target domains. Shapes from distinct domains are first organized to part-level features and then represented by part-template features. The transformed part-level features, dubbed aligned part-based representations, are then aggregated by part-based feature aggregation module. To improve the robustness of the part-based represen- tations, we further propose a contrastive learning framework upon part-based shape representation. Experiments and ablation studies 3DDA and 3DDG bench- marks justify the efficacy of the proposed approach for domain generalization, compared with the previous state-of-the-ar methods. Our code will be available on http:// //g thub......... com/ h/s//exxiee homm//exiaat////mmmmmmmmmmmm xmath/mPas
Introduction
The 3D shape understanding and reasoning play a critical role in wide applications such as automatic drive, archacology, virtual reality augmented reality, etc. Point cloud is popular representation of 3D shape attributed to its simpleness and effectiveness. Recently, with the thriving of deep learning, numerous deep architectures [1-10] have been proposed for 3D point cloud analysis. Although these methods have achieved impressive results on the 3D shape classification task, all of them strongly rely on the i.i.d assumption on the source and target data but ignore the nut-of-distributen situation in real-world practice. For example, the performance of these methods drops dramatically when trained on CAD datasets [11, 12] and tested on real scanned datasets [13, 14]. Therefore, it is crucial and valuable to investigate how to learn a 3D classification model that generalizes well on a related test domain with domain shift to the training domain [15]. Domain shift refers to the existence of significant divergence between the distributions of the training and test datasets [15]. The domain
shift may degrade the performance of network trained on a training dataset when generalizing to the
test dataset. Domain generalization methods for improving domain robustness of deep network have
been extensively investigated for 2D images [16-21]. However, little work [22, 23] addresses the domain generalization (DG) problem for point cloud deep networks. For point clouds, the major cause of domain shift is the geometry variations generated by inconsistent data collection processes, e.g., the realistic sensor noises, the non- non-uniform density of point clouds, and self-occlusion. These domain shifts specific for 3D point clouds hinder the idea that directly adopts 2D image-oriented
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

DG methods to point clouds, and this inspires us to design a 3D domain generalization model for addressing the domain shifts of point clouds.
To improve the model robustness to geometry variations, straightforward way is to simulate geometry shift dur- ing training. MetaSets [22] designs three point transformation tasks of sim- ulating occlusions, missing parts and changes in scanning density, which have been proven to effective as data augmentations. They further learn fea- ture representations by meta-learning using these tasks. Instead of improv- ing model's generalization ability using globally pooled point cloud features, in this paper, we tackle the domain gener- alization problem for 3D point clouds based on our observation that the local geometric structures of point clouds are more likely to be shared across distinct domains, therefore being more generalizable to the geometry variations.
Source data
Part-template space
Target data
Figure 1: The 3D point clouds in source domain (i.e., source data on the left) and target domain (i.e., target data having, missing points on the right) can be both recognized as "chair" because they can be represented by the shared parts in the part-template space of "chair" category.
Parsing objects into parts is crucial for humans to understand the world. For the task of object recognition, the visual system of people decomposes shapes into parts and recognizes objects based on the description and spatial relations of parts [24]. It is hard for machines to understand the chair on the right of Fig. when it overfits on a set of chairs similar to the left one in Fig. 1. However, humans can easily interpret them by decomposing two chairs and inferring their labels from some similar parts (e.g., legs and a plain), even when having broken parts of the right chair in Fig. 1. This motivation seems to be intuitive, and we will provide more experimental evidence in Sect. 2. Based on this observation, we are interested in learning generalizable representation of 3D point clouds at the part-level instead of the global level-
Along this idea, in this work, we propose a novel part-based domain generalization network for 3D point cloud classification. We build a part-template feature space that is shared to source and target domains. Shapes from distinct domains are first organized as part-level features and then aligned to part-template features by cross attention mechanism. Aligned part-based features are then aggregated by a part-based feature aggregation module for each point cloud. To improve the robustness of part-based representation, we further propose contrastive learning framework to enforce that the feature representations a point cloud under different transformations are consistent in part-level and seape-level. Extensive experiments conducted on 3DDA [25] and 3DDG [22] benchmarks demonstrate the effectiveness of our approach, and our method outperforms the compared methods by notable margin.
Our contribution can be summarized in three folds. First, we empirically observe that the geometry shift induced domain gap of point clouds could be reduced by part-level representation, thus we propose to learn past-based 3D feature representation improve the generalization ability of point cloud classification models. Second, we propose novel part-based domain generalization network for 3D point cloud classification. contrastive learning framework upon part-based shape representation is further designed to improve the robustness of learned representations. Third, our method achieves the best performance by comparisons on 3D benchmarks for domain generalization.
2 Reducing Geometry Shift by Part-based Feature Representation
Problem definition. Let X' and y be input and label spaces. A domain is defined by (D,gr) where D is probability distribution on X' and g2 : X - 3 a function mapping input to its ground-truth label. The objective of domain generalization is train classifier F on the source domain D that predicts well on target domain DT when the target samples are not available during training. In general, assume the label space is shared by source and target domains.
In 3D domain generalization, the input is a point cloud P E RNNx3 where N the number of points. We consider point cloud classification model J composed of feature extractor fo

---

100
2.0
Shape-level
12-points-part-leeee
256-points-parte
90
Shape-level
12-points-part-llee
80
256 356ppinns-per pert--eeeeeee
Distacee
70
Accuracy
60
1.0
50
40
30
0.3
20
bed
cabinet
display
shelf
toilet
Target-test
A-distance
(h) Discrimination
Figure 2: (a) A-distance between source and target domain features (shape-level, 5 $12-points-part level, 256-points (o--ponts-part-levell for each class. (b) Discrimination ability of shape-level features and part-level features measured by classification accuracy on source and target domains.
P Z and a classifier Co FF + RC, where is global representation of P. and C' is the number of classes. The final prediction is given by ú softmax CC (pooling(fo(P))
RNxd is point-wise features, F Rd
where the pooling operation is determined by F. The parameters (0, vi) are optimized with respect to the cross entropy loss Cly.9) = = - Directly minimizing the cross entropy loss C will produce discriminative network the source domain, however, it may overfit the source domain and degeneralize on the target domain.
log(
Comparison of generalization and discrimination abilities of shape-level and part-level features. To understand shape, people may process it in various granularities. The fine- gra-ggained parts represent local geometric structures while coarsen parts contain more global semantic information. Thus global shape-level features may be more discriminative than part-level features. However, this advantage drops sharply when encountering large distribution discrepancy between training and test domains for shape-level features. On the contrary, part-level features encode the local geometric structures which are shared across different shapes in distinct distributions, while they are short of semantic information. We next experimentally verify the above analysis. We use A-distance as a measure to evaluate distribution discrepancy [26, 27J. It is defined dist 2(1 2e), where is the test error of classifier trained to discriminate the source and the target domain data. We train a PointNet [5] on source domains A/ (ModelNet dataset [12]) and test on target domain SO (ScanObjectNN [14]). We first get point-wise features Z E R xd for each shape that has N points. Then point-wise features are max- max-pooled on all points to produce a global shape-level feature Z We also split each shape into some overlapped parts in distinct scales, i.e., 16 parts (256 points in each part) and 8 parts (512 points in each part) and max- mox-pooled on each part to get pert- features {2?"), Ji= and [ Fig. 2 (a) shows the A-distance on each class of M + SO with shape-level features Z and part-level features {2 smaller than dist of shape-level features and dist decreases as the size of parts becomes smaller, indicating that part-level features are able to better reduce domain gap. We also train linear SVM on source domain training data and test on source domain test data and target domain test data to evaluate the discrimination abilities. For part-level features, the prediction of a shape is by voting the predictions parts. As shown in Fig. 2 (b), the discrimination abilities of features are weakened when the scale decreases. Though part-level features could reduce the geometry shift between source and target domains, aggregating them simply by voting could not improve the discrimination ability.
1i=1 and { 1==1- We observe that dist of part-level features is
Based on the above observations, we are inspired to represent point cloud by part-level features to reduce the domain discrepancy. We build part-template feature space that is shared with source and target domains. Part-level features in distinct domains are aligned to the part-template features, resulting in part-based feature representations with better generalization. To improve the discrimination of the part-level features, the features of parts fused with a part-based feature aggregation module to achieve a global representation for each point cloud.
3 Learning Part-based Representation of Point Clouds
We first introduce our proposed part-based representation of point clouds, taken as the main operation of our bart- domain generalization network for 3D point cloud classification presented in Sect. 4.
3

---

Part-level
Aligned Part-
Global Representation
Features
based Features
Source
F6i
Z
Fin
ps
Part-level Feature
Feature Encoder
4
Aggregation
Part-template Features
f1
Hau
Target data
2/.
FT
Figure 3: Illustration of our part-based feature representation for point clouds. Given a point cloud P RN from source or target domain, it is first processed by feature encoder and organized to part-level features Zq = (Zq, M
== E R.Mxd Then part-level features are transformed to aligned part-based features FQ (Fq. ) by aligning them to part-template features // {H Nu Then they are aggregated global representation F by the part-based feature aggregation module.
The major objective of this representation is to transform part-level features of shapes distinet domains into common space, i.e., part-template feature space. All these part-level features aligned to set of learnable part-template features in the part-template feature space. The transformed features are dubbed as the aligned part-based representations of point clouds.
As shown in Fig 3, taking two point clouds from source and target domains, they are first organized part-level features, and then transformed to aligned part-based representations in part-template feature space by cross-attention. In order aggregate the aligned part-based representations representation, we further propose part-based feature aggregation module to aggregate features according to their importance.
global
3.1 From point clouds to part-level features
Given point cloud P {I1, D2, ..., R xd are first extracted by feature extractor fo. ie., Z = (e(P; We represent point cloud by union of M overlapped parts P Q UQ2U...QQM where each part Q; { ...,, Tia E kx3 defined as center point T3 with & nearest neighbor points { ..., Liah J. We use farthest point sampling (FPS) to sample M center points for constructing M parts since FPS has better coverage of
ENN NN3 point-wise features Z {21, 22 .... 2N
the entire point cloud. For each part Q,, corresponding part-level feature Zo, E Rd is derived by maxpooling point-wise features of part Q;:
Zo, maxpooling (.. zik).
(1)
3.2 Representing part-level features using part-template features
As discussed in Sect. 2, part-level features reflect the local geometric structures of 3D shapes, which could reduce geometry shifts more effectively. Thus are inspired to use this general information improve the generalization ability of the classification model. Given part-level features in the source domain, part-level geometry priors, modeled part-template features, extracted from the source domain and provide references for the target domain. Specifically, we could construct a part-template space H, where H = {H.), N E RN is a set of part-template features. H serves as a basis set and each basis vector could encode a local geometric prior. Our proposed part-template features similar to the dictionary of local geometric priors and each input part-level feature is represented as combination of them. common choice get dictionary is to apply k-means clustering on part-level features of the entire training data in each training step. However, the dictionary computed by k-means can only reflect local geometric prior in the current step. Instead, we propose to use set of learnable part-template features to represent various shapes in the whole training process, which can be updated automatically by gradient propagation. Leveraging this simple yet effective representation, could capture rich part-level geometry priors.

---

Given part-level features 2 = 1Z R and part-template features H H E
RN xd we want to find a feature transformation T : RNxxd + RMxx for representing part-level features with the part-template features, defined as
Fg. - T(H); = -T.,,.
(2)
where T RMxNN is linear transform map implementing T and Fg - (Fe: ) is transformed features, dubbed aligned part-based features. Briefly speaking, each aligned part-based feature is linear combination of part-template features, and the combination weight is decided by the relation between part-level features and part-template features.
Recently, some researchers [28-31] have adopted cross-attention align features from different modalities. [32, 33] further apply cross attention module to solve the unsupervised domain adaptation problem based on its great power in feature alignment and robustness to noises. In our case, we use cross-attention module to learn dependencies between part-level features and part-template features. The transformation map is defined as
T softmax ZOHT
where the softmax function is applied on the rows of the scaled similarity matrix. The i-th row of T normalized similarity vector between part-level feature Zo. and all the part-template features {H:) Part-template features which are more similar to part-level feature Zg, will contribute more to the combination, while the less similar part-template features make little impact.
Optimal transport [34] is another possible approach to distribution transport and alignment. We can use optimal transport to learn T for transporting the part-template features such that the transported feature distribution is the same as that part-level feature distribution. We conduct the experiment that PDG with OT for feature alignment in the ablation study. In this work, we employ cross-attention for aligning part-level features part-template features because of its better performance and lower computation cost.
Once the T is derived, the corresponding aligned part-based features are derived by Fq TH. For each part-level feature Zo., we can observe that the aligned part-based feature Fq. is a weighted sum of part-template features [
Nn
3.3 Part-level features aggregation
After getting set of aligned part-based features, a global representation is often obtained by pooling these features. However, various parts of shape may have different contributions for shape recognition. Given the part-level representation FQ = (Fq, 1M define the score of each part as
= (ax.
(4)
for i ..., M. S(-) is part score module with parameters Op,, outputting the vector of prob- abilities of part belonging to C shape classes, and max operation returns the maximal value of this vector. (.) (.) is implemented as the same structure as classifier C+)(). In a word, the score s; indicates the confidence score for classification of part Fq.. Then each part is weighted by this part score, and the global representation of shape derived by
F = maxpooling(s1 FQ.....,MM FQu
(5)
4 Contrastive Learning Upon Part-based Shape Representation
We build a contrastive learning framework upon part-based shape representation to improve the robustness of the learned aligned part-based features. The main idea is to maximize the agreement between same shape under different augmentations in part-level and shape-level via contrastive loss. We randomly sample mini-batch of N point clouds {Pk J and the contrastive learning task is defined on 2NV point clouds which are the pairs of samples under different augmentations. We use three point cloud transformations including non-uniform density, dropping, and self-occlusion. For more details, please refer to [22]. For each point cloud Pk. denote Ps as the same point cloud with
5

---

another augmentation. Augmented shapes are encoded to derive the aligned part-based representation Fo E R2N x xx and global representation E R2Nxd Following [35]. they are further processed by a projection head Proj(.). resulting in Fo and F. Proj(.) is implemented as a two-layer MLPs with 4d hidden dimensions. We adopt the contrastive loss as the form of InfoNCE [36], which learns representation by attracting positive samples and dispelling negative samples. The shape-level contrastive loss is defined supervised contrastive loss [37]:
2N
Eez exp(sim FT)/T)
Shape
log
(6)
EB exp(sim F + Σr. exp(sim(Fx FF)/7)*
where sim(:;)) denotes the cosine similarity of two inputs, and is temperature hyper-parameterr empirically set to 0.07. Fx is the anchor, F denotes F positively paired sample (excluding anchor itself) with the same label as g. and the negative sample F is with different label to Fx. By this shape-level contrastive loss, the global representation of same class under different transformations will cluster together while those from different classes will be pushed apart.
Part-level contrastive loss Cpar: is defined in self-supervise form:
2NN
M
exp(sim(Fqq FQT.)/T)
part ΣΣun log
Eeplsim(Fq... Fe., )/T + IFaa.. exp(siml Pou.: D/T.
For the i-th part of point cloud Pk. whose aligned part-based feature is FQ..., the positive sample 'Q... is the aligned part-based feature of i-th part of Px. The positive part has the same center point as the anchor part. The negative samples are the other aligned part-based features in this batch. Part-level contrastive loss Le encourages learned aligned part-based feature of part of shape under different local transformations be consistent the feature space.
By the contrastive tasks in both part-level and shape-level, aligned part-based representations are robust varying local point transformations and the combination of these aligned part-based repre- sentations could be discriminative for shape classification.
4.1 Overall loss
The overall training loss consists of shape loss Lshape part loss Cpart, shape-level contrastive loss LC shaper and part-level contrastive loss L'rre-- Given global shape representation F, it is sent classifier Cs followed by softmax layer. The total training loss is
M
= (F), Xp (par
Ac + Epart):
where y is the class label and Cshape is a cross entropy loss based on global shape representation F. Cpart is the cross-entropy loss defined on aligned part-based feature S(FQ.). Lp enforces the part score module S(-) could discriminate the shape category based on the aligned part-based features.
Experiments
5.1 Datasets
Sim-to-Real [22]. Real-to-Sim [22] is 3DDG benchmark consisting of three domains:
ModelNet [12], ShapeNet [11] and ScanObjectNN [14]. (a) ModelNet (M) contains 12,311 clean 3D CAD models from 40 categories, with 9,483 training models and 2,468 test models. (b) ShapeNet (S) contains 51,162 3D models categorized into 55 classes and its objects have
larger structure variances compared with ModelNet. (c) ScanObjectNN (SO) is recently proposed real-world 3D object classification dataset with scanned indoor scene data. contains 2,902 object instances from 15 categories. ScanObjectNN offers more practical challenges including background occurrence, object partiality, and different deformation variants. Two versions of ScanObjectNN, respectively with background (ScanObjectNN-BG, denoted as

---

SOB) and with hardest perturbations (ScanObjecNNN-S50.RS, denotes as SOH), are also utilized for testing. Then six synthetic-to-real point cloud domain generalization tasks are built. M- + SO, M + SOB, and M + SOH refer to training network on ModelNet and test on ScanOhjectNN, ScanObjectNN-BG. and ScanOhjeetNN-T50-RR where they have shared classes, while S + SO, S - S0g and S - SOH are defined as generalizing network from ShapeNet to ScanObjectNN, ScanObjeetNN- -BG and ScanObjecNNN-TS0-RS within shared classes.
We follow the data preparation and experiment setting in [22]. Specifically, we use the official training and test split strategy for each dataset. Each point cloud from three domains contains 2,048 points and is normalized within a unit ball. We report two versions of results, i.e., "Best" that is the best achievable results among the epochs in training a network, while "Last five" indicating the average test results of the last five epochs of the training procedure. Note that the reported "Best" is unreasonable since the labels of test data are inaccessible, but it provides an upper-bound of performance for domain generalization approach. Thus we mainly report the "Last five" as the comparison measure in this paper. We calculate the average of "Last five" of four DG tasks to evaluate the performance of methods. We conduct each experiment three times and report the average top-1 classification accuracy in tables.
PointDA [25]. PointDA [25] dataset is a widely used point cloud domain adaptation benchmark, which collects shapes of 10 shared classes from ModelNet [12] (M), ShapeNet (S), and Scan- Net [13] (S*). Six point cloud domain adaptation tasks including M -> S, M - S" S + M, S - S* 5t M, and S* + S are built upon PointDA [25]. We follow the data preparation, dataset splitting, experiment setting in [25]. Each point cloud in PointDA [25] has 1,024 points. We implement our method on this benchmark under the setting of domain generalization.
5.2 Compared methods
For Sim-to-Real, we first compare our method with three state-of-the-at point classification methods including PointNet [5], DGCNN [6] and PointMLP [4]. We find that these methods achieve better results than the results reported in [22], if a random jittering [5] is performed on the training data which is widely used in 3D classification model training. We further compare our method with state- of-the-art 3DDG method MetaSets [22]. We report their results by directly running their published codes of http:/// gighh........ coom/thul///ee http//thum//Metttt..
For PointDA [25], we compare PDG with current 3D domain adaptation methods which use target domain data in training procedure, including DANN [38], PointDAN [25], RS [39], DefRec + PCM [40], GAST [41] and GLRV [42]. We also compare it with state-of-the-art 3DDG method, iee.
MetaSets [22], in which target domain data is inaccessible during training.
5.3
Implementation details
For our PDG, we adopt PointNet and DGCNN as backbones of feature extractor fo and classifier Gg. For fair comparison, we do not change the architecture of backbone and train all methods except MetaSets for 160 epochs with batch-size 32 on one NVIDIA V100 GPU. MetaSets is trained for 200 epochs with batch-size 32 on two NVIDIA V100 GPUs. We use Adam as the optimizer. The initial learning rate and weight decay are 10-3, 10-4, The learning rate reduced to 10 following a cosine quarter-cycle. The part number M, points number k in each part, and the number of part-template features Nu are respectively set to 8, 512, 384. Xp and Ae in training loss are 0.05 and 0.01.
5.4 Experimental Results
Comparison with single domain state-of-the-art methods. As shown in Table PDG with DGCNN as backbone outperforms previous state-of-the-ar 3D classification methods by more than 5.5% in average accuracy of four tasks. PointMLP [4] is a powerful method and achieves state-of-the-art results on ModelNet40 (94.1%) and ScanObjectNN (83.9%) for single domain point cloud classification, as shown their paper. Compared with it, our PDG (DGCNN) shows better generalization ability and outperforms them by 5.5%. Compared with PointNet [5] and DGCNN [6], PDG with PointNet and DGCNN as backbone networks surpass them on every 3DDG task, improving average accuracy by 4.2% and 6.2% respectively. We also find that DGCNN [6]
7

---

Table 1:
Test accuracy (in %) on Real-to-Sine dataset. We report the "Last five" of each methods as main results for comparison. "Best" of each methods are also presented in bracket.
Method
M + S0
M S0p
M + SO#
S
SO
- S0B S SO#
Avg
PointNet [5]
59.8 (61.5)
51.5 (53.8)
50.0 (519)
55.9 (57.4) 51.0 (54.0)
49.0 (50.9)
52,9 (54.9)
DGCNN [6]
60.2 (62.3)
54.7 (58.5)
49.2 (51.1)
54.9 (59.0) 51.9 (54.3)
47.2 (54.0)
53.0 (56.5)
PointMLPP [4]
59.0 (62.5)
47.9 (59.4)
53.7 (57.3) 55.3 (56.8)
47.4 (50.9)
53.7 (56.3)
MetaSets (P) [22]
60.3 (66.3)
52.4 (57.5)
47.4 (54.4)
(55.3) 44.3 (50.3)
45.6 (50.0)
50.3 (55.6)
MetaSets (D) [22]
58.4 (64.2)
59.3 (60.6)
48.3 (53.1)
49.8 (60.3)
47.4 (57.8)
42.7 (50.8)
51.0 (57.8)
PDG (P)
67.6 (69.4)
58.5 (61.1)
56.6 (57.2)
57.3 (61.8)
51.3 (55.5)
51.3 (53.9)
57.1 (59.8)
PDG (D)
65.3 (68.8)
65.4 (68.0)
55.2 (58.0)
59.1 (64.3) 59.3 (64.3)
51.0 (56.6) 59.2 (63.3)
Table 2: Classification accuracy (in %) of various 3DDA and 3DDG methods on PointDA-10 10 dataset. Results of methods which do not use target data during training are in bold.
Method
DA/DG M
M
S
S +
S -
S
Average
Supervised
93.9
78.4
96.2
78.4
96.2
93.9
89.5
w/o Adaptation
83.3
43.8
75.5
42.5
63.8
64.2
62.2
DANN [38]
74.8
57.5
50.9
43.7
56.8
PointDAN [25]
83.9
44.8
63.3
45.7
43.6
56.4
56.3
RS [39]
79.9
46.7
75.2
51.4
71.8
71.2
66.0
DA
DefRec PCM [40]
81.7
51.8
78.6
54.5
73.7
71.1
68.6
GAST [41]
84.8
59.8
80.8
56.7
81.1
74.9
73.0
GLRV [42]
85.4
60.4
57.7
77.8
76.2
72.7
MetaSets [22]
86.0
67.3
69.8
69.5
64.5
DG
PDG (Ours)
85.6
57.9
73.1
50.0
70.3
66.3
67.2
outperforms PointNet [5] by only 0.1% while PDG (DGCNN) outperforms PDG (PointNet) by 2.1%. This demonstrates that, given the part-level features better encoding local geometric structures, our aligned part-based representations could further improve the robustness and discrimination ability. These significant performance gains (4.2% and 6.2%) of PDG over the baseline prove the effectiveness of our designed part-based domain generalization network, which improves both the generalization and discrimination abilities of point cloud classification models.
Comparison with 3DDA and 3DDG methods. For Sim-to-Real [22], MetaSets [22] state- of-the-art 3D domain generalization model which meta-learns point cloud representations from some point transformation tasks. As shown in Table 1, with the same backbone network PointNet and data augmentation, PDG outperforms MetaSets by 6.8%. When equipped with DGCNN, the improvement 8.2%. In contrast with MetaSets which aims improve the generalizable ability of shape-level features, our proposed PDG reduces geometry shift more effectively by the part-based feature representation.
For PointDA [25], from Table 2, PDG improves the baseline methods by 5.0% in the average accuracy of all tasks and outperforms 3DDG method Metasets [22] by 2.7%. Specifically, PDG beats MetaSets in four tasks M + S* S - M, S -> S*, and S* - M. For two synthetic-to-re tasks M -> S* and S S* the improvements 5.6% and 7.9%. We also find that PDG exceeds some 3DDA methods including DANN [38], PointDAN [25], and RS [39].
5.5 Ablation study
In this section, we take closer look at the effects of components of our network, including cross- attention module for feature alignment, part-level aggregation module and contrastive learning framework upon past-based shape representation. We choose PointNet [5] backbone network referred as "bascline" and experiments are conducted on - SO task on Real-to-Sim [22] dataset. We also investigate the effects of the scale of the parts and visualize the both part-level features and shape-level features of the baseline and our PDG. Results various architectures of PDG are presented in Table 3.

---

(a) Shape-level (PointNet) (b) Shape-level (PDG) (c) Part-level (PointNet)
(d) Part-level (PDG)
Figure 4: (a)()) The t-SNE [43] visualizations of shape-level features of PointNet and PDG (PointNet as backbone). (c)(d) The part-level features of PointNet, and aligned part-based features of PDG (PointNet as backbone). In each figure, the features of "chair" and "sofa" are denoted "*" and +" respectively, while data from source (Md) and target (S*) are in blue and red.
Effectiveness of each component. We first replace cross-attention module with optimal transport to find the transformation map T for feature alignment as discussed in Sect. 3.2. Table shows the results of PDG employing optimal transport for feature align- ment, which is denoted as PDG-OT. Compared with PDG-OT, PDG with cross-attention improves the per- formance by 2.4%. As mentioned in Sect. 3.3, we design part-based feature aggregation module to ag- gregate the aligned part-based features according to their confidence score for classification. We compare it with two baseline pooling operations, i.e. max-pooling and average- serrageppooling. As shown in Table 1, PDG with part-level aggregation improves PDG-max and PDG-
Table 3: Results (in %) of variants of PDG for shape classification on ModelNet to ScanobjectNN with different architectures.
Method
Last five
Best
baseline
59.8
61.5
PDG (w/o CL)
66.4
68.1
PDG (w/o SCL)
67.1
69.3
PDG-max
65.3
67.8
PDG-avg
65.4
69.0
PDG-OT
65.2
65.7
PDG-
65.4
67.1
PDG
67.6
69.4
avg by more than 2.2%. This shows that our part-based feature aggregation module is superior way aggregate the aligned part-based features and brings significant performance boost to our network. We finally examine the effects of the contrastive learning framework defined in Sect. 4. As shown in Table 3, with the contrastive learning framework,
method achieves better results than PDG (w/o CL) with 1. 1.2% improvement. We also remove the shape-level contrastive loss in PDG, denoted as PDG (w/o SCL), which performs slightly worse than PDG by 0.5% (0.1%), while still outperforming baseline by 7.3% (7.8%). This proves the effectiveness of our contrastive learning framework to improve the robustness of the aligned part-base based representations and the major performance gain of PDG is derived from the design of part-based feature representation.
Selection the scale of part. As discussed Sect. the scale of the part-level features will influence the generalization and discrimination abilities. Specifically, the discrimination ability of part-level features decreases when the scale reduces, while the generalization ability improves. We show the results of PDG with 8 parts (512 points in each part) and 16 parts (256 points in each part) in Table 3, PDG with 8 parts (i.e.. PDG Table 3) outperforms PDG- 166arss by 2.2%. Thus we choose parts with 512 points in each part for better balance of discrimination and generalization abilities.
Feature visualization and analysis. We first visualize the learned shape-level and part-level features of PointNet [5] of class "chair" and "sofa" from source domain (M) and target domain (S*) in Fig. We can observe that Fig. 4 (a), the shape-level features in source domain display high inter-class variance, while the features of "sofa" in target domain are closer to the features of "chair" in source domain due to the geometry shift between two domain data. This implies better discrimination ability of shape-level representation in source domain while worse in target domain. Conversely, the pert-level features of distinct domains and classes cluster together shown in Fig. (c). confirming our idea that the local geometric structures represented by part-level features are shared across different domains and classes and could reduce geometry shift effectively. However, the less discriminative part-level features are hard to be aggregated into a global representation for classification. Our PGD solves this problem effectively as shown in Fig. (b) Our aligned part-based representations in Fig. 4 (d) improve the discrimination ability of part-level features while

---

keeping the generalization ability. And in Fig. 4 (b), we observe both lower inter-domain variance and higher inter-class variance in the global representations of PDG compared with shape-level features of PointNet in Fig. 4 (a), resulting in the improvement in classification accuracy (67.6% vs. 59.8%).
6 Related Work
Deep classification on point clouds. The early work PointNet [5] processes points independently using MLP and aggregates them by max-pooling to obtain the permutation invariant representation, ignoring the vital local geometric structure of point clouds. DGCNN [6] models point clouds as dynamical graphs and proposes dynamic edge convolution for classification of point clouds. Succeeding methods [1-4, 6-10] achieve improved performance for 3D shape recognition. However, all these methods assume that the training and test data are from the same domain thus can be weak at generalizing learned representations to new domains. A method that could improve the domain generalization ability of point cloud representation desirable.
Domain Generalization and Adaptation for Point Cloud classification. How to represent 3D point clouds in different domains is an old but unsolved problem. [44] is one of the first methods to explore the 3D domain adaptation problem, which leverages synthetic scans of 3D scenes from Google 3D Warchouse to train object detection system for 3D point clouds real-scan data. [45] and [46] applied hough transform to the problem of robust 3D shape feature learning and evaluated on point cloud classification and retrieval tasks of 3D domain generalization problem. [47] factored low-dimensional deformations and pose variations of the 3d shapes and recognized them in the scanned cluttered indoor scene, which is also a 3D domain generalization problem. [48] focused on the CAD-to-scan retrieval task of the 3D domain generalization problem and introduced a method called CAD-Deform obtain accurate CAD-to-scan fits by non-rigidly deforming retrieved CAD models. Recently, domain adaptation or generalization problem on deep point cloud classification network [22, 25, 40-42, 49-52] has drawn more attention. PointDAN [25] proposes a 3D domain adaptation network for point cloud classification by minimizing an MMD loss to align local features across domains. [40-42, 49-52] further improve the results by designing a series of self-training tasks. However, all these methods are designed for 3D domain adaptation with known target domain data distribution. MetaSets [22] explores 3D domain generalization problem that the target domain data is inaccessible when training, and proposes to meta-learn point cloud representations from a group of point cloud transformations. Compared with them, we tackle the domain generalization problem by reducing geometry shifts using aligned part-based representation. Experiments on 3DDA and 3DDG benchmarks show that our PDG outperforms Metasets method by a large margin.
Contrastive self-supervised learning. Contrastive learning [53] which learns representation by contrasting positive pairs and negative pairs has achieved increasingly success for self-supervised learning [35, 54-59]. The learned representation could capture information shared between different augmentations of the same input. In this paper, our motivation that aligned part-based representation should be similar under different local point transformations. To this end, we propose a contrastive learning framework upon part-based shape representation and improve the robustness of aligned part-based representation effectively.
7 Conclusion
In this work, we propose a novel part-based domain generalization network for 3D point cloud classification. We propose to learn generalizable part-based feature representations of point clouds to reduce the geometry gap of distinct domains. A contrastive learning framework is further designed to improve the robustness of the part-based feature representation. Extensive experiments justify its effectiveness. The current method is designed for point cloud classification, and we plan to extend point cloud segmentation detection in the future work.
Acknowledgment This work was supported by NSFC with grant numbers of 11971373, 12125104, U20B2075, 12090021, 61721002.
10

---

References
[1] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution x-transformed points. In Neurips, volume 31, 2018.
[2] Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In CVPR, pages 8895-8904, 2019.
[3] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space. In Neurips, volume 30, 2017.
[4] Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. In ICLR, 2022.
[5] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652-660, 2017.
[6] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. ACM ToG, 38(5):1-12, 2019.
[7] Haoxi Ran, Wei Zhuo, Jun Liu, and Li Lu. Learning inner- relations point clouds. In ICCV, pages 15477-15487, 2021.
[8] Hugues Thomas, Charles R Qi, Jean- Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In ICCV, pages 6411-6420, 2019.
[9] Yifan Xu, Tianqi Fan, Mingye Xu, Long Zeng, and Yu Qiao. Spiderenn: Deep learning on point sets with parameterized convolutional filters. In ECCV, pages 87-102, 2018.
[10] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In ICCV, pages 16259-16268, 2021.
[11] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv: 1512.03012, 2015.
[12] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912-1920, 2015.
[13] Angela Dai, Angel X Chang, Manolis Savva. Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-anootated 3d reconstructions of indoor scenes. In CVPR, pages 5828 5839,,589,220 2017.
[14] Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In ICCV, pages 1588-1597, 2019.
[15] Torralba and AA Efros. Unbiased look dataset bias. In CVPR, pages 1521-1528, 2011.
[16] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In ICML, pages 10-18. PMLR, 2013.
[17] Da Li, Yongxin Yang, Yi-Zhe Song. and Timothy Hospedales. Deeper, broader and artier domain generalization. In ICCV, pages 5542-5550, 2017.
[18] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, 2018
[19] Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi. Domain gen- eralization for object recognition with multi-task autoencoders. In ICCV, pages 2551-2559, 2015.
[20] Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain generalization using meta-reguularzzttin. In Neurips, volume 31, 2018.
11

---

[21] Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu. Yi-Zhe Song, and Timothy M Hospedales. Episodic training for domain generalization. In ICCV, pages 1446-1455, 2019.
[22] Chao Huang, Zhangjie Cao, Yunbo Wang, Jianmin Wang, and Mingsheng Long. Metasets: Meta-leaming point sets for generalizable representations. In CVPR, pages 2021.
[23] Li Yi, Boqing Gong, and Thomas Funkhouser. Complete & label: A domain adaptation approach to semantic segmentation of lidar point clouds. CVPR, pages 15363-15373, 2021.
[24] Donald D Hoffman and Whitman Richards. Parts of recognition. Cognition, 18(1-3):65-98)
1984.
[25] Can Qin, Haoxuan You, Lichen Wang, C-C Jay Kuo, and Yun Fu. Pointdan: A multi-scale 3d domain adaption network for point cloud representation. In Neurips, volume 32, 2019.
[26] Shai Ben-David John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine learning. 79(1):151-175, 2010.
[27] Yishay Mansour. Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In COLT, 2009.
[28] Yao-Hung Hubert Tsai, Shaojie Bai. Paul Pu Liang, J Zico Kolter, Louis-Philippe Moreney, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In ACL, volume 2019, page 6558. NIH Public Access, 2019.
[29] Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, and Ming Liu. Neural speech synthesis with transformer network. In AAAI, volume 33, pages 7706-6713 2019.
[30] Ronghang Hu and Amanpreet Singh. Unit: Multimodal multitask learning with unified transformer. ICCV, pages 1439-1449 2021.
[31] Xiangyu Li, Yonghong Hou, Pichao Wang, Zhimin Gao, Mingliang Xu, and Wanging Li. Trear: Transformer-based rgb-d egocentric action recognition. IEEE TCDS, 14(1):246-22,, 2022.
[32] Tongkun Xu, Weihua Chen, Pichao Wang, Fan Wang, Hao Li, and Rong Jin. Cdtrans: Cross- domain transformer for unsupervised domain adaptation. In ICLR, 2022.
[33] Ren Chuan-Xiand Zhai Yi-Ming, Luo You-Wei, and Li Meng- Xue. Towards unsupervised domain adaptation via domain tiaiin-tranffomer.. arXiv preprint arXiv:2202.13777 2022.
[34] Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
[35] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. In Neurips, volume 33, pages 22243-22255, 2020.
[36] Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv e-prints, pages arXiv-1807, 2018
[37] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In Neurips, volume 33, pages 18661-18673, 2020.
[38] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17())2099-20330, 2016.
[39] Jonathan Sauder and Bjarne Sievers. Self-supervised deep learning on point clouds by recon- structing space. Neurips, 32, 2019.
[40] Idan Achituve, Haggai Maron, and Gal Chechik. Self-supervised learning for domain adaptation on point clouds. In WACV, pages 123-133, 2021.
12

---

[41] Longkun Zou, Hui Tang, Ke Chen, and Kui Jia. Geometry-aware self-training for unsupervised domain adaptation on object point clouds. In ICCV, pages 4403-6412, 2021.
[42] Hehe Fan, Xiaojun Chang, Wanyue Zhang, Yi Cheng, Ying Sun, and Mohan Kankanhalli. Self-supervised global-local structure modeling for point cloud domain adaptation with reliable voted pseudo labels. In CVPR, pages 6377-6386, 2022.
[43] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne, JMLR, 9(11), 2008.
[44] Kevin Lai and Dieter Fox. Object recognition in 3d point clouds using web data and domain adaptation. IJRR, 29(8):1019-1037 2010.
[45] Jan Knopp, Mukta Prasad, Geert Willems, Radu Timofte, and Lue Van Gool. Hough transform and 3d surf for robust three dimensional classification. In ECCV, pages 589-602. Springer, 2010.
[46] Oliver J Woodford, Minh-Th Tri Pham, Atsuto Maki, Frank Perbet, and Bjöm Stenger. Demisting the hough transform for 3d shape recognition and registration. IJCV, 106(3):332-344 3411, 2014.
[47] Young Min Kim, Niloy Mitra, Dong-Ming Yan, and Leonidas Guibas. Acquiring 3d indoor environments with variability and repetition. ACM TOG, 31(6):1-11,200 2012.
[48] Vladislav Ishimtsev, Alexey Bokhovkin, Alexey Artemov, Savva Ignatyev, Matthias Niessner, Denis Zorin, and Evgeny Burnaev. Cad-deform: Deformable fitting of cad models to 3d scans. In ECCV, pages 599-628. Springer, 2020.
[49]
Liangwei Shi, Zhimin Yuan, Ming Cheng, Yiping Chen, and Cheng Wang. Dfan: brancbranch feature alignment network for domain adaptation on point clouds. IEEE TGRS, 60:1-12, 2022.
[50] Adriano Cardace, Riccardo Spezialetti, Pierluigi Zama Ramirez, Samuele Salti, and Luigi Di Stefano. Refrec: Pseudo-labels refinement via shape reconstruction for unsupervised 3d domain adaptation. In 3DV, pages 331 331-341, 2021.
[51] Feiyu Wang, Wen Li, and Dong Xu. Cross-dataset point cloud recognition using deep- shallow domain adaptation network. IEEE TIP, 30:7364-7377, 2021.
[52] Jonathan Sauder and Bjarne Sievers. Self-supervised deep learning on point clouds by recon- structing space. Neurips, 32, 2019.
[53] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping, In CVPR, volume 2, pages 1735-1742, 2006.
[54] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, pages 9729-9738, 2020.
[55] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, arXiv preprint arXiv:2003.04220 2020.
[56] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV, pages 9640-9649, 2021.
[57] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent-a new approach to self-supervised learning. In Neurips, volume 33, pages 21271 11271-21284, 2020.
[58] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, pages 15750-15758, 2021.
[59] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Nearips, volume 33, pages 9912-9924, 2020.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Sect.7.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] This work does not present any significant societal consequence.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See Sect.5.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Sect.5.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No] We run each experiment three times and report the average results.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Sect.5.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A] We use public datasets.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "sj3b80TdN__", "reviewer": "Reviewer_KRL8", "summary": "The authors present a new method for generalizing point cloud classification from synthetic to real data. The authors argue that the local geometric features are more generalizable than the whole shape. They focus on part-based feature representation, and design a part-based domain generalization network for the point cloud classification task. The key idea is to build a common feature space using a part template, and then align the part-level features of the source and the target domain to this template. The authors demonstrate that the proposed method achieves state-of-the-art performance on the 3DDG benchmark. ", "questions": "(1) For completeness, I expect to find some information about how the part template is defined. However, it seems not provided except that in Line 246 the authors mentioned that they followed the experiment setting in MetaSets [22]. Could you clarify how the parts are defined? How does this affect the final performance as the proposed method is largely based on part features? \n\n(2) While the benchmarks are built upon ModelNet/ShapeNet and ScanObjectNN, I see that not all data are utilized. For example, the experiments are only done on two basic variants of ScanObjectNN including OBJ-ONLY and OBJ-BG. How does the transfer work if the target is the hardest variant of the ScanObjectNN? \n\n(3) As a domain generalization problem, it seems insufficient to conduct only sim-to-real transfer experiment. How about real-to-sim, i.e., training on ScanObjectNN and testing on ModelNet/ShapeNet? In practice, this could be used to build applications such as retrieving a CAD model from a given scan. As all data is available, I wish to further understand what challenges remain that make this experiment not conducted in both MetaSets and this paper. \n", "limitations": "I think the related work section in this paper is quite short and needs some revisions. First, while not exactly the same, I found in the literature there are some 3D tasks that link different domains together such as scans and CAD object retrieval. I think this is worth some further discussions about the connections of these specific tasks with the domain generalization problem presented in this paper. \n[A] SHREC'17: RGB-D to CAD retrieval with ObjectNN dataset, 2017, 2018.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- I like the idea to align local geometric features to solve domain generalization on point clouds. This idea is novel and significant. The technical approach to implement this idea is sound, and the experimental results demonstrate good performance.\n\n- I also like the idea to verify the hypothesis that local geometric features are more generalizable than global features in Fig. 2. However, I would like to point out a few issues here. \n  (1) It is true that in general reducing the part size leads to better generalization. But where is the limit? At the very least each part can be reduced to a point, but I do not believe that point-based features are the most generalizable. It could be more interesting to identify by how many points per part we would reach to limit of generalization here. \n  (2) 512-part-level and 256-part-level mean 512 and 256 points per part, respectively I guess. This sounds confusing as I can also think of it as 512 parts and 256 parts. It is better to revise this wording, like 512-points-per-part and 256-points-per-part. \n\n- I also value the clarity of the writing, which is very nice and easy to read. \n\n- Despite its great values, the paper suffers from the following issues. \n  (1) In terms of technical approach, the contrastive learning part is less well connected to the part-based features for domain generalization. For example, if the authors wish to use contrastive learning, at least shape-level contrastive loss should be used for the baseline methods as well. Or the comparisons should be separated with a table with no contrastive learning utilized. In Table 1, as I understand, the baselines are without contrastive loss but the PDG is with contrastive loss. Please correct me if I misunderstood. \n  (2) My second concern is that the experiments conducted are somewhat simplistic. I expect deeper analysis and more experimental settings to be done. Please see my comments in the question section. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fWklBDO-Aiu", "reviewer": "Reviewer_KHBY", "summary": "The authors propose to utilize the part-level feature representation for the cross-domain generalization problem in point cloud applications. Given part-level features grouped from point-wise representations, the authors first align them to a learned feature dictionary via cross-attention, and then aligned-features are aggreged with a part-weighted maxpooling strategy. In addition, contrastive learning is conducted in both shape-level and part-level. Empirical results on standard DG benchmark datasets are presented for validation. ", "questions": "See weaknesses.", "limitations": "Not found.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The method is well motivated.  The authors find that part-level features present smaller distribution divergence than shape-level feature in the cross-domain tasks. Therefore, they propose to adopt part level features in the DG tasks.\n\n2. Some interesting components are proposed and well justified. The proposed part-template features implicitly achieves the domain alignment by aligning both domains to the learned feature dictionary. The proposed part feature aggregation module outperfoms the popular max pooling module. \n\n3. The proposed method achieves state-of-the-art performance on DG benchmarks. \n\nWeaknesses:\n1. I am wondering the relationship between the proposed part feature based DG method and the general point cloud models (e.g., PointNet++) that utilize part/local features.   For example, in the PointNet++, the part level feature is extracted and aggregated hierarchically, which is quite similar to the strategy adopted in this paper.  Could you clarify it? \n\n2. Based on the first question, could the proposed module be adopted in general point cloud models? For example, could we replace the last max pooling layer of PointNet++ with the part feature aggregation module proposed in this paper? \n\n3. As for the proposed techniques, the contrastive loss is widely adopted as the learning regularization and utilizing part level features is also a common practice. In my opinion, the main contribution is the implicit domain alignment with the learned feature dictionary and the part feature aggregation module. So I suggest that the authors include more related work on the application of dictionary learning in cross-domain problems, such as [1]\n\n[1] Li, Shuai, et al. \"Category dictionary guided unsupervised domain adaptation for object detection.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 3. 2021 ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "WYGlUXl_G6", "reviewer": "Reviewer_GcXJ", "summary": "The presented method detects “global” features (PointNet or DGCNN)  locally on sampled points. Then learn relations between those local representations as part-level aggregation. The performance is further improved by contrastive learning. \n\nAuthors evaluate the approach on several cross domain datasets where the method is learnt on one domain and tested on another one. The target (test) domain is inaccessible during training.\n\nDomain adaptation is an important and well old problem for 3d point cloud processing.\n", "questions": "-Why did authors not report M->S like a comparison to other methods? It would be a very nice comparison to other papers [Achituve21, Shen22,..]. It looks strange that evaluation is done with datasets [11],[12],[14], but popular [13] (ScanNet) is missing, ScanNet is often used in previous methods  [Achituve21, Shen22,..]. Readers can think about many reasons why authors didn't report it and it looks strange that authors perform too many results, but the most important settings are somehow missing.\n\n-Authors say that they executed the competitors' code, still, they have different performance values than in other papers (in other papers they also vary from paper to paper).  E.g. M->S*, MetaSet paper claims they have 72% (so better than the presented method), but authors report ~66% as best for Metaset. Is it a problem of the field that once you download a repository of a paper you obtain different results that were originally published? The difference is higher than the paper’s improvement :( \n\n\n-If one looks on the proposed method as using \"global\" features (e.g. PointNet) locally on parts of the object and then learning relations of those local responses, then this is an old method how to represent 3d point cloud object in the scene between different domains, for example [Woodford13, Knopp10] learns spatial configuration of such features for the nice 3d shapes and apply it on the 3d scans, [Kim12] learns parts deformation of the 3d shapes and recognize them in the scanned cluttered scene. [Lai10] Looks like one of the first method to do 3d point domain adaption. Here is list of related missing work:\n[Lai10] Lai and Fox, Object Recognition in 3D Point Clouds Using Web Data and Domain Adaptation, IJRR 2010\n[Ishimtse20] Ishimtse at all, CAD-Deform: Deformable Fitting of CAD Models to 3D Scans, ECCV 2020\n[Qi19] Qi et all, Deep Hough Voting for 3D Object Detection in Point Clouds, ICCV 2019\n[Golovinskiy09] Golovinskiy et al, Shape-based Recognition of 3D Point Clouds in Urban Environments, ICCV'09\n[Knopp10] Knopp et all; Hough Transform and 3D SURF for Robust Three Dimensional Classification, ECCV 2010\n[Woodford13] Woodford et all, Demisting the Hough Transform for 3D Shape Recognition and Registration, IJSV 2013\n[Kim12] Y.M. Kim et al., Acquiring 3D Indoor Environments with Variability and Repetition, ACM ToG, SIGGRAPH Asia 2012\n", "limitations": "It looks there is no potential for negative social impact of the work.\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n-Clear motivation, I like the motivation by features distance between domains (Fig. 2)\n\n\n-While the idea of learning part-based models from local features is old and highly researched, the presented method on 3D point clouds with Neural Networks focused for the domain adaptations seems novel.\n\n-The problem of domain adaptation of 3d point cloud processing when the target domain is unavailable  during training is a very important and unsolved problem.\n\n-Most of questions I had during reading were further answered\n\n\nCons:\n-The approach is motivated from many previous works that focus on domain adaptation for 3d point cloud that are not cited, though the approach is novel by using it in neural networks.\n\n-Authors did not evaluate on some datasets pairs (training-testing) that will allow much broader comparison. That raises several questions why. I believe it will be also good to report why presented numbers of other methods differ from original papers.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept