Title: Improving Robustness of 3D Point Cloud Recognition from a Fourier Perspective
Venue: NeurIPS 2024
Accepted: yes
Keywords: 3D point cloud recognition

corruption robustness

Frequency Adversarial Training
Abstract: Although 3D point cloud recognition has achieved substantial progress on standard benchmarks, the typical models are vulnerable to point cloud corruptions, leading to security threats in real-world applications. To improve the corruption robustness, various data augmentation methods have been studied, but they are mainly limited to the spatial domain. As the point cloud has low information density and significant spatial redundancy, it is challenging to analyze the effects of corruptions. In this paper, we focus on the frequency domain to observe the underlying structure of point clouds and their corruptions. Through graph Fourier transform (GFT), we observe a correlation between the corruption robustness of point cloud recognition models and their sensitivity to different frequency bands, which is measured by the GFT spectrum of the model’s Jacobian matrix. To reduce the sensitivity and improve the corruption robustness, we propose Frequency Adversarial Training (FAT) that adopts frequency-domain adversarial examples as data augmentation to train robust point cloud recognition models against corruptions. Theoretically, we provide a guarantee of FAT on its out-of-distribution generalization performance. Empirically, we conduct extensive experiments with various network architectures to validate the effectiveness of FAT, which achieves the new state-of-the-art results.

Improving Robustness of 3D Point Cloud Recognition from a Fourier Perspective

Abstract

Although 3D point cloud recognition has achieved substantial progress on standard benchmarks, the typical models are vulnerable to point cloud corruptions, leading to security threats in real-world applications. To improve the corruption robustness, various data augmentation methods have been studied, but they are mainly limited to the spatial domain. As the point cloud has low information density and significant spatial redundancy, it is challenging to analyze the effects of corruptions. In this paper, we focus on the frequency domain to observe the underlying structure of point clouds and their corruptions. Through graph Fourier transform (GFT), we observe a correlation between the corruption robustness of point cloud recognition models and their sensitivity to different frequency bands, which is measured by the GFT spectrum of the model's Jacobian matrix. To reduce the sensitivity and improve the corruption robustness, we propose Frequency Adversarial Training (FAT) that adopts frequency-domain adversarial examples as data augmentation to train robust point cloud recognition models against corruptions. Theoretically, we provide a guarantee of FAT on its out-of-distribution generalization performance. Empirically, we conducted extensive experiments with various network architectures to validate the effectiveness of FAT, which achieves the new state-of-the-art results.

1 Introduction

3D point cloud recognition based on deep neural networks (DNNs) [35, 36, 65] has achieved unprecedented performance on typical benchmarks [5, 67], which assume that the data are independently and identically distributed. However, in practical scenarios, point clouds suffer from severe corruptions (e.g., noise, density change, transformation) due to sensor imprecision and scene complexity [66, 76]. When the testing distribution is different from the training distribution caused by corruption, point cloud recognition models have significant performance degradation [41, 51], indicating that they lack the robustness of human visual system [40], while also raising concerns about safety and reliability of these models. As deep 3D point cloud recognition has been increasingly deployed in safety-critical applications, such as autonomous driving [6, 84], robotics [60, 92], and medical image processing [54], it is of crucial importance to improve the robustness of 3D point cloud recognition models to out-of-distribution (OOD) point cloud data induced by corruptions [10].

Jitter

Rotate

Robustness vs. Sensitivity

in High-frequency Bands

Robustness vs. Sensitivity

in Low-frequency Bands

(a)

(b)

PointNet

CurveNet

GDANet

PCT

DGCNN

RSCNN

PointNet

++

PAConv

CurveNet

GDANet

PCT

DGCNN

PAConv

PointNet

++

RSCNN

PointNet

GF-

GF

GF

GF

254

512

952

To improve the corruption robustness, the most effective approaches to date are based on carefully designed data augmentation techniques [41, 51]. Inspired by 2D image augmentations [85, 89], some methods blend two point clouds for data augmentation using shortest-path interpolation (e.g., PointMixup [7]), random blending (e.g., PointCutMix-R [90]), and rigid transformation (e.g., RSMix [24], PointCutMix-K [90]). PointWOLF [22] enriches data diversity by applying non-rigid deformation to object parts. WOLFMix [41] deforms objects first and then rigidly blends two deformed objects. Although these data augmentation techniques improve the corruption robustness to some extent, they are all based on spatial-domain transformations. Raw point clouds in the spatial domain have low information density and heavy spatial redundancy [8], making it challenging to analyze which specific information is corrupted. To address this challenge, we shift our attention from the spatial domain to the frequency domain to analyze the underlying structure of point clouds that is not easily observable from the raw point clouds. In the frequency domain, point clouds are compactly represented, facilitating a better understanding of low-level distortions that are free of high-level semantics.

To design robust models, the first step is to understand how corruption is represented in the frequency domain. We achieve this by transforming the raw point clouds and the corresponding corruptions into compact representations in the frequency domain using the graph Fourier transform (GFT) [43]. By visualizing the transformed signals, we observe that different corruptions affect varying frequency bands, as shown in Fig. 1(a). Motivated by the differences, we investigate the relationship between the corruption robustness of various point cloud recognition models and their sensitivity to different frequency bands [81]. To measure the sensitivity, we design a novel metric based on GFT spectrum of the Jacobian matrix of the model, as shown in Fig. 3. Our key insight is that our proposed high/low frequency sensitivity metric is negatively correlated with the model's robustness under high/low frequency corruptions , as shown in Fig. 1(b). This correlation emphasizes the importance of the model's sensitivity to high and low frequencies for corruption robustness. However, it is still challenging to simultaneously reduce the sensitivity of point cloud recognition models to both high and low frequencies.

To address this issue, we propose Frequency Adversarial Training (FAT) to improve the corruption robustness of 3D point cloud recognition models. FAT trains a model with adversarial examples that add perturbations to the frequency-domain representations of point clouds. Intuitively, a model robust to worst-case perturbations should be more resistant to real-world corruptions [72, 80]. We provide a theoretical analysis that demonstrates the effectiveness of FAT in ensuring OOD generalization of the model , as shown in Theorem 1. To eliminate potential performance degradation due to mutual interference between high and low frequency signals, we utilize the AdvProp training framework [72], based on which we use three separate batch normalization (BN) statistics for clean samples, highfrequency adversarial samples, and low-frequency adversarial samples, respectively.

We conducted extensive experiments to validate the effectiveness of our approach in improving the robustness of point cloud recognition models under common corruptions [41, 51]. With various

network architectures, our method improves the corruption robustness by a large margin. By integrating our approach with previous data augmentation techniques, we achieve the new state-of-the-art performance.

2 Related work

Deep learning on 3D point clouds. Deep 3D point cloud recognition [16, 35, 38, 55, 70, 73, 79] has emerged in recent years as a prominent research area with diverse applications in several fields such as 3D object classification [46, 83, 86], 3D scene segmentation [20, 64, 75], and 3D object detection in autonomous driving [77, 95]. One of the pioneering works is PointNet [35], which employs a multilayer perceptron to learn point features and utilizes a max-pool module to aggregate them efficiently. Many subsequent works [13, 30, 36, 78] improve upon PointNet. Several approaches focus on designing special convolutions on 3D domains [26, 31, 56] or developing graph neural networks [14, 44, 65] to improve point cloud recognition, such as DGCNN [65] which builds a dynamic graph for point cloud data. Recently, drawing inspiration from research in the frequency domain [4, 49, 61, 81], GDANet [74] introduces a geometry-disentangle module to dynamically separate point clouds into the contour and flat parts of 3D objects, thereby capturing complementary 3D geometric semantics. PCT [17] uses Transformer to improve point cloud learning. Additionally, there is a growing discussion on point cloud augmentation, including mix-based augmentations [7, 90], deformation-based augmentations [22], and auto-augmentations [25].

Robustness in 3D point cloud recognition. Following the previous studies on robustness in the 2D image domain [53, 3, 15, 19, 32, 42, 59, 9, 82], several works [18, 50, 52, 63, 69, 34, 97] have explored the robustness of 3D point cloud classifiers. Concerning adversarial robustness, Xiang et al. [69] first demonstrate that point cloud recognition is vulnerable to adversarial point generation attacks. Further research [21, 28, 29, 57, 91, 2] has employed gradient-based point perturbation attacks. Some defensive techniques are proposed, such as input randomization [12, 93] and geometryaware framework [68] to defend against such vulnerabilities. Sun et al. [47, 48] have studied the effectiveness of adversarial training and pre-training on self-supervised tasks in enhancing robustness. In terms of corruption robustness, some works have studied the problem using invariant feature extraction [71], and adaptive sampling [76]. Recently, two benchmarks [41, 51] are developed for the robustness of 3D point cloud recognition under corruptions and demonstrate the effectiveness of data augmentation. However, unlike the existing spatial-domain data augmentation techniques [22, 25], in this paper, we focus on the frequency domain and propose Frequency Adversarial Training (FAT) to improve the model's out-of-distribution generalization ability.

3 Methodology

The existing 3D point cloud recognition models exhibit significant performance degradation under point cloud corruptions [41, 51]. Although data augmentation techniques have shown the effectiveness in improving robustness, they are typically based on spatial-domain transformations, which suffer from low information density and heavy spatial redundancy of the raw point clouds. Consequently, it is difficult to analyze which specific information has been lost due to corruptions within the spatial domain. To address this challenge, we shift our focus to the frequency domain, which enables us to analyze the underlying structure of point clouds.

In the following, we first provide the background knowledge of graph Fourier transform (GFT) in Sec. 3.1, then analyze the point cloud corruptions in the frequency domain in Sec. 3.2, and investigate the relationship between the model's corruption robustness and sensitivity to frequency changes in Sec. 3.3. Based on the analyses, we propose a Frequency Adversarial Training (FAT) method detailed in Sec. 3.4 with a theoretical analysis to guarantee its effectiveness in Sec. 3.5.

3.1 Graph Fourier transform

Images are typically transformed and recovered in the frequency domain with the 2D discrete Fourier transform (DFT) and inverse DFT [39]. Unlike images, although 3D point clouds are highly structured, they reside on irregular domains without an ordering of points, hindering the deployment of traditional Fourier transforms. However, graphs provide a natural and accurate representation of

Clean

Rotate

Jitter

Scale

Drop Global

Drop Local

Add Global

Add Local

C2

C2

C

C2

C2

C2

G,F

GF

G.F

GF

GF

GF

GF

GF

GF

GF

GF

GF

irregular point clouds. Once a graph is constructed to represent the point cloud, the graph Fourier transform (GFT) [43] can compactly transform it into the frequency domain.

Given a point cloud P := { p i } n i =1 ∈ R n × 3 of n points, where p i denotes the xyz coordinates of a point, we construct a directed graph G = {P , E , W } to represent it. The graph consists of a vertex set P , an edge set E connecting the vertices, and an adjacency matrix W . The entry w i,j in the adjacency matrix represents the weight of the edge from vertices i to j , which is used to capture the similarity between adjacent vertices. Here, we construct a weighted k -nearest neighbor graph (i.e., each vertex is only connected to its k -nearest neighbors) using the Euclidean distance d ij = ∥ p i -p j ∥ 2 between vertices i and j , and the weight of the edge is w i,j = e -d 2 ij .

After constructing the graph representation of the point cloud, we focus on the combinatorial graph Laplacian [45], defined as L := D -W , where D is a diagonal matrix with the i -th diagonal entry d i,i = ∑ n j =1 w i,j representing the degree of the i -th node. L is symmetric and positive semidefinite, and can be eigen-decomposed as L = U Λ U ⊤ , where U = [ u 1 , ..., u n ] is an orthogonal matrix containing the eigenvectors u i , and Λ = diag( λ 1 , ..., λ n ) is a diagonal matrix containing the eigenvalues. The eigenvalues are sorted in ascending order, representing frequencies from low to high. For a point cloud P , the graph Fourier transform (GFT) can be applied to transform it into a compact representation in the frequency domain: ˆ P = G F ( P ) := U ⊤ P . The low-frequency components represent the coarse shape of the point cloud, while the high-frequency components represent the fine details. The inverse graph Fourier transform (IGFT) can be used to recover the point cloud in the spatial domain as P = G F -1 ( ˆ P ) := U ˆ P .

3.2 Analyzing point cloud corruptions in the frequency domain

We employ GFT to transform point clouds and their corruptions into compact representations in the frequency domain, allowing us to analyze the underlying structures of these low-level distortions that are hardly observable in the spatial domain. For raw point clouds, we transform them to the frequencydomain representations and calculate E P [ | G F ( P ) | ] by averaging over all validation point clouds in ModelNet40 [67]. For each corruption type in ModelNet-C [41], we calculate E P [ | G F ( C ( P ) -P ) | ] similarly, where C denotes the corruption function. As the input point clouds have three spatial axes ( x, y, z ) , we take the average over these channels. In Fig. 2, we visualize the graph frequency-domain representations of raw point clouds and the corruptions in ModelNet-C. We can see that the raw point clouds have higher power in the low-frequency region, while the corruption 'Jitter' leads to higher power in the high-frequency region. For corruptions such as 'Rotate' and 'Scale', the corrupted power is concentrated more on the low-frequency components. The results demonstrate that different corruptions of point clouds affect different frequency bands.

3.3 Relationship between corruption robustness and sensitivity to frequency bands

Motivated by the different effects of corruptions on varying frequency bands observed in the graph frequency-domain representations, we investigate the relationship between the corruption robustness of 3D point cloud recognition models and their sensitivity to different frequency bands.

Input

Robustness vs. Sensitivity

in Low-frequency Bands

GFT Spectrum

CurveNet

GDANet

PCT

DGCNN

PAConv

PointNet

++

RSCNN

PointNet

GF

128

Frequency

To measure the sensitivity of a model on different frequency bands, we propose to perform graph Fourier transform (GFT) on the Jacobian matrix of the model's output loss with respect to its input point cloud. Intuitively, the Jacobian matrix represents how the model's output changes with small variations in its input point cloud, revealing its sensitivity to different points in the spatial domain [1]. With GFT, we can obtain the frequency-domain representation of the Jacobian matrix, which reveals the model's sensitivity to different frequency bands of input. If a model's Jacobian matrix has a high proportion of low/high frequency components, it will be sensitive to low/high frequency bands.

Fig. 3 illustrates the computation of the frequency-domain Jacobian matrix for a single point cloud.

Specifically, given an input point cloud P , a classification model h , and a standard cross-entropy loss function L h for the classification task, the Jacobian matrix J ( P ) := ∇ P L h of the loss with respect to the input point cloud can be calculated. We then perform GFT on J ( P ) to obtain its frequency-domain representation, denoted as ̂ J ( P ) = U ⊤ J ( P ) in a compact form, using the original point cloud's neighborhood relations and feature vector matrix. Since the input point cloud has three axis channels ( x, y, z ) , we take the average of these channels and normalize the result. We measure the model's sensitivity to input perturbations in the low-frequency band by summing the squares of the amplitudes of the first λ frequencies of the Jacobian matrix's graph Fourier spectrum. The sensitivity to high-frequency perturbations is measured by summing the squares of the amplitudes of the remaining 1024 -λ frequencies. A higher value of the metric indicates greater sensitivity to perturbations in that frequency band.

We can now measure the importance of the sensitivity to different frequency bands of point cloud recognition models on their corruption robustness. First, we measure and establish the relationship between sensitivity to high/low frequency bands of different point cloud models and their accuracy under high/low frequency corruptions. As illustrated in Fig. 1(b), our proposed frequency sensitivity metrics are negatively correlated with the corruption robustness. Therefore, point cloud models that are less sensitive to high/low frequency bands exhibit better robustness to high/low frequency corruptions. This correlation indicates that the sensitivity of models to different frequency bands affects their corruption robustness, providing insights for further improving the robustness of point cloud recognition models.

3.4 Frequency adversarial training

The above analyses highlight the importance of the sensitivity of point cloud recognition models to high and low frequencies on their corruption robustness. However, reducing the sensitivity of point cloud models to both high and low frequencies is still challenging. To address this problem, we propose Frequency Adversarial Training (FAT) to improve the corruption robustness of point cloud recognition models using adversarial examples in the frequency domain. Intuitively, a model trained to be robust to worst-case adversarial perturbations should be naturally robust to real-world corruptions [72, 80], as also theoretically demonstrated in Sec. 3.5.

To simultaneously reduce the sensitivity of point cloud recognition models to high and low frequencies, we generate high-frequency adversarial examples and low-frequency adversarial examples, which are added to the training set. We generate high-frequency adversarial examples that alter the details of

the point clouds, and low-frequency adversarial examples that change the rough shapes of the point clouds. To prevent the mutual interference of high-frequency and low-frequency adversarial examples that may lead to a decrease in model performance, we adopt the AdvProp training framework [72], where clean samples, high-frequency adversarial samples, and low-frequency adversarial samples are separately processed using three batch normalizations during adversarial training. Specifically, for an input point cloud P with the ground-truth label y , our optimization objective is

where D is the underlying data distribution, L h is the loss function, θ is the network parameter, ϵ h and ϵ l are high-frequency and low-frequency adversarial perturbations, and S h and S l are the high-frequency and low-frequency perturbation ranges, respectively.

3.5 Theoretical analysis

To verify the claim that a model robust to frequency-domain worst-case perturbations should be more resistant to real-world corruptions, we provide a theoretical analysis that demonstrates the effectiveness of FAT in ensuring OOD generalization of the model.

Suppose ( x, y ) is a pair of training sample x and its label y . The loss on ( x, y ) with model parameter θ is L ( θ, ( x, y )) , where L ( θ, ( x, y )) is continuous and differentiable for both θ and ( x, y ) . We let f ( x ) := L ( θ, ( x, y )) for simplicity. Let F and F -1 denote the Fourier transform and inverse Fourier transform, respectively. The norm ∥ · ∥ p denotes the ℓ p -norm. We have the following theorem:

Theorem 1. If f satisfies that: f ( x ) ∈ [0 , M ] for all x , | f ( F -1 ( F ( x ) + α )) -f ( x ) | ≤ ϵ for all x and ∥ α ∥ p ≤ δ , then for any distribution P o and P a satisfying that Was p ( P o , P a ) := (inf u ∈ Π( P o ,P a ) E ( x,z ) ∼ u [ ∥F ( x ) -F ( z ) ∥ p p ]) 1 /p ≤ η , where η < δ , then, with probability 1 -γ , we have:

where { x i } i m =1 are i.i.d. samples from P a .

Remark 1. Intuitively, OOD corresponds to the shifted distribution P o that approaches the training distribution P a . Thus Was p ( P o , P a ) defines OOD from the perspective of measuring the distance between distributions. E z ∼ P o [ f ( z )] -1 m ∑ m i =1 f ( x i ) represents the OOD generalization error of the model. | f ( F -1 ( F ( x ) + α )) -f ( x ) | ≤ ϵ and ∥ α ∥ p ≤ δ indicate that the model is robust under frequency-domain perturbations. The bound (2) implies that models that are adversarially robust in the frequency-domain have smaller generalization bounds on OOD data.

The proof of Theorem 1 is deferred to Appendix A. Thus, the frequency-domain adversarial robustness of the model guarantees the generalization on OOD data. We have the following observations:

The right-hand side of Eq. (2) implies that models that are more robust to frequency domain adversarial samples (i.e., larger δ and smaller ϵ ) have smaller OOD generalization bounds and thus perform better on OOD data.

For Eq. (2), a larger number of training samples m leads to a smaller OOD generalization bound. This indicates that more training samples can compensate for the degradation of generalization performance.

4 Experiments

In this section, we first detail the experimental settings in Sec. 4.1, then present the main results in Sec. 4.2 to show the effectiveness of our method. We further integrate our method with other data augmentation techniques in Sec. 4.3 and perform ablation studies in Sec. 4.4.

4.1 Experimental setup

Dataset. To validate the effectiveness of our FAT method in enhancing the corruption robustness of 3D point cloud recognition models, we train all models on the standard ModelNet40 training set [67]. In addition to reporting the performance of the models on the original ModelNet40 validation set, we also evaluate the corruption robustness on ModelNet-C [41] in the main paper and ModelNet40-C [51] in Appendix B. The ModelNet40 dataset [67] contains 12,311 CAD models with 40 common object categories in the real world. We use the official split [35], where 9,843 examples are used for training and the remaining 2,468 examples are used for testing. The ModelNet-C dataset [41] is designed for measuring the network robustness to common point cloud corruptions. It consists of 7 different corruption types, including 'Scale', 'Jitter', 'Rotate', 'Drop Global', 'Drop Local', 'Add Global', and 'Add Local'. Each type of corruption has five severity levels. ModelNet40-C [51] is a similar dataset with 15 corruptions, which will be detailed in Appendix B.

Model architectures. Following [41, 51], we select four representative model architectures: PointNet [35], DGCNN [65], PCT [17], and GDANet [74]. These models represent different architectural designs and have been widely applied in 3D visual tasks.

Evaluation metrics. To measure the corruption robustness of different methods, we follow [41] and use the mean corruption error (mCE) as the main evaluation metric. We adopt the official baseline DGCNN and first compute the corruption error (CE) for a given corruption type i by averaging over 5 severity levels: CE i = ∑ 5 l =1 (1 -OA i,l ) ∑ 5 l =1 (1 -OA DGCNN i,l ) , where OA i,l is the overall accuracy on a corruption test set i at severity level l , and OA DGCNN i,l is the overall accuracy of the baseline. Then, we average over the 7 corruption types to compute the mean corruption error: mCE = 1 N ∑ N i =1 CE i . In addition, we also report the clean overall accuracy (OA), the corruption overall accuracy (mOA), and the relative mCE (RmCE) following [41]. Due to space constraints, we provide the definition of RmCE and report mOA and RmCE in Appendix B.

Implementation details. For each method, we train 250 epochs using the smooth cross-entropy loss [65] and Adam optimizer [23], and select the best performant model for further evaluation. We follow the DGCNN protocol [16]. For our method, we set k = 30 for the k -nearest neighbor graph and λ = 100 for dividing high-frequency and low-frequency [29]. We use PGD [33] and AOF [27] to generate high-frequency adversarial examples and low-frequency adversarial examples, respectively. We constrain S h and S l by 0.3 and 0.5, respectively. For more detailed training settings, please refer to Appendix B.

(a) DGCNN

(b) PointNet

(c) PCT

(d) GDANet

4.2 Main results

In this section, following [41, 51], we compare our proposed Frequency Adversarial Training (FAT) method with vanilla training, adversarial training and DUP Defense [93] on the ModelNet-C test set, demonstrating the effectiveness of FAT in enhancing corruption robustness. Table 1 presents a comparative analysis of different methods based on mean corruption error (mCE), clean overall accuracy (OA), and corruption error (CE) for each corruption type.

As shown in Table 1, our proposed Frequency Adversarial Training (FAT) outperforms all other methods in terms of mean corruption error (mCE), while exhibiting comparable performance in terms of overall accuracy (OA). The improvement in corruption robustness across the four different model architectures demonstrates the generalizability/universality of our method across different architectures. In Fig. 4, we visualize the sensitivity maps based on Jacobian matrices of Frequency Adversarial Training (FAT) and vanilla training under four different model architectures. FAT reduces the sensitivity of the model across different frequency bands.

It is noteworthy that GDANet introduces a geometry-disentangle module to dynamically disentangle point clouds into the contour and flat part of 3D objects, capturing complementary 3D geometric semantics. In contrast, FAT does not modify the network architecture to focus on the frequency domain but instead employs adversarial training in the frequency domain. As shown in Table 1, the two methods are complementary and synergistic, leading to improved model robustness. We report the performance of different methods in terms of overall corruption accuracy (mOA) and relative mCE (RmCE) in Appendix B, where the improvement in robustness of FAT is also significant under these metrics. The comparisons in Table 1 and Appendix B confirm that our proposed FAT enhances the OOD generalization ability of the model.

4.3 Data augmentation

To further validate the effectiveness of our proposed Frequency Adversarial Training (FAT), following [41], we investigate the performance of FAT in combination with different data augmentation strategies, including RSMix [24], PointWOLF [22], and WOLFMix [41]. These strategies respectively represent mix-based augmentation, deformation-based augmentation, and a combination of both mix-based and deformation-based augmentation. RSMix involves rigidly blending two point clouds using a transformation. PointWOLF enriches data diversity by applying non-rigid deformations to object parts. WOLFMix, designed based on PointWOLF and RSMix, first deforms the objects and then rigidly blends two deformed objects. When combining the data augmentation strategies, we first perform data augmentation on the input point cloud and then generate adversarial examples. For mix-based augmentation, we perform untargeted adversarial attacks on both labels being mixed to generate the adversarial examples.

In Table 2, we show the performance of FAT when integrated with different data augmentation strategies in terms of mean corruption error (mCE), clean overall accuracy (OA), and corruption error (CE) for each corruption type. Compared with a single data augmentation strategy, the combination of FAT and data augmentation strategies achieves a better mCE, which is attributed to the complementary and compatible information from both the spatial and frequency domains. The improvement in corruption robustness under three different data augmentation strategies demonstrates the generalization capability of our proposed method. As shown in Table 2, training GDANet with the combination of our proposed FAT with WOLFMix achieves a new state-of-the-art performance, with an impressive 0.537 mCE.

4.4 Ablation study

In this section, we conduct ablation study among our proposed Frequency Adversarial Training (FAT), as well as FAT variants: FAT w/o low-frequency, FAT w/o high-frequency, FAT w/o frequencydivision, and FAT w/o Advprop. FAT w/o low-frequency generates only high-frequency adversarial samples, while FAT w/o high-frequency generates only low-frequency adversarial samples. FAT w/o frequency-division randomly generates adversarial samples within a certain frequency range, without dividing the high and low frequency bands. FAT w/o Advprop does not use the AdvProp training framework [72]. We compare these methods in Table 3 based on mean corruption error (mCE), clean overall accuracy (OA), and corruption error (CE) measurements for each corruption type.

Compared with other methods, FAT w/o low-frequency has a lower mCE for high-frequency corruptions such as 'Jitter', while FAT w/o high-frequency has a lower mCE for low-frequency corruptions such as 'scale'. As discussed in Sec. 3.3, this is because adversarial training on high/low frequencies reduces the high/low frequency sensitivity, thus improving robustness to high/low-frequency corruptions. The performance of FAT w/o frequency-division falls between FAT w/o low-frequency and FAT w/o high-frequency. Although FAT w/o Advprop has a better mCE, its clean overall accuracy (OA) is worse than the other methods due to mutual interference between samples from different distributions, which may cause potential performance degradation. Compared with these methods, FAT achieves the lowest mCE, showing the effectiveness of our algorithm. More experimental results can be found in Appendix B.

5 Conclusion

In this paper, we study the robustness of 3D point cloud recognition models under common corruptions. We focus on the frequency domain to analyze the underlying structure of point clouds and common corruptions. Through graph Fourier transform (GFT), we identify a correlation between the corruption robustness and the model sensitivity to different frequency bands. Motivated by the analysis, we propose Frequency Adversarial Training (FAT), an adversarial training method based on frequency-domain adversarial examples to improve the corruption robustness of 3D point cloud recognition models. Extensive experiments demonstrate that the proposed method significantly improves the corruption robustness of various point cloud models, and can be integrated with other data augmentation techniques to achieve the state-of-the-art performance.

Limitation and broader impact. A limitation of our proposed method is that it reduces the clean accuracy a bit, e.g., FAT reduces the clean accuracy of DGCNN by 0.1%, PointNet by 0.5%, PCT by 1.0%, and GDANet by 0.6%. This may be caused by the inherent trade-off between accuracy and robustness [88]. Additionally, despite the complexity in implementation, FAT does not affect the efficiency of model inference, ensuring unhindered deployment of well-trained models in practical applications. The robustness of 3D point cloud recognition under corruptions is a severe problem towards safe and reliable 3D perception. Our work proposes an effective method to solve this issue, which does not have any negative social impact.

Acknowledgments

This work is supported by NKRDP grant No.2018YFA0704705, NSFC grants No.62276149 and No.12288201, and grant GJ0090202. Y. Dong is supported by the China National Postdoctoral Program for Innovative Talents. The authors thank anonymous referees for their valuable comments. L. Yu is supported by CAS Project for Young Scientists in Basic Research, Grant No.YSBR-040, ISCAS New Cultivation Project ISCAS-PYFX-202201, and ISCAS Basic Research ISCAS-JCZD202302.

References

Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems , 31, 2018.

Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. Adversarial objects against lidar-based autonomous driving systems. arXiv preprint arXiv:1907.05418 , 2019.

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39-57. IEEE, 2017.

Alvin Chan, Yew-Soon Ong, and Clement Tan. How does frequency bias affect the robustness of neural image classifiers against common corruption and adversarial perturbations? arXiv preprint arXiv:2205.04533 , 2022.

Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 , 2015.

Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages 1907-1915, 2017.

Yunlu Chen, Vincent Tao Hu, Efstratios Gavves, Thomas Mensink, Pascal Mettes, Pengwan Yang, and Cees GM Snoek. Pointmixup: Augmentation for point clouds. In Computer VisionECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16 , pages 330-345. Springer, 2020.

Yunpeng Chen, Haoqi Fan, Bing Xu, Zhicheng Yan, Yannis Kalantidis, Marcus Rohrbach, Shuicheng Yan, and Jiashi Feng. Drop an octave: Reducing spatial redundancy in convolutional

neural networks with octave convolution. In Proceedings of the IEEE/CVF international conference on computer vision , pages 3435-3444, 2019.

Shuyu Cheng, Yibo Miao, Yinpeng Dong, Xiao Yang, Xiao-Shan Gao, and Jun Zhu. Efficient black-box adversarial attacks via bayesian optimization guided by a function prior. In Forty-first International Conference on Machine Learning , 2024.

Wenda Chu, Linyi Li, and Bo Li. Tpc: Transformation-specific smoothing for point cloud models. arXiv preprint arXiv:2201.12733 , 2022.

Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning , pages 854-863. PMLR, 2017.

Xiaoyi Dong, Dongdong Chen, Hang Zhou, Gang Hua, Weiming Zhang, and Nenghai Yu. Self-robust 3d point recognition via gather-vector guidance. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11513-11521. IEEE, 2020.

Yueqi Duan, Yu Zheng, Jiwen Lu, Jie Zhou, and Qi Tian. Structural relational reasoning of point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 949-958, 2019.

Xiang Gao, Wei Hu, and Guo-Jun Qi. Graphter: Unsupervised learning of graph transformation equivariant representations via auto-encoding node-wise transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7163-7172, 2020.

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 , 2018.

Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, and Jia Deng. Revisiting point cloud shape classification with a simple and effective baseline. In International Conference on Machine Learning , pages 3809-3820. PMLR, 2021.

Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pct: Point cloud transformer. Computational Visual Media , 7:187-199, 2021.

Abdullah Hamdi, Sara Rojas, Ali Thabet, and Bernard Ghanem. Advpc: Transferable adversarial perturbations on 3d point clouds. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16 , pages 241-257. Springer, 2020.

Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781 , 2019.

Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. Learning semantic segmentation of large-scale point clouds with random sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.

Qidong Huang, Xiaoyi Dong, Dongdong Chen, Hang Zhou, Weiming Zhang, and Nenghai Yu. Shape-invariant 3d adversarial point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15335-15344, 2022.

Sihyeon Kim, Sanghyeok Lee, Dasol Hwang, Jaewon Lee, Seong Jae Hwang, and Hyunwoo J Kim. Point cloud augmentation with weighted local transformations. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 548-557, 2021.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Dogyoon Lee, Jaeha Lee, Junhyeop Lee, Hyeongmin Lee, Minhyeok Lee, Sungmin Woo, and Sangyoun Lee. Regularization strategy for point cloud via rigidly mixed sample. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15900-15909, 2021.

Ruihui Li, Xianzhi Li, Pheng-Ann Heng, and Chi-Wing Fu. Pointaugment: an autoaugmentation framework for point cloud classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6378-6387, 2020.

Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointcnn: Convolution on x-transformed points. Advances in neural information processing systems , 31, 2018.

Binbin Liu, Jinlai Zhang, and Jihong Zhu. Boosting 3d adversarial attacks with attacking on frequency. IEEE Access , 10:50974-50984, 2022.

Daizong Liu and Wei Hu. Imperceptible transfer attack and defense on 3d point cloud classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Daizong Liu, Wei Hu, and Xin Li. Point cloud attacks in graph spectral domain: When 3d geometry meets graph signal processing. arXiv preprint arXiv:2207.13326 , 2022.

Yongcheng Liu, Bin Fan, Gaofeng Meng, Jiwen Lu, Shiming Xiang, and Chunhong Pan. Densepoint: Learning densely contextual representation for efficient point cloud processing. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5239-5248, 2019.

Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8895-8904, 2019.

Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, and Ekin D Cubuk. Improving robustness without sacrificing accuracy with patch gaussian augmentation. arXiv preprint arXiv:1906.02611 , 2019.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Yibo Miao, Yinpeng Dong, Jun Zhu, and Xiao-Shan Gao. Isometric 3d adversarial examples in the physical world. In Advances in Neural Information Processing Systems , volume 35, pages 19716-19731, 2022.

Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 652-660, 2017.

Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems , 30, 2017.

Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in neural information processing systems , 35:23192-23204, 2022.

Yongming Rao, Jiwen Lu, and Jie Zhou. Global-local bidirectional reasoning for unsupervised representation learning of 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5376-5385, 2020.

Mohammed H Rasheed, Omar M Salih, Mohammed M Siddeq, and Marcos A Rodrigues. Image compression based on 2d discrete fourier transform and matrix minimization algorithm. Array , 6:100024, 2020.

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning , pages 5389-5400. PMLR, 2019.

Jiawei Ren, Liang Pan, and Ziwei Liu. Benchmarking and analyzing point cloud classification under corruptions. In International Conference on Machine Learning , pages 18559-18575. PMLR, 2022.

Evgenia Rusak, Lukas Schott, Roland S Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16 , pages 53-69. Springer, 2020.

Aliaksei Sandryhaila and José MF Moura. Discrete signal processing on graphs: Graph fourier transform. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing , pages 6167-6170. IEEE, 2013.

Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4548-4557, 2018.

David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine , 30(3):83-98, 2013.

Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision , pages 945-953, 2015.

Jiachen Sun, Karl Koenig, Yulong Cao, Qi Alfred Chen, and Z Morley Mao. On adversarial robustness of 3d point cloud classification under adaptive attacks. arXiv preprint arXiv:2011.11922 , 2020.

Jiachen Sun, Yulong Cao, Christopher B Choy, Zhiding Yu, Anima Anandkumar, Zhuoqing Morley Mao, and Chaowei Xiao. Adversarially robust 3d point cloud recognition using selfsupervisions. Advances in Neural Information Processing Systems , 34:15498-15512, 2021.

Jiachen Sun, Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Dan Hendrycks, Jihun Hamm, and Z Morley Mao. A spectral view of randomized smoothing under common corruptions: Benchmarking and improving certified robustness. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IV , pages 654-671. Springer, 2022.

Jiachen Sun, Weili Nie, Zhiding Yu, Z Morley Mao, and Chaowei Xiao. Pointdp: Diffusiondriven purification against adversarial attacks on 3d point cloud recognition. arXiv preprint arXiv:2208.09801 , 2022.

Jiachen Sun, Qingzhao Zhang, Bhavya Kailkhura, Zhiding Yu, Chaowei Xiao, and Z Morley Mao. Benchmarking robustness of 3d point cloud recognition against common corruptions. arXiv preprint arXiv:2201.12296 , 2022.

Jiachen Sun Sun, Yulong Cao Cao, Qi Alfred Chen, and Z Morley Mao. Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In USENIX Security Symposium (Usenix Security'20) , 2020.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 , 2013.

Abdel Aziz Taha and Allan Hanbury. Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool. BMC medical imaging , 15(1):1-28, 2015.

Gusi Te, Wei Hu, Amin Zheng, and Zongming Guo. Rgcnn: Regularized graph cnn for point cloud segmentation. In Proceedings of the 26th ACM international conference on Multimedia , pages 746-754, 2018.

Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision , pages 6411-6420, 2019.

Tzungyu Tsai, Kaichen Yang, Tsung-Yi Ho, and Yier Jin. Robust adversarial objects against deep learning models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 954-962, 2020.

James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. Physically realizable adversarial examples for lidar object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 13716-13725, 2020.

Puru Vaish, Shunxin Wang, and Nicola Strisciuglio. Fourier-basis functions to bridge augmentation gap: Rethinking frequency augmentation in image classification. arXiv preprint arXiv:2403.01944 , 2024.

Jacob Varley, Chad DeChant, Adam Richardson, Joaquín Ruales, and Peter Allen. Shape completion enabled robotic grasping. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages 2442-2447. IEEE, 2017.

Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8684-8694, 2020.

Jie Wang, Lihe Ding, Tingfa Xu, Shaocong Dong, Xinli Xu, Long Bai, and Jianan Li. Sampleadaptive augmentation for point cloud recognition against real-world corruptions. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14330-14339, 2023.

Ruibin Wang, Yibo Yang, and Dacheng Tao. Art-point: Improving rotation robustness of point cloud classifiers via adversarial rotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14371-14380, 2022.

Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann. Sgpn: Similarity group proposal network for 3d point cloud instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2569-2578, 2018.

Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) , 38(5):1-12, 2019.

Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer. Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud. In 2019 International Conference on Robotics and Automation (ICRA) , pages 4376-4382. IEEE, 2019.

Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1912-1920, 2015.

Ziyi Wu, Yueqi Duan, He Wang, Qingnan Fan, and Leonidas J Guibas. If-defense: 3d adversarial point cloud defense via implicit function based restoration. arXiv preprint arXiv:2010.05272 , 2020.

Chong Xiang, Charles R Qi, and Bo Li. Generating 3d adversarial point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9136-9144, 2019.

Tiange Xiang, Chaoyi Zhang, Yang Song, Jianhui Yu, and Weidong Cai. Walk in the cloud: Learning curves for point clouds shape analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 915-924, 2021.

Chenxi Xiao and Juan Wachs. Triangle-net: Towards robustness in point cloud learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 826-835, 2021.

Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc V Le. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 819-828, 2020.

Mutian Xu, Runyu Ding, Hengshuang Zhao, and Xiaojuan Qi. Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3173-3182, 2021.

Mutian Xu, Junhao Zhang, Zhipeng Zhou, Mingye Xu, Xiaojuan Qi, and Yu Qiao. Learning geometry-disentangled representation for complementary understanding of 3d object point cloud. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 3056-3064, 2021.

Qiangeng Xu, Xudong Sun, Cho-Ying Wu, Panqu Wang, and Ulrich Neumann. Grid-gcn for fast and scalable point cloud learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5661-5670, 2020.

Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, and Shuguang Cui. Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5589-5598, 2020.

Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning object bounding boxes for 3d instance segmentation on point clouds. Advances in neural information processing systems , 32, 2019.

Jiancheng Yang, Qiang Zhang, Bingbing Ni, Linguo Li, Jinxian Liu, Mengdie Zhou, and Qi Tian. Modeling point clouds with self-attention and gumbel subset sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3323-3332, 2019.

Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 206-215, 2018.

Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhiming Ma. Improved ood generalization via adversarial training and pretraing. In International Conference on Machine Learning , pages 11987-11997, 2021.

Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective on model robustness in computer vision. Advances in Neural Information Processing Systems , 32, 2019.

Lijia Yu, Shuang Liu, Yibo Miao, Xiao-Shan Gao, and Lijun Zhang. Generalization bound and new algorithm for clean-label backdoor attack. In Forty-first International Conference on Machine Learning , 2024.

Tan Yu, Jingjing Meng, and Junsong Yuan. Multi-view harmonized bilinear network for 3d object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 186-194, 2018.

Xiangyu Yue, Bichen Wu, Sanjit A Seshia, Kurt Keutzer, and Alberto L Sangiovanni-Vincentelli. A lidar point cloud generator: from a virtual world to autonomous driving. In Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval , pages 458-464, 2018.

Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision , pages 6023-6032, 2019.

Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems , 30, 2017.

Bohang Zhang, Tianle Cai, Zhou Lu, Di He, and Liwei Wang. Towards certifying l-infinity robustness using neural networks with l-inf-dist neurons. In International Conference on Machine Learning , pages 12368-12379, 2021.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning , pages 7472-7482, 2019.

Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 , 2017.

Jinlai Zhang, Lyujie Chen, Bo Ouyang, Binbin Liu, Jihong Zhu, Yujin Chen, Yanmei Meng, and Danfeng Wu. Pointcutmix: Regularization strategy for point cloud classification. Neurocomputing , 505:58-67, 2022.

Tianhang Zheng, Changyou Chen, Junsong Yuan, Bo Li, and Kui Ren. Pointcloud saliency maps. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1598-1606, 2019.

Boxuan Zhong, He Huang, and Edgar Lobaton. Reliable vision-based grasping target recognition for upper limb prostheses. IEEE Transactions on Cybernetics , 2020.

Hang Zhou, Kejiang Chen, Weiming Zhang, Han Fang, Wenbo Zhou, and Nenghai Yu. Dup-net: Denoiser and upsampler network for 3d adversarial point clouds defense. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1961-1970, 2019.

Shenchen Zhu, Yue Zhao, Kai Chen, Bo Wang, Hualong Ma, et al. { AE-Morpher } : Improve physical robustness of adversarial objects against { LiDAR-based } detectors via object reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24) , pages 7339-7356, 2024.

Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Wei Li, Yuexin Ma, Hongsheng Li, Ruigang Yang, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar-based perception. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.

Yi Zhu, Chenglin Miao, Tianhang Zheng, Foad Hajiaghajani, Lu Su, and Chunming Qiao. Can we use arbitrary objects to attack lidar perception in autonomous driving? In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , pages 1945-1960, 2021.

Yifan Zhu, Yibo Miao, Yinpeng Dong, and Xiao-Shan Gao. Toward availability attacks in 3d point clouds. In International Conference on Machine Learning , 2024.

A Proof

Theorem 1. If f satisfies that: f ( x ) ∈ [0 , M ] for all x , | f ( F -1 ( F ( x ) + α )) -f ( x ) | ≤ ϵ for all x and ∥ α ∥ p ≤ δ , then for any distribution P o and P a satisfying that Was p ( P o , P a ) := (inf u ∈ Π( P o ,P a ) E ( x,z ) ∼ u [ ∥F ( x ) -F ( z ) ∥ p p ]) 1 /p ≤ η , where η < δ , then, with probability 1 -γ , we have:

where { x i } i m =1 are i.i.d. samples from P a .

Proof. Assume u is a joint distribution of P o and P a , such that ( E ( x,z ) ∼ u [ ∥F ( x ) -F ( z ) ∥ p p ]) 1 /p ≤ η . Firstly, by Markov inequality, we have that:

Then, we have that:

Now, let { ( x u i , z u i ) } i m =1 be i.i.d. sampled from distribution u . Then, by Hoeffding inequality, we have that:

(1): with probability 1 -γ/ 4 , there are

which indicates that there are at least m (1 -η p δ p -√ ln (4 /γ ) 2 m ) number of i ∈ [ m ] makes that ∥F ( x u i ) -F ( z u i ) ∥ p ≤ δ ;

(2): with probability 1 -γ/ 4 , there are

(3): with probability 1 -γ/ 4 , there are

Let { x i } i m =1 are i.i.d. samples from distribution P a , then, by Hoeffding inequality, we have that: (4): with probability 1 -γ/ 4 , there are

So, with probability 1 -γ makes that (1), (2), (3) and (4) stand, at this times, we can estimate the E z ∼ P o [ f ( z )] -1 m ∑ m i =1 f ( x i ) , there are:

We get our conclusion.

Our generalization bound can also be extended to Lipschitz neural networks, which are a class of networks with global Lipschitz constants [11, 87].

Corollary A.1. If f satisfies that: f ( x ) ∈ [0 , M ] for all x ∈ [0 , 1] n , | f ( F -1 ( F ( x ) + α )) -f ( x ) | ≤ ϵ ∥ α ∥ p for all x ∈ [0 , 1] n and α , then for any distribution P o and P a in [0 , 1] n satisfying that Was p ( P o , P a ) := (inf u ∈ Π( P o ,P a ) E ( x,z ) ∼ u [ ∥F ( x ) -F ( z ) ∥ p p ]) 1 /p ≤ η , then, with probability 1 -γ , we have:

where { x i } i m =1 are i.i.d. samples from P a , v is any real number greater than 1.

Proof. Assuming u is a joint distribution of P o and P a , and makes that ( E ( F ( x ) , F ( z )) ∼ u [ ∥ x -z ∥ p p ]) 1 /p ≤ η . Firstly, by Markov inequality, we have that:

Then, we have that:

Now, let { ( x i , z i ) } i m =1 are i.i.d. samples from distribution u . Then, by Hoeffding inequality, we have that:

(1): with probability 1 -γ/ 4 , there are

(2): with probability 1 -γ/ 4 , there are

(3): with probability 1 -γ/ 4 , there are

Let { x i } i m =1 are i.i.d. samples from distribution P a , then, by Hoeffding inequality, we have that:

(4): with probability 1 -γ/ 4 , there are

So, with probability 1 -γ makes that (1), (2), (3) and (4) stand, at this times, we can estimate the E z ∼ P o [ f ( z )] -1 m ∑ m i =1 f ( x i ) , there are:

We get our conclusion.

B Supplementary experimental results

In this section, we provide more experimental results. All of the experiments are conducted on NVIDIA Tesla V100 GPUs.

B.1 The performance in terms of mOA and RmCE

In this section, we present full results for corruption overall accuracy (mOA) and relative mCE (RmCE) [41]. The mOA is computed as the average OA over all corruptions. The RmCE quantifies the performance drop compared to a clean test set. We adopt the official baseline DGCNN and

initially calculate the relative corruption error (RCE) for a given corruption type i by averaging over 5 severity levels: RCE i = ∑ 5 l =1 (OA clean -OA i,l ) ∑ 5 l =1 (OA DGCNN clean -OA DGCNN i,l ) , where OA clean is the overall accuracy on the clean test set. Subsequently, we compute the relative mean corruption error (RmCE) by averaging over the 7 corruption types: RmCE = 1 N ∑ N i =1 RCE i . In Tables B.1 and B.2, we compare different methods based on the mOA and RmCE metrics, confirming that our proposed FAT enhances the model's out-of-distribution generalization ability.

B.2 The performance on the ModelNet40-C

In this section, we evaluate the corruption robustness on ModelNet40-C [51]. The ModelNet40C dataset is specifically designed to assess the network robustness against prevalent point cloud corruptions. It consists of 15 different corruption types, including 'Uniform', 'Gaussian', 'Impulse', 'Upsampling', 'Background', 'Occlusion', 'LiDAR', 'Local Density Inc', 'Local Density Dec', 'Cutout', 'Rotation', 'Shear', 'FFD', 'RBF', and 'Inv RBF'. Each type of corruption has 5 severity

levels. In Table B.3, we compare different methods on the ModelNet40-C test set, confirming that our proposed FAT enhances the OOD generalization ability of the model.

In Table B.4, we show the performance of FAT when integrated with data augmentation strategy in terms of mCE and ERcor. In previous studies [51], PCT with CutMix-R achieves the best robustness with the 0.635 mCE and 0.163 ERcor. However, training GDANet with the combination of our proposed FAT with WOLFMix achieves a new state-of-the-art performance, with the impressive 0.555 mCE and 0.147 ERcor.

B.3 The performance on the ScanObjectNN-C

We further conduct experiments on the ScanObjectNN dataset, which is collected by LiDAR sensors and represents more realistic conditions under real-world scenarios [2, 58, 96, 94]. The experimental settings and evaluation metrics on ScanObjectNN-C [62] are consistent with those on ModelNet-C. The results are shown in Table B.5. It can be seen that our FAT generally leads to lower mCE on ScanObjectNN-C. The experimental results on ScanObjectNN-C further validate the generalizability and applicability of our FAT under real-world conditions.

B.4 The performance on the PointNeXt

We further conduct experiments for the updated point cloud model PointNeXt [37]. The results in Table B.5 and Table B.6 demonstrate that FAT achieves consistent performance on the advanced network architecture PointNeXt, similar to observations on PointNet and more. Our FAT outperforms all other methods in terms of mCE. This indicates that FAT's performance is largely independent of the underlying model architecture, making it applicable to both traditional and modern networks.

B.5 The performance in combination with AdaptPoint

We further conduct experiments for comparison with AdaptPoint [62] on ScanObjectNN-C. AdaptPoint follows the official experimental settings. The results are shown in Table B.7. It is evident that incorporating FAT achieves a lower mCE, indicating its superiority.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our paper supports the claims made in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed limitations in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have provided the full set of assumptions in every theorem and made a complete proof in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have provided reproductive details in Section 4.1 and Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided our codes in the supplemental matrial.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have provided experimental details in Section 4.1 and Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: For fair comparison, we do not provide error bars because there are many baseline methods, it is computationally expensive to reproduce all of these methods.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Justification: We have provided them in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our paper conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed them in Appendix 5.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We use open-source dataset and models in our paper, and have cited the original paper of these dataset and models.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Our paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a method called Frequency Adversarial Training (FAT) to improve the robustness of 3D point cloud recognition models and examines the robustness of models under 3D point cloud corruptions, including analysis on the power of different corruption effects in the frequency domain.  FAT generates adversarial samples by adding perturbations to the frequency representations of point cloud data. The authors also provide a theoretical analysis demonstrating the effectiveness of FAT in improving  OOD generalization performance of models.\n\nSTRENGTHS:\n1. The paper is well-structured and organized, making it easy to understand and follow. \n2. Introduces a novel concept of frequency augmentation for 3D point cloud data, while previous works  are mainly mix-based and deformation-based on original point cloud data rather than their frequency representation.\n3. The proposed method considers distribution shifts caused by adversarial perturbations and thus uses separate batch normalization layers,    improving model robustness against both low-frequency and high-frequency corruptions. The authors also provide an ablation study demonstrating the effectiveness of each component in improving robustness.\n4. The authors provide theoretical analysis showing that adversarial robustness in the frequency domain enhances real-world corruption robustness, and a frequency sensitivity measurement which offers valuable insights for analyzing model robustness from a frequency perspective.\n\nWEAKNESSES:\n1. There is confusion in the caption of Figure 3, which suggests the Jacobian matrix for an input point cloud. According to the context, it should be the Jacobian matrix of the model regarding the input point cloud.\n2.  As FAT is based on adversarial training, it is expected to have comparisons with other adversarial training techniques rather than only mixing- and deformation-based approaches. \n3. Minors: \n*The figures are hard to read as the font size is too small.\n*Missing space before the fourth sentence in the caption of Figure 3.\n\nQUESTIONS:\n1. Adversarial augmentation approaches may require substantial computational resources, leading to inefficiency. How much computational resources does FAT require regarding computational time, memory usage, etc., and how does it compare with other augmentation approaches.\n\nLIMITATIONS:\nThe authors address the limitation of their approach in term of the degraded standard accuracy. There might be a trade-off between corruption robustness and standard performance, left for future investigation.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Frequency Adversarial Training (FAT), leveraging Graph Fourier Transform (GFT) to enhance robustness against point cloud corruptions by training models with frequency-domain adversarial examples, demonstrating significant improvements in robustness across various architectures through extensive experiments.\n\nSTRENGTHS:\n1.\tInnovative Approach: The use of the frequency domain for analyzing and improving the robustness of point cloud recognition models is novel and well-motivated. The application of GFT to understand the impact of corruptions on different frequency bands is a significant contribution.\n2.\tComprehensive Evaluation: The paper provides a thorough evaluation of the proposed method, comparing it with existing approaches and demonstrating its effectiveness across multiple models and datasets.\n3.\tTheoretical and Empirical Validation: The authors provide both theoretical analysis and empirical evidence to support the effectiveness of FAT, strengthening the credibility of their claims.\n4.\tPractical Relevance: Improving the robustness of 3D point cloud recognition models is highly relevant for safety-critical applications such as autonomous driving and robotics.\n5.\tWriting quality: This paper is written and organized well\n\nWEAKNESSES:\n1.\tComplexity of Implementation: The proposed FAT method involves several complex steps, including the generation of high-frequency and low-frequency adversarial examples and the use of multiple batch normalizations. This complexity might hinder the adoption of the method in practical scenarios.\n2.\tLimited Impact on Clean Accuracy: The paper mentions a slight reduction in clean accuracy when using FAT, which might be a concern for applications where both robustness and accuracy are critical.\n3.\tGenerality of Results: While the experiments demonstrate the effectiveness of FAT on several models and datasets, additional experiments on more diverse datasets and real-world scenarios could further validate the generalizability of the approach, e.g.ShapeNet-C[1] and ScanObjectNN-C[2] \n4.\tLack of Updated Baselines in Experiments: In Table 1, the paper lacks experiments comparing some updated point cloud baselines, like RPC[3] and PointNeXt[4], which could provide a more comprehensive evaluation of the proposed method.\n5.\tMissing Comparison with Updated Augmentation Methods: In Table 2, the paper lacks comparison with updated augmentation methods such as AdaptPoint[2], which could highlight the relative performance of FAT against newer augmentation techniques.\n\n[1]: PointCloud-C: Benchmarking and Analyzing Point Cloud Perception Robustness under Corruptions\n[2]: Sample-adaptive Augmentation for Point Cloud Recognition Against Real-world Corruptions\n[3]: Benchmarking and Analyzing Point Cloud Classification under Corruptions\n[4]: PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies\n\nQUESTIONS:\nRefer to the Weakness. I will improve my rates when the weaknesses are solved.\n\nLIMITATIONS:\nRefer to the Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to enhance the robustness of 3D point cloud recognition. The authors propose Frequency Adversarial Training (FAT) to improve the corruption robustness of 3D point cloud recognition models. FAT trains a model with adversarial examples that add perturbations to the frequency-domain representations of point clouds.\n\nSTRENGTHS:\n- The problem studied in this paper is important. \n\n- The authors propose generating the adversarial sample in the frequency domain, which is interesting.\n\n- This paper considers several baseline methods.\n\nWEAKNESSES:\n- This paper considers limited real-world applications. 3D point cloud recognition often fails due to LiDAR sensor inaccuracies and changes in the physical environment. However, this paper only utilizes one dataset, ModelNet40, where points are generated via 3D modeling techniques instead of being collected from real-world sensors, which is not convincing. If the paper aims to address real-world problems, it would be more convincing to include more real-world datasets, such as the KITTI dataset, where point clouds are collected from LiDAR sensors. \n\n- The defense performance does not significantly outperform baseline methods. As shown in Table 1, FAT only outperforms baseline methods by a few points. Moreover, the proposed method appears to be effective only when combined with other data augmentation techniques. Could the authors provide more details and an explanation of the phenomenon of the ‘surprising results’ mentioned in the paper? For example, why does mixing methods result in much better defense?\n\n- This paper does not analyze the relationship between the frequency and spatial domains in the context of the 3D point cloud. The two domains are mutually transformable, but the paper does not provide insights into their relationships or general observations.\n\n- The paper only studies the defense effects against general 3D data corruptions without considering specific attack methods. It remains unclear if this approach can also defend against specific attack methods targeting 3D perception, such as attacks using adversarial points with specific shapes, locations, and rotation angles. The following are a few works about attacks against LiDAR object detection in autonomous driving scenarios [1-4] for reference.\n\n[1] Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. 2019. Adversarial objects against lidar-based autonomous driving systems. arXiv preprint arXiv:1907.05418 (2019).\n[2] James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. 2020. Physically realizable adversarial examples for lidar object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.\n[3] Yi Zhu, Chenglin Miao, Tianhang Zheng, Foad Hajiaghajani, Lu Su, and Chunming Qiao. 2021. Can we use arbitrary objects to attack lidar perception in autonomous driving?. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security.\n[4] Shenchen Zhu, Yue Zhao, Kai Chen, Bo Wang, Hualong Ma, and Cheng’an Wei. 2024. AE-Morpher: Improve Physical Robustness of Adversarial Objects against LiDAR-based Detectors via Object Reconstruction. In Proceedings of the 33rd USENIX Security Symposium.\n\nQUESTIONS:\nQ1: As mentioned earlier in your paper, the corruption of 3D point clouds often occurs in real-world applications due to environmental noise and reflections. For instance, in autonomous driving scenarios, the LiDAR sensor may only capture partial points of the target vehicle because of occlusion and distance effects. However, this paper does not study those real-world data. I am curious whether your method would be effective on real-world datasets, such as the KITTI and Waymo driving datasets. Solely using ModelNet40 is not convincing.\n\nQ2: Can the proposed approach maintain good performance when considering specific attack methods, such as those using adversarial points with specific shapes, locations, and rotation angles?\n\nQ3: Modifications in the frequency domain can sometimes result in physically unrealizable transformations in the spatial domain. Will this limit the application of your work in real-world scenarios?\n\nQ4: Can the authors provide more details on the reasons for using GFT rather than DCT or other frequency transformation methods?\n\nQ5: What would be the outcome if medium frequency bands were modified? The proposed methods seem to only consider high and low frequencies.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach to improving the robustness of 3D point cloud recognition models by introducing Frequency Adversarial Training (FAT). By analyzing the frequency space of point clouds through the graph Fourier transform, the authors found that models are sensitive against different frequency bands of corruptions. To reduce the effect of both low and high frequency corruptions, the authors propose FAT, which introduces adversarial examples during training that were generated by perturbing the point cloud’s frequency space and taking the inverse graph Fourier transform. From the authors’ experiments, FAT significantly improved the robustness of point cloud recognition models and was able to achieve a new state-of-the-art performance.\n\nSTRENGTHS:\nThe authors introduce an original analysis of point cloud recognition models by quantifying the sensitivity of these models against low-frequency disruptions, such as rotations, and high-frequency disruptions, such as jittering. Frequency adversarial training is also an innovative approach to improving the robustness of these models against these disruptions, utilizing the graph Fourier transform to adversarially train against low and high-frequency distortions within the frequency space. The writing, extensive experiments, and theoretical analysis provided by the authors effectively and clearly show the positive impact FAT has on the robustness of point cloud recognition models. The use of a point cloud’s frequency space and application of FAT have the capacity to have significant impacts on future research in point cloud models beyond 3D recognition models.\n\nWEAKNESSES:\nWhile the paper mentions the importance of robust models for real-world applications, it exclusively evaluates models trained with FAT on synthetic datasets like ModelNet-C and ModelNet40-C. Evaluating FAT using data from sensors such as LiDAR would further show FAT’s applicability under real-world conditions (such as KITTI dataset, if possible). Additionally, the paper does not discuss the complexity and efficiency of FAT, which are important factors for real-world applications.\n\nQUESTIONS:\nIt will be great to see some results on real-world datasets such as KITTI.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-11"}