Point Cloud Dataset Distillation
Authors: Deyu Bo
Xinchao Wang
Venue: ICML 2025 poster (2025)
Keywords: Dataset Distillation
Point Cloud
Primary area: applications->computer_vision
TL;DR: A rotation-invariant dataset distillation method for point cloud datasets

Abstract: This study introduces dataset distillation (DD) tailored for 3D data, particularly point clouds. DD aims to substitute large-scale real datasets with a small set of synthetic samples while preserving model performance. Existing methods mainly focus on structured data such as images. However, adapting DD for unstructured point clouds poses challenges due to their diverse orientations and resolutions in 3D space. To address these challenges, we theoretically demonstrate the importance of matching rotation-invariant features between real and synthetic data for 3D distillation. We further propose a plug-and-play point cloud rotator to align the point cloud to a canonical orientation, facilitating the learning of rotation-invariant features by all point cloud models. Furthermore, instead of optimizing fixed-size synthetic data directly, we devise a point-wise generator to produce point clouds at various resolutions based on the sampled noise amount. Compared to conventional DD methods, the proposed approach, termed DD3D, enables efficient training on low-resolution point clouds while generating high-resolution data for evaluation, thereby significantly reducing memory requirements and enhancing model scalability. Extensive experiments validate the effectiveness of DD3D in shape classification and part segmentation tasks across diverse scenarios, such as cross-architecture and cross-resolution settings.

Point Cloud Dataset Distillation
Deyu Bo"
Xinchao Wang"
Abstract
significantly reduces the computational cost of training neu- ral networks from scratch. Due to its remarkable efficiency and effectiiveness, DD has been used in variety of do- mains, such as image (Zhao et al., 2021; Zhao & Bilen, 2023; Cazenavette et al., 2022), video (Wang et al., 2024), graph (Jin et al., 2022: Liu et al., 2024), etc.
This study introduces dataset distillation (DD) tai- lored for 3D data, particularly point clouds. DD aims to substitute large-scale real datasets with a small set of synthetic samples while preserv- ing model performance. Existing methods mainly focus on structured data such as images. How- ever, adapting DD for unstructured point clouds poses challenges due to their diverse orientations and resolutions in 3D space. To address these challenges, we theoretically demonstrate the im- portance of matching rotation-ing invariiat features between real and synthetic data for 3D distillation. We further propose a plug-and-ple rotator to align the point cloud to a canonical orientation, facilitating the learning of rotation- invariant features by all point cloud models. Fur- thermore, instead of optimizing fixed-size syn- thetic data directly, we devise point-wise gener- ator to produce point clouds at various resolutions based on the sampled noise amount. Compared to conventional DD methods, the proposed ap- proach, termed DD3D, enables efficient training low-resolution point clouds while generating high-resolution data for evaluation, thereby sig- nificantly reducing memory requirements and en- hancing model scalability. Extensive experiments validate the effectiveness of DD3D in shape clas- sification and part segmentation tasks across di- verse scenarios, such as arrhttarc and cross-resolution settings.
Despite great progress, existing DD methods have primarily been applied to structured ID and 2D data, while the dis- tillation of unstructured 3D data, e-g., point cloud, remains largely unexplored. Point cloud data exists in large quan- tities in machine vision. For example, MVPNet (Yu et al., 2023) scans more than 87K point clouds from real-world videos, and Objaverse-XI. (Deitke et al., 2023) provides more than 10M high-quality 3D assets. Training on such datasets from scratch is time- and resource- resource-intensive, high- lighting the need for more efficient alternatives.
point cloud
However, extending DD to 3D point clouds presents unique challenges. First, point clouds with different orientations represent the same semantic information, e.g., shapes. How- ever, existing DD methods do not take the symmetry of data into account, which cannot handle the randomly rotated data and result in sub-optimal performance. As shown in Figure (a), directly applying DD to the point clouds with different orientations cannot obtain meaningful synthetic data. Second, point clouds have flexible resolutions, i.e., the number of points, depending on specific models and applications. Generally, a larger resolution encodes more fine fine-grain information but also increases the computational costs (Huang et al., 2024; Qiu et al., 2021). Existing DD methods initialize the synthetic data as a fixed-size tensor, which cannot be customized for different point cloud mod- els. Moreover, the memory budget for fixed-size tensors will increase rapidly when dealing with dense scenes, e.g., segmentation (Chang et al., 2015).
1. Introduction
Dataset distillation (DD) (Wang et al., 2018) aims to distill the knowledge of large-scale dataset into a few synthetic samples, where the models trained on the real and synthetic data will have comparable performance. By doing so, DD
Once the weaknesses of existing methods are identified, is natural ask: How can we build a distillation frame- work that overcomes the orientation and resolution issues of 3D point clouds? To answer this question, we first theo- retically prove that random rotations weaken the principle components of real data, thereby degenerating the distil- lation performance. Based on this discovery, we propose DD3D, the first DD framework for 3D point clouds, illus- trated in Figure 1(b). Specifically, DD3D first uses rotator to convert the point cloud into canonical orientation by
National University of Singapore. Correspondence to: Xin- chao Wang <xinchao@ Sainnhao@nnu.cduun edu.cdu.sg>.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Point Cloud Dataset Distillation
(256 Points)
Distill
Rotate
Distill
Generate
(512 Points)
(Generator)
(1024 Points)
Match
Match
(a) DD for point clouds
(b) DD3D for point clouds
Figure 1: Differences between vanilla DD and DD3D when distilling 3D point clouds.
learning a rotation- eeuiivrraat projection matrix to offset random rotation. Then, the knowledge of rotation-invarian data is distilled into a point-wise generator to predict the point coordinates from noise, where the resolution is based on the number of sampled noises. Finally, the rotator and generator are jointly optimized by minimizing the gradient differences between the real and synthetic data.
et al., 2023; Zhang et al., 2024a), which can generate diverse synthetic data with unlimited samples. Although valid, these methods rely on the prior knowledge of generative models pre-trained on large-scale datasets, which not feasible for point clouds. A recent work' also applies GM to point cloud data. However, neither of them considers the orientation and resolution issues. For more detailed introduction to DD, please refer to the recent surveys (Yu et al., 2024b; Lei & Tao, 2024; Geng et al., 2023; Sachdeva & McAuley, 2023).
The contributions are summarized as follows. (1) We pro- pose the first 3D distillation framework, DD3D, which can eliminate the influence of random rotations and synthesize point clouds at arbitrary resolutions. (2) We theoretically prove that matching the conation-invariant features can pre- serve the principal components of real data and prevent data degeneration. (3) DD3D can be trained with low-resolution point clouds and generates high-resolution data for evalua- tion, significantly reducing memory usage and enhancing model scalability. (4) Extensive experiments on shape clas- sification and part segmentation tasks validate the effective- ness of DD3D over baselines.
Point Cloud Analysis. Deep learning on point clouds plays a vital role in 3D data analysis (Guo et al., 2021b). Tradi- tional methods can be classified into three categories: Point- based methods, e.g.. PointNet (Qi et al., 2017a) and Point- Net++ (Qi al., 2017b), convolution- based methods, e.g., PointCNN (Li et al., 2018) and PointConv (Wu et al., 2019), and relation- methods, egg., DGCNN (Wang et al., 2019) and Point Transformer (Guo et al., 2021a). However, these methods are rotation- sensitive and cannot handle point clouds with different orientations. Some advanced meth- ods are designed to learn rotation-equivarrint or invariant features, such as vector neuron (Deng et al., 2021), spheri- cal harmonic (Poulenard et al., 2019), tensor field (Thomas et al., 201 (0)8 and graph features (Kim et al., 2020; Zhao et al., 2019). However, these methods introduce additional operators and cannot be applied to rotation ods. Another way to project point clouds into the same orientation. For example, principal component analysis (PCA) leverages the eigenvectors of the covariance matrix to transform point clouds into the direction with maximum variance. But this approach suffers from the sign- aebiguiiyy issue (Xiao et al., 2020; Yu et al., 2020; Li et al., 2021).
2. Related Work
Dataset Distillation. Research on DD can be roughly di- vided into two directions. The first is to explore advanced matching objectives to improve the distillation performance. For example, performance matching (Wang et al., 2018), gradient matching (Zhao et al., 2021; Zhao & Bilen, 2021), distribution matching (Zhao & Bilen, 2023; Wang et al., 2022), trajectory matching (Cazenavette et al., 2022: Guo et al., 2024; Du et al., 2023) and feature regression (Zhou et al., 2022; Loo et al., 2022; Nguyen et al., 2021). On the other hand, some methods innovate efficient data parameter- izations to avoid directly optimizing the synthetic data. For example, neural networks (Liu et al., 2022), spectral repre- sentation (Shin et al., 2023), linear transformation (Deng & Russakovsky, 2022), and up-sampling (Kim et al., 2022). Among them, special parameterization technique is to dis- till the knowledge into generative model (Zhao & Bilen, 2022; Wang et al., 2023: Zhang et al., 2023; Cazenavette
meth-
3. Preliminary
Task Formulation. Suppose that T = {(C,, 9: 1 is large-scale training dataset, where C, is a point cloud with label yı for the shape classification task. Each point
'https://githie com/ /ghdaadurodddddd
2

---

Point Cloud Dataset Distillation
cloud has points, represented as C = (P.V) V). where P R represents the 3D coordinates of points and V E ga indicates the part to which the point belongs in segmentation task and v is the number of parts. The goal of DD3D is to synthesize a much smaller point cloud dataset S {C,, wy) where |S] < IT], such that classification or segmentation model So trained on T and S will have comparable performance. Other tasks, such as detection, are left for future studies.
Theorem 3.2 states that random rotations reduce the max- imum singular value of the data representations, implying that the principle component of X is weakened. In this case, the synthetic data cannot effectively capture the dis- tribution of the real data, degenerating model performance. All proofs can be seen in Appendix A.
4. The Proposed Method
4.1. Plug-and-Play Point Cloud Rotator
Dataset Distillation. In order to effectively optimize the synthetic data, existing DD methods adopt a bi-level opti- mization paradigm, which can be formulated as:
Our analysis highlights the importance of learning rotation- invariant representations for effective point cloud distillation. However, many existing point cloud models lack this capa- bility. To address this limitation, we introduce a plug-and- play point cloud rotator that transforms point clouds into a consistent canonical view. This transformation ensures that all models can learn potation-invarian representations, enhancing their generalization and performance.
min CDD (for fer
(1)
S
s.t. 0* = min Cate (folS)) yS),
(2)
where the inner loop updates the model fo on the synthetic data, and the outer loop optimizes the synthetic data. In particular, LDD is a metric that measures the distance be- tween real and synthetic data. For example, gradient match- ing (Zhao et al., 2021) minimizes the gradient differences.
Rotation-equivarianee. We leverage the orthogonality of the rotation matrix to eliminate its influence, i.e., RR I, where PCA a typical method:
Dataset Distillation with Rotations. Before detailing the proposed method, we first give a general analysis of how rotations affect the performance of DD. Let Xs E xd denote the representations learned
1 Σ (PR-PR) T (PR-PR)- = UAU R (7)
where P is the center of P and U represents the eigen- vectors of the covariance matrix. Importantly, the pro- jection R R maintains equivariance with respect to co- ordinate rotations, ensuring (PR) U) = PU remains invariant. However, eigenvectors suffer from sign am- biguity, implying that both u, and -u, are valid solu- tions. As result, the canonical view PU is not unique and has 8 ambiguities in 3D space (Xiao et al., 2020; Yu et al., 2020). i.e., PUQ = P [ Lug Lug], where - -1 Qij 0, vi +
R| ,X7 E | by fo on the synthetic data and real training data, respec- tively, and d is the hidden dimension.
Theorem 3.1. Assume the classifier is a linear layer W and Lats can be simplified to the mean-squared error ||W - -II The objective of gradient matching is equal to variance preserving:
min LaM = -innn D (Vw CS. WWC...
(3)
S
{Q E 3x33 Qu =
min S
|X]Xs
XI
(4)
Sign-invariant. Our proposed rotator r: =x + Rax3 is designed to enhance PCA by addressing the sign ambiguity issue. To achieve this, the rotator learns a sign-equivariant reflection matrix Q for each point cloud. This ensures that the transformed representation satisfies PUQ Q PU, making it sign-invariant and improving the robustness of ination-invariane learning. Specifically, the rotator first lifts the scalar coordinates to the vector representations:
where 'D is a distance metric and V is the gradient with respect to W.
Theorem 3.1 reveals that synthetic data preserves the vari- ance information of real data. We then analyze how random rotations affect the variance of real data. Without loss of generality, we assume that fo is rotation-equivariann i.e., fe (PR) = fe (P) R, where R e SO(d) is a random rota- tion matrix.
H = [sin PU), sin) ... (in Pin)++ = sinn((U))) (in(PPPP)).sss((PP) Q.
(8)
Theorem 3.2. Assume XT follows a d-dimensional mul- tivariate Gaussian distribution N N(p, E). Let Xt be the rolated representations of X such that:
where sin(-) is the sine function and f is the period of Fourier features. An average pooling is then applied on H to learn the representations of the whole point cloud. Finally, a learnable vector E TR' is used to decode the reflection matrix. The overall architecture of the rotation is formulated as follows:
Amaz
E X
x+)) & Amaz
Xx [x xx))
(5)
Tmax
[x1+ < Jmax (EE [XT]),
(6)
where Amaz and Tmax are the maximum eigenvalues and singular values, respectively.
r(P) = PUQ Q = PUQ Sign(w -Pol( (Q)), (9)
3

---

Point Cloud Dataset Distillation
Engine
Tail
Partition
Wing
(Generator)
Noise
Labels
Fuselage
Figure 2: DD3D for part segmentation task. Each noise is first pee-paattiooed into different parts according to its value, e.g., the noise within (0, 0.45) is marked as fuselage. Then the generator maps the noise into different parts for gradient matching.
where Sign means the signs of a matrix. The reflection matrix Q has the same signs as Q because the sinusoidal features, pooling function, and linear decoder preserve the sign information of HQQ, which can solve the sign ambiguity and learn sign-invariant representations.
( o 22 0... 1.. Wp, D = sinn(;w; + b:), (10)
where L is the number of layers, denotes the cascade of neural networks, D, is multi-layer perceptron (MLP) with sine activation function in the i-th layer, and Wp E Rdx3 is the decoder to generate 3D point coordinates.
Alternative Approaches. Several methods (Zhang et al., 2024b; Yu et al., 2024a; Melnyk et al., 2024; Li et al., 2022; Xu et al., 2021) have been proposed for learning rotation- invariant representations, such as vector neurons (Deng et al., 2021) and graph-based features (Kim et al., 2020). However, these approaches modify the original point coor- dinates, making it difficult to integrate with existing models. Another line of work addresses the sign ambiguity issue using pooling (Yu et al., 2020) and attention mechanisms (Xiao et al., 2020; Li et al., 2021). While effective, these methods are computationally expensive, as they require eval- uating representations across all possible ambiguous views.
Conditional Modulating. While the point-wise genera- tor synthesize point clouds at arbitrary resolutions, it lacks class-conditionn erate category srregory-specific data. To address this, we introduce modulator c : Rd + Rd, which is implemented as an- other MLP V to encode the label information and generate conditions for the point cloud generation:
control, limiting its ability to gen-
= v o U o LL, Vr = ReLU (many b)), (1I)
where ReLU(-)) max(0, -), m, E IRd denotes the condi- tional representations. The first layer input, m1, is one-hot matrix encoding class labels. Assume that there are K classes in total, and each class has N synthetic samples, then m, RKN and w/ KNxd
4.2. Point-wise Generator
Beyond rotation alignment, point cloud distillation must also account for the variations in resolution. Unlike images, point clouds do not have a fixed structure, making traditional DD methods, which directly optimize fixed-size tensors, unsuitable. To solve this issue, a promising solution is to parameterize data with implicit neural representation (INR), which has been widely used to generate data at arbitrary resolutions (Sitzmann et al., 2020; Park et al., 2019; Chen et al., 2021; Singh et al., 2023).
The learned conditional representations are then used to modulate each layer of the generator, adjusting the fre- quency and phase features dynamically. The complete ar- chitecture is as follows:
[ o +1) (V2 O 12...........................).. .... (V, C 21). W.
(12)
where @ is the dlement-wise multiplication. For clarity, in the following sections, we use g(c. k)) to denote O c with the k-th condition.
Point Denoising, Our solution is to use INR a point-wise generator g IR. R33 which takes random noise input and predicts its corresponding 3D coordinates. Therefore, the number of points is the same as the sampling noise, which enables us to achieve low-resolution training and high-resolution evaluation, thus significantly reducing the computational costs and memory budget. See Section 5.6 for more details. For implementation, we choose SIREN (Sitz- mann al., 2020) as generator, which formulated
Noise Distribution. For noise sampling, we use uniform distribution instead of Gaussian distribution. The reasons are two-fold. First, INR requires inputs to be normalized within [0, 1]. which aligns naturally with the uniform distri- bution. Second, in the part segmentation task, each point must be assigned a label beforehand. A uniform distribution enables a straightforward division of noise samples accord-

---

Point Cloud Dataset Distillation
ing the category ratio, ensuring balanced representation across different parts. See Figure 2 for intuitive example.
Algorithm DD3D for part segmentation
Input: Training dataset T Ouput: Model f, Rotator Generator g repeat
4.3. Distillation Tasks
for k: 1 K do
To comprehensively validate the effectiveness of DD3D, we conduct experiments on both the basic shape classifica- tion task and the challenging part segmentation task. Shape classification aims to assign each point cloud a label, empha- sizing global information, while part segmentation predicts the label of each point, which is more ane-ggrined.
Sample a batch BT V ~ Sample noise ~ (O,,1)
Generate V M by partitioning noise Generate point clouds B =g(e, k) for p E k do
Apply mask M MM on B BY Compute VLS. and VLT end for end for
Shape Classification, The distillation objective of the shape classification task is defined as:
K
Lshape
k), YE,,
Update with Lyart repeat
(13)
V.cieffo (L YL
Update f, r with L until inner-loop end until outer-loop end
where K is the total classes of shapes, BT and Y are batch of real training data and labels.
Part Segmentation. In the part segmentation task, each shape is divided into multiple parts. For example, an air- plane can be divided into its fuselage, wings, engines, and tail. Assigning these fine-graine fine-grained labels before distillation helps stabilize the training process. Therefore, DD3D first partitions the noise into different segments based on its value. Then, the partitioned noise is fed into the generator and rotator to produce synthetic data. To effectively capture fineggrimed details, DD3D aligns the gradients of each seg- ment individually, rather than simply matching the gradients of the entire shape. This improves the preservation of local geometric features while maintaining overall structural co- herence. A conceptual illustration is provided in Figure 2. The distillation of part segmentation task is formulated as:
foreground and background points. Second, the detection task requires learning continuous bounding box coordinates, which cannot be predefined like segmentation labels, adding another layer of complexity.
5. Experiments
We benchmark our method on two fundamental tasks of point cloud analysis: shape classification (Section 5.1) and part segmentation (Section 5.2), followed by a series of analyses, including generalization (Section 5.3), ablation (Section 5.4), and visualization (Section 5.5).
Datasets. We employ three datasets of different scales for the shape classification task: (i) ScanObjectNN (OBJ BG) (Uy et al., 2019) is the smallest dataset but con- sists of real-world data, which is challenging to distillate. (ii) ModelNet40 (Wu al., 2015) larger synthetic dataset generated from CAD models. (iéi) MVPNet (Yu et al., 2023) is the largest dataset, containing 87K point clouds scanned from real-world videos. We use its sub- set MVPNet100, which includes data from the 100 most populous categories, to alleviate the influence of long-tail distribution, similar to the CAFIR-10M dataset. For the part segmentation task, we follow Qi et al. (2017a) and choose ShapeNet-part (Yi et al., 2016) dataset for evaluation. All the datasets use the standard data splits, and their detailed statistic information can be found in Appendix C.
K
Lpart=
Σ (o OM) M VS),
1= PEk
(14)
where p E k indicates parts belonging to shape, VT Vs represents the real and synthetic part labels, and MS M are the part-specific mask extract gradients corresponding to each part. See Algorithm 1 for detailed descriptions.
4.4. Discussion
DD3D has demonstrated strong potential in capturing both global shape structures and local details in object-level point clouds. Visualizations in Section 4 further illustrate its ef- fectiveness. However, applying DD3D to scene-level tasks, such as object detection, remains challenging. This lim- itation can be attributed to two key factors: First, scene- level tasks often involve significant imbalance between
Data Preparation and Metrics. Each cloud contains 1,024 points and is normalized into a unit sphere. We consider two settings: Aligned and Rotated. In the Aligned setting, both training and test point clouds have the same orienta-
5

---

Point Cloud Dataset Distillation
Table 1: Shape classification results of different methods, mean accuracy (%) + standard deviation. Bold indicates the best performance, and "_" means out- -o-moorommry during distillation. CPC: Number of Clouds Per Class.
Coreset-based
Distillation-aase
Full Dataset
Dataset
CPC
Random
Herding
K-Center
GM
DM
TM
DD3D
22.00+2.56
16.29=1.37
18.18:1.04
26.34+2.07
25.90=1.34
26.4242.08
3.76
ScanObjeclNN (Aligned)
32.63=1.51
31.94+3.31
33.46#1.46
39.87=3.00
37.61+2.78
36.44+2.74
43.77=2.63
66.96
54.17=1.77 50
51.70+1.87
54.22+1.30
57.52+2.03
56.91=1.17
61.96#1..4
14.90+2.10
18.10#1.55
19.9112.16
14.6423.04
18.74*2.44
19.2943,90 90
23.59+2.17
ScanObjecINN
10
1..50:1.22
20.20+2.19
22.05=1.76
20.55=3.99
20.26+4.31
19.20+4.52
25.84+3.11
54.84
(Rotated)
50
1..98+1.84
13.39&1.34
44,2942.07
1..74
48. 11+..30
50.26*11.42
40.53+0.36
43.41+0081
43.90#1.51
53.38=0.86
53,21
52.37=0.99
53.822+0.28
ModeINet40 (Aligned)
10
01.8940.29
74.6320.48
73.1340.78
75.45=0.82
74,45+0.47
75.39
76.31#0.44
88.05
50
82.37=0.45
82.75+0.49
82.7340.28
01.74
83.02#1.16
83.91#0.23
34.65.0.71
30.03=1.42
30.0540.50
1.93
1..7
37.3642.98
42.3620.83
ModelNee40
10
58.87=0.65
56.03+0.62
57.6940.97
55 55.69=1.63
55,45# 1.00
56.21=1.14
1..14
80.45
(Rotated)
70.13 50
70.02+0.71
69.68+0.59
68. 68.92+0.73
69.31=0.79
71.27=0.32
5.21
8.14
8.41
00.5220.83
11.73+0.49
10.7440.57
13.68+0,48
MVPNet100
10
15.99=0.30
22.11+0.21
25.68=0.77 20.54+0.21
25.71+0.69
31.14+1.31
55.63
50
10.1440.27
35.87+0.24
35.48+0.44
37.41=0.57
36.83=0.20
40.61+0.38
Note: All methods with rotated data are trained with the point cloud rotator. Ablations can be seen in Table 4.
Table 2: Part Segmentation results (%) on ShapeNet dataset.
makes trade-off between time and memory consumption. See Appendix D for the detailed hyperparameters.
Ratio
Method
OA
Instance loU
Class loU
Backbones. We provide a lightweight PointNet as the back- bone, which abandons the transformation network because previous literature (Yu et al., 2024b) pointed out that com- plex network architecture may lead to degraded distillation performance. In the evaluation stage, we adopt various ad- vanced backbones to evaluate the generalization ability of distilled datasets, including PointNet++ (Qi et al., 201 20),,), DGCNN (Wang et al., 2019), Point Transformer (Guo et al., 2021a). PointMLP (Ma al., 2022), and PointNext (Qian et al., 2022). Results can be found in Table 3.
Coreset
61.24
48.21
31.61
CPC=1
GM
65.56
50.98
33.96
DD3D
73.06
60.27
37.73
Coreset
77.78
65.03
48.19
CPC=10
GM
78.32
65.79
49.88
DD3D
80.37
66.70
50.59
100%
Full
90.04
77.38
65.63
Experimental Setup. For each method, we perform the dis- tillation process twice, evaluate each synthetic point cloud dataset five times (10 results in total), and report the mean and standard deviation. Baselines are all initialized with original data, while DD3D trained from scratch. For the shape classification task, we consider three different distilla- tion ratios with 1, 10, and 50 synthetic point clouds per class (CPC). For the part segmentation task, we choose CPC=1 and CPC=10 due to the limitation of GPU memory.
tion, while in the Rotated setting, both training and test data are rotated randomly. For the rotated data, we project them along the direction of maximum variance during pre- processing. Note that the point clouds in MVPNet only have 180 views, so do not randomly rotate them. The details of pre-processing can be found in Appendix C. We report the Overall Accuracy (OA, %) of each method in the shape classification task and the average class intersection of union (IoU, %) in the part segmentation task.
5.1. Shape Classification
Baselines. To demonstrate the effectiveness of our method, we choose two types of baselines: (1) Coreset-based meth- ods, including Random, Herding (Welling, 2009) and K- Center (Sener& & Savarese, 2018). (2) ietilllation-basee methods, including Gradient Matching (GM) (Zhao et al., 2021), Distribution Matching (DM) (Zhao & Bilen, 2023),
The results of different methods on the shape classification task are shown in Table from which we have the follow- ing observations. Firstly, the results of distillation- based methods consistently outperform coreset-based methods, demonstrating the effectiveness of DD. However, as the amount of synthetic data increases, the performance of the
and Trajectory Matching (TM) (Cazenavette et al., 2022).
We choose GM as the distillation objection for DD3D as it
6

---

Point Cloud Dataset Distillation
Table 3: Cross-architecture results (%) with CPC=50.
Table 5: DD3D under different resolutions.
Dataset
Method
PointNetr+
DGCNN
PCT
PoistMPP
PointNeX:
Resolution
DM
56.02
51.47
52.72
51.33
CPC=50
ScanObjeotNN
GM
55.38
52.98
53.28
51.33
52.81
256
512
1024
Avg.
DD3D
54.04
$2.50
DM
74.35
74.84
76.92
72.49
71.48
ScanObjectNN
61.27
60.59
61.96
61.27
ModelNeed0
76.51
73.38
77.31
74.11
ModelNet40
83.03
83,59
83.91
83,51
DD3D
77.71
75.36
79.21
75.36
73,99
MVPNet100
39.88
40.13
40.61
40.21
DM
33.20
31.26
33.92
32.58
MVPNet100
GM
31.35
29.88
31.43
31.79
30.82
DD3D
34.19
35.82
33,08
Scan ObjectNN
Mid Note0
25.6
256
Table 4: Ablation studies of the point cloud rotator.
Matching Loss
1.0
512
512
1024
1024
ModelNer40 (CPC=50)
Random
GM
DM
DD3D
120
-3
PointNet
14.75
9.47
10.16
17.91
PoinuNet PCA
60.77
53.55
55.57
62.72
PointNet Rotator
70.13
68.92
69.31
71.27
50
Iteration
1:0
150
200
100
200 Iteration
700
4:00
Full Dataset
80.45
Figure 3: Matching losses of different resolutions.
coreset increases rapidly. Secondly, DD3D achieves state- of-the-art performance on all five datasets, demonstrating its superiority over traditional DD methods. Notably, DD3D obtains more improvements over baselines as the number of CPCs increases, possibly because the generator provides more diverse data. Thirdly, the results on the rotated data are weaker than those on the aligned data. Although we project the rotated data to the canonical orientation, i.e., direction with maximum variance, these point clouds still have slightly different orientations, while the aligned data is manually registered, which is strictly towards the direction of gravity and therefore has better performance.
Notably, we use synthetic data with CPC=50 to alleviate the randomness. The results are shown in Table 3, from which we can see that DD3D consistently outperforms DM and GM across different datasets and evaluation methods, proving that the synthetic data distilled by DD3D has better generalizability. This may be attributed the generator that provides various point clouds in each epoch by sampling different noises, which plays role like data augmentation. However, we can also observe that the results of evaluation methods are not as good as PointNet, emphasizing that the synthetic data is still biased by the distillation model.
5.4. Ablation Studies
5.2. Part Segmentation
Point Cloud Rotator. We first verify the effectiveness of the proposed point cloud rotator the rotated ModelNet40 dataset. Specifically, we consider three different models: (1) PointNet, which is cotation-sensitive (2) PointNet + PCA, which is rotation-invariant but sign-variant; (3) PointNet Rotator, which is rotation- and sign-invarians It can be observed from Table 4 that the performance of all methods drops rapidly when the data is randomly rotated. On the other hand, leveraging PCA to transform the point clouds into canonical orientation can significantly improve the distillation performance. However, the results are still far from the model with the point cloud rotator, which reflects that sign ambiguity will seriously prevent the distillation model from learning meaningful synthetic data. Finally, can be observed that the proposed rotator can help point cloud models to rotation-invariant representations, thus ben- efiting the learning of synthetic data.
Table 2 presents the results of the part segmentation task the ShapeNet dataset. Unlike classification, part segmen- tation requires learning both global shape structures and fine-grained part details, making it a more challenging task for DD. Since some traditional DD methods struggle with segmentation, we only compare DD3D against random core- set selection and GM. The results show that GM is slightly better than coreset selection it is initialized by the real data. On the other hand, DD3D consistently outperforms both methods across all metrics by a large margin, demon- strating its effectiveness in learning the fine-grained features of point clouds. As expected, DD3D's performance does not yet reach the full dataset baseline. Nevertheless, achieves 90% of the performance of the entire dataset using only 1% of the data, demonstrating its potential in 3D distillation.
5.3. Cross-architecture Generalization.
We evaluate whether DD3D can benefit different point cloud models. Specifically, we use PointNet as the distillation method and utilize five advanced point cloud models as eval- uation methods, trained on the synthetic data from scratch.
Point-wise Generator. Next, we explore the performance of DD3D under different resolutions to verify the effective- ness and efficiency of the proposed generator. Typically, the shape classification task needs 1,024 points for training and
7

---

Point Cloud Dataset Distillation
Raw Images
DD3D
GM
Figure 4: Visualizations of different methods. Top: ModelNet (Airplane). Bottom: ShapeNet (Guitar, Laptop, and Pistol).
Times (c) Per Iteration
DD3D
DD3D
Time (s)
(GB)
20
10-
(a) Airplane
(b) Earphone
(c) Cap
256 512
3026
Clouds Classes
Clouds Class
Number of
(a) Time
(h) Budget
(c) Resolution
Figure 6: Time and space overhead between DD and DD3D.
in ShapeNet, the point clouds of GM are squeezed, making its shape inconsistent with the real dataset, while results of DD3D are more realistic and encode the spatial relation- ship between parts. Additionally, Figure 5 illustrates that DD3D not only preserves geometric details but also gener- ates informative samples, further validating its effectiveness in 3D dataset distillation.
(d) Car
(e) Guitar
Pistol
Figure 5: Geometric details of points generated by DD3D.
evaluation. In this experiment, randomly sample 256 and 512 points from real data supervise the distillation of DD3D. Once trained, we leverage DD3D to generate 1,024 points for evaluation. It is visible from Figure that training on high-resolution point clouds can accelerate the convergence of DD3D but the final matching losses are similar. Moreover, Table 5 shows that different resolutions have similar performance. In some cases, low-resolution data also outperforms high-resolution point clouds, e.g., ScanObjectNN. This discovery shows that DD3D can not only achieve stable results but also significantly reduce com- putational costs and GPU memory overhead.
5.6. Time and Space Overhead
We compare the overhead between DD and DD3D from multiple views. Firstly, Figure 6(a) shows that the time overhead of DD3D is slightly higher than DD due to the generation of synthetic data. Then, we can observe from Figure 6(b) that the memory budget of DD grows faster than DD3D as the value of CPC increases. DD3D can save the budget of synthetic data by sharing the generator be- tween different classes, and its memory nearly 4x smaller than DD when CPC=10. Figure 6(c) illustrates the changes in time and space overhead of DD3D different resolu- tions. We can see that training with low-resolution point clouds significantly reduces overhead, which is important for resource-comee constraied scenarios, such as edge computing.
5.5. Visualization
We visualize the real and synthetic point clouds in Figure 4 for more intuitive comparison. The results of DD3D and GM are placed in the last two columns. It can be observed that the point clouds generated by GM tend to condense to some clusters, while some isolated points are left as noise. On the contrary, the point clouds generated by DD3D are coherent and encode the global geometric shapes. Moreover,
6. Conclusion
This paper introduces DD3D for 3D point cloud distillation, which matches the rotation- ination-invarian data distribution be-
8

---

Point Cloud Dataset Distillation
Tween real and synthetic data by transforming point clouds into canonical orientation. Once trained, DD3D can syn- thesize point clouds at arbitrary resolutions, reducing mem- ory budget and improving scalability. Extensive experi- ments on both classification and segmentation tasks validate the superiority of DD3D over traditional DD methods. A promising direction is to initialize DD3D with real data to improve its performance.
Deng, Z. and Russakovsky, O. Remember the past: Dis- tilling datasets into addressable memories for neural net- works. In NeurlPS, 2022.
Du, J., Jiang, Y. Tan, V. Y. F., Zhou, T., and Li, H. Minimizing the accumulated trajectory error to improve dataset distillation. In CVPR. pp. 3749-3758. IEEE, 2023.
Geng. J., Chen, Z., Wang. Y., Woisetschlaeger, H., Schimm- ler, S., Mayer, R., Zhao, Z., and Rong, C. survey on dataset distillation: Approaches, applications and future directions. In IJCAI, pp. 6610-6618, ijcai.org. 2023.
Acknowledgment
This project is supported by the National Research Foun- dation, Singapore, under its Medium Sized Center for Ad- vanced Robotics Technology Innovation.
Guo, M., Cai, J., Liu, Z., Mu, T., Martin, R. R., and Hu, S. PCT: point cloud transformer. Comput. Vis. Media, 7(2): 187-199, 2021a.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. In particular, it aims to accelerate the training of deep neural networks and reduce memory overhead. There are many potential socictal consequences of our work, none of which we feel must be specifically highlighted here.
Guo, Y., Wang, H., Hu, Q., Liu, H., L., and Bennamoun, M. Deep learning for 3d point clouds: A survey, IEEE Trans. Pattern Anal. Mach. Intell., 43(12):4338-433 2021b.
Guo, Z., Wang, K., Cazenavette, G., Li, H., Zhang, K., and
You, Y. Towards lossless dataset distillation via difficulty- aligned trajectory matching. In ICI.R OpenReview.net, 2024.
References
Cazenavette, G., Wang, T., Torralba, A., Efros, A. A., and Zhu, J. Dataset distillation by matching training trajecto- ries. CVPR, pp. 10708-10717. IEEE, 2022.
Huang, Z., Johnson, J., Debnath, S., Rehg. J. M., and Wu, Pointinfinity: Resolution-invariant point diffusion models. In CVPR, 2024.
Cazenavette, G., Wang, T., Torralba, A., Efros, A. A., and Zhu, Generalizing dataset distillation via deep genera- tive prior. In CVPR, pp- 3739-3748. IEEE, 2023.
Jin, W., Zhao, L.., Zhang, S., Liu, Y., Tang, J., and Shah, N. Graph condensation for graph neural networks. In ICLR. OpenReview.net, 2022.
Kim, J., Kim, J., Oh, S. J., Yun, S., Song, H., Jeong, J., Ha. J., and Song, H. O. Dataset condensation via efficient synthetic-data parameterization. In ICMI,, volume 162 of Proceedings of Machine Learning Research, pp. 11102- 11118. PMLR, 2022.
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song. S., Su, H., Xiao, J., Yi, L., and Yu, F. Shapenet: An information- rich 3d model repository. CoRR, abs/1512.03012, 2015.
Chen, Y., Liu, S., and Wang, X. Learning continuous image representation with local implicit image function. In CVPR, pp. 8628-8638. Computer Vision Foundation IEEE, 2021.
Kim, S., Park, J., and Han, B. Rotation-invarian local- to-global representation learning for 3d point cloud. In NeurlPS, 2020.
Lei, and Tao, D. comprehensive survey of dataset distillation. IEEE Trans. Pattern Anal. Mach. Intell., 46 (1):17-32, 2024.
Deitke, M., Liu, R., Wallingford, M., Ngo, H., Michel, O.,
Kusupati, A., Fan, A., Laforte, C., Voleti, V., Gadre, S. Y.,
VanderBilt, E., Kembhavi, A.. Vondrick, C., Gkioxari, G.,
Li, F., Fujiwara, K., Okura, F,, and Matsushita, Y. A closer look at omation-invariaan deep point cloud analysis. In ICCV, pp. 16198-16207. IEEE, 2021.
Ehsani, K., Schmidt, L.., and Farhadi, A. Objaverse-xl: A
universe of 10m+ 3d objects. In NewrIPS, 2023.
Deng, C., Litany, O., Duan, Y., Poulenard, A., Tagliasac- chi, A., and Guibas, L. J. Vector neurons: A general framework for equivarrat networks. ICCV. pp. 12180-12189, JEEE, 2021.
Li, X., Li, R., Chen, G., Fu, C., Cohen-Or, D., and Heng. P. A rotation-invariant framework for deep point cloud analysis. IEEE Trans. Vis. Comput. Graph., 28(12):244)))))) 48)):335033 4514, 2022.
9

---

Point Cloud Dataset Distillation
Li, Y., Bu, R., Sun, M. Wu, W., Di, X., and Chen, B. Pointenn: Convolution on x-transformed points. In NeurlPS, pp. 828-838, 2018.
Shin, D., Shin, S., and Moon, I. Frequency domain-based dataset distillation. In NeurlPS, 2023.
Singh, R., Shukla, A., and Turaga, P. K. Polynomial implicit
Liu, S., Wang, K., Yang, X., Ye, J., and Wang. X. Dataset distillation via factorization. In NeurlPS, 2022.
neural representations for large diverse datasets. In CVPR, pp. 2041-2051. IEEE, 2023.
Liu, Y., Bo, D., and Shi, C. Graph distillation with eigenba- sis matching. In /CML, 2024.
Sitzmann, V., Martel, J. N. P., Bergman, A. W., Lindell, D. B., and Wetzstein, G. Implicit neural representations with periodic activation functions. NeurIPS, 2020.
Loo, N., Hasani, R. M., Amini, A., and Rus, D. Efficient dataset distillation using random feature approximation. In NeurIPS, 2022.
Thomas, N., Smidt, E., Kearnes, S., Yang, L.,, Li, L., Kohlhoff, K., and Riley, P. Tensor field networks: Rotation- and translation-equiiariant neural networks for 3d point clouds. CoRR, abs/ 802.08219, 2018.
Ma, X., Qin, C., You, H., Ran, H., and Fu, Y. Rethinking network design and local geometry in point cloud: A sim- ple residual MLP framework. In ICLR. OpenReview.net, 2022.
Uy, M. A., Pham, Q. Hua, B., Nguyen, D. T., and Yeung, S. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In ICCV, pp.
Melnyk, P., Robinson, A., Felsberg, M., and Wadenbäck, M. Tetrasphere: A neural descriptor for o(3)-invariant point cloud analysis. In CVPR, pp. 5620-5630. IEEE, 2024.
IEEE, 2019.
Wang. K., Zhao, B., Peng, X., Zhu, Z.,, Yang. S., Wang, S., Huang, G., Bilen, H., Wang, X., and You, Y. CAFE: learning condense dataset by aligning features. In CVPR, pp.
Nguyen, T., Chen, Z., and Lee, J. Dataset meta-learning from kernel regreseegr,s..n. In ICLR. OpenReview.net, 2021.
IEEE, 2022.
Park, J. J., Florence, P. R., Straub, J., Newcombe, R. A., and Lovegrove, Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, pp. 165-174. Computer Vision Foundation / TEEE, 2019.
Wang. K., Gu, J., Zhou, D., Zhu, Z., Jiang, W., and You, Y. Dim: Distilling dataset into generative model. CoRR, abs/2303.04407, 2023.
Wang, T., Zhu, J., Torralba, A., and Efros, A. A. Dataset 2018
Poulenard, A., Rakotosaona, M., Ponty, Y., and Ovsjanikov, Effective rotation-invarian point CNN with spherical harmonics kernels. 3DV, pp. 47-56. IEEE, 2019.
distillation. CoRR,
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., and Solomon, M. Dynamic graph CNN for learning on point clouds. ACM Trans. Graph., 38(5): 2019.
Qi, C. R., Su, H., Mo, K., and Guibas, L. J. Pointnet: Deep learning on point sets for 3d classification and segmen- tation. CVPR, pp. 77-85. IEEE Computer Society, 2017a.
Wang, Z., Xu, Y., Lu, C.,, and Li, Y.-L. Dancing with images: Video distillation via static-dynamic disentanglement. In CVPR, 2024.
Qi, R., Yi, L., Su, H., and Guibas, L. J. Pointnet++: Deep hierarchical feature learning on point sets a metric space. In NIPS, pp. 5099-5108, 2017b.
Welling, M. Herding dynamical weights to learn. In ICML, volume 382, pp. 1121-1128, 2009.
Qian, G., Li, Y. Peng, H., Mai, Hammoud, H., Elhoseiny, M.. and Ghanem, B. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. In NewrIPS, 2022.
Wu, W., Qi, Z., and Li, F. Pointconv: Deep convolutional networks on 3d point clouds. CVPR. pp. 9621-9630. Computer Vision Foundation / IEEE, 2019.
Qiu, S., Anwar, S., and Barnes, N. Dense-resolution network for point cloud classification and segmentation. In WACV, pp. 3812-3821. IEEE, 2021.
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pp. 1912-1920, 2015.
Sachdeva, N. and McAuley, J. Data distillation: A survey. CoRR, abs/2301.044 104272, 2023.
Xiao, Z., Lin, H., Li, R., Geng, L., Chao, H., and Ding, S. Endowing deep 3d models with rotation invariance based on principal component analysis. In ICME, pp. 1-6. IEEE, 2020.
Sener, O. and Savarese, S. Active learning for convolutional neural networks: core-set approach. In ICLR, 2018.
10

---

Point Cloud Dataset Distillation
Xu, J., Tang, X., Zhu, Y., Sun, J., and Pu, S. Sgmnet: Learning -nvaiinninvariant point cloud representations via sorted gram matrix. In ICCV, pp. 10448-10457. IEEE, 2021.
Zhao, C., Yang. J., Xiong, X., Zhu, A., Cao, Z.,, and Li, X.
Rotation invariant point cloud classification: Where local geometry meets global topology. CoRR, abs/1911.00199, 2019.
Yi, L., Kim, V. G., Ceylan, D., Shen, I., Yan, M., Su, H., Lu, C.,, Huang, Q., Sheffer, A., and Guibas, L. J. scal- able active framework for region annotation in 3d shape collections. ACM Trans. Graph., 35(6):210:1-210:11 2016.
Zhou, Y., Nezhadarya, E., and Ba, J. Dataset distillation using neural feature regression. In NeurlPS, 2022.
Yu, H., Hou, J., Qin, Z., Saleh, M., Shugurov, Wang, K., Busam, B., and Ilie, S. RIGA: rotation-invariant and globally-aware descriptors for point cloud registra- tion. IEEE Trans. Pattern Anal. Mach. Intell., 46(5): 3796-3812, 2024a.
Yu, R., Wei, X., Tombari, F., and Sun, J. Deep positional and relational feature learning for rotation-invariant point cloud analysis. In ECCV, volume 12355, pp. 217-233. Springer, 2020.
Yu, R., Liu, S., and Wang. X. Dataset distillation: A compre- hensive review. IEEE Trans. Pattern Anal. Mach. Intell., 46(1):150-170, 2024b.
Yu, X., Xu, M., Zhang, Y., Liu, H., Ye, C., Wu, Y., Yan, Z., Zhu, C., Xiong, Z., Liang, T., Chen, G., Cui, S., and Han, X. Mvimgnet: A large-scale dataset of multi-view images. In CVPR, pp. 9150-9161. IEEE, 2023.
Zhang, J., Wang, H., Xue, C., Yan, R., Zhang, W., Bai, S., and Shou, M. Z. Dataset condensation via generative model. CoRR, abs/2309.07698 2023.
Zhang, H., Su, S., Zhu, Y., Sun, and Zhang, Y. GSDD: generative space dataset distillation for image super- resolution. In AAAI, pp. 7069-7077. AAAI Press, 2024a.
Zhang, Z., Yang. L., and Xiang, Z. Risurconv: Rotation invariant surface attention
attention-augmentte augmented convolutions for 3d point cloud classification and segmentation. In ECCV (28), volume 15086 of Lecture Notes in Computer Sci- ence, pp. 93-109. Springer, 2024b.
Zhao, B. and Bilen, H. Dataset condensation with differen- tiable siamese augmentation. In ICML, volume 139 of Proceedings of Machine Learning Research, pp. 12674- 12685. PMLR, 2021.
Zhao, B. and Bilen, H. Synthesizing informative training samples with GAN. CORR, abs/2204.07513, 2022.
Zhao, B. and Bilen, H. Dataset condensation with distribu- tion matching. In WACV, pp. 6503-6512, IEEE, 2023.
Zhao, B., Mopuri, K. R., and Bilen, H. Dataset condensation with gradient matching. In /CLR. OpenReview.net, 2021.

---

Point Cloud Dataset Distillation
A. Proof of Theorems
Theorem A.1. Assume the classifier is a linear layer W and Lots can be simplified to the mean mean-squared squunn- error || - -I The objective of gradient matching is equal to variance preserving:
minCaas
min D (Vw C.S. Vw
min |x X7X711
(15)
S
where D a distance metric and Vw w is the gradient with respect to W.
Proof. The gradient of ||W - 1 is denoted as = (XW - Y). We can then match the gradients between the real and synthetic data:
V VTIß = |X (XsW - Ys) - (X - -YT)1I?
(16)
| / |X - x I - 1
(17)
Variance
Mcan
We can see that the first term is to preserve the variance of real data, and the second term aligns the average representations of samples belonging to the same class. These terms be combined if we set Xs Xs X and X NT-X7Y for each class. Then we only need to match the variance between Xs and X7.
Theorem A.2. Assume XT follows d-dimensional multivariate Gaussian distribution (H, E). Let X'y be the rotated representations of X such that:
Amaz (B (x(x>Tx;) = (B xx)) mmax (L S maax (B (xT)),
(18)
where Xmaz and maa the maximum eigenvalues and singular values, respectively.
Proof. Firstly, the largest eigenvalue of the covariance matrix Xy T Xx is equal to the largest singular value of Xy.
Therefore, we only prove the first inequality.
Secondly, as X X = Ei=l T. Tyt for X ~ (M,, E), we have:
E X E
E I. ms)
(H 4 ++ E),
(19)
i=1
E X7 X/
E
Σ
R. 24
24
R
EE
E
2. R.
= R
Σ
[T. R.
(20)
1=1
Thirdly, we have:
Amax
xT]
ndmar (μ T μ + Σ).
(21)
n
Amaz
E
X T x7)
Amax Σ RTTH 1 x3 R₂
Maa (R. [1.] x.] R.)
(22)
Σ Amar (Re 1 nR, + R. ER,) S Amar (R [[xxxT]]
(23)
The above inequality shows that the largest eigenvalue of E [X]x7] is the upper bound of Xy x7} The equality
holds if and only if the random rotation matrices are commutative, which infeasible in practice.
B. Implementation Details of DD3D
Here, we explain some details of DD3D, consisting of two important components: a point cloud rotator and a pisnt-wise generator. Both components are built based on the SIREN (Sitzmann et al., 2020) model, which stacks multiple fully connected layers with sin(-) activation to capture the high-frequency information. The PyTorch code is shown in Algorithm 2, where some details are highlighted.
12

---

Point Cloud Dataset Distillation
Algorithm 2 PyTorch code of DD3D
import torch
import torch.nn Import STREN
3
Rotator (nn.Moulee) def in't tin't_iae (gel hiddar_dit wO)
supes
to to adjust the fraquency of sine function ail..aign_enoode SIREN(1, hidden_dim,, (0=w3) self.sign_decoder (hidddiidee 1, w0-1.)
15
def forward x):
x.unsqueeze
16
fes: = self.sign_enzeed enzoder (x) mean (dimmi, keepdim-Trusl [B, N, 3, 11 -> [8, 1, 3, d]
fest gelffaain_dedseee (feat) #
d] => [3,
sign torch.aign (feat) sign-equivariant
19
sign [8,
[8,
return x. s.ueeeze (-1)
22
asa conditiona" Generator Mon.Modul
25
del init iint__ntsee (self, genetator, nus_classes, epe, condition_din, aum_laye:s):
26
supe: -rit_
gele.genetator genetator
29
self, cokup Rn.eddidi (num_elasses cordition_ddmi clasa index as condition self.nun_layers aun__ayers
31
32
Moddlelettttt(
for in zange man.f..m=_laayrr.. 1): (nn, Linear (cordition cordition_dim) an.8eta
35
self. (tn.errs.ppenddn..cooo
37
def forward(self, noise, class_indices):
38
noise [8, 11 class_ini cas [8,
41
self lockup (class mods [mad]
44
for self self.lsyers: layer (mod)
47
mods.append (mod)
return self.genetator (ncise, tuple (mods)
C. Details of Datasets
The detailed statistical information of the datasets used in this paper is shown in Table We list the sources of the datasets and their licenses in the following.
ModelNet40: htteectttettt///ccccccccccccccce
MVPNet: NVW.atttee///tttuuooccc/a//aa111
ShapeNet: Stangt/t/tssssssaccco/c////////s aaaattllodd-aallsiss analysis
D. Hyperparameters
The hyperparameters of baselines and DD3D are listed in Tables 7 and 8, respectively.
13

---

Point Cloud Dataset Distillation
Table 6: Details of datasets
ScanObjectNN
ModelNer40
MVPNet
ShapeNet
Shape Classes
15
40
100
16
Part Classes
50
Training Samples
2,322
9,843
62,494
14,007
Validation Samples
580
2,468
15,670
2,874
Resolution
1,024
1,024
1,024
2,048
Table 7: Hyperparameters used for Data Synthesis.
ScanObjectNN
ModelNet40
MVPNet100
ShapeNet
Optimizer Initial LR
Adam
Adam
Adam
Adam
0.001
0,001
0.001
0.001
Batch Size
32
32
64
32
Iterations
200
400
600
200
Weight Decay
0.0005
0.0005
0.0005
0.0005
Augmentation
Scale, Jitter, Rotate StepLR
Scale, Jitter, Rotate StepLR
Scale, Jitter, Rotate StepLR
Scale, Jitter, Rotate
Scheduler
(Decay 0.1 / 100 iter)
(Decay 0.1 100 iter)
(Decay 0.5 200 iter)
Table 8: Hyperparameters used for Validation.
ScanObjectNN
ModelNer40
MVPNet100
ShapeNet
Optimizer Initial LR
Adam
Adam
Adam
Adam
0.001
0.001
0.001
0.001
Batch Size
32
Epuchs Weight Decay
200
200
200
200
0.0005 Scale, Jitter, Rotate StepLR
0.0005
0.0005
0.0005
Augmentation
Scale, Jitter, Rotate StepLR
Scale, Jitter, Rotate
Scale, Jitter, Rotate
Scheduler
CosineAneellingL.R
(Decay 0.1 / 100 epoch)
(Decay 0.1 100 epoch)
14


Reviews:
[{"review_id": "d57kLWjTjO", "reviewer": "Reviewer_vcTR", "summary": "This paper proposes DD3D, the first dataset distillation method designed specifically for 3D point cloud data. DD3D addresses two critical challenges in point cloud distillation: orientation misalignment and varying resolutions. The authors first establish theoretically that an ideal dataset distillation should preserve the intrinsic variance of the dataset and that orientation-misaligned samples introduce undesirable perturbations. To solve this, they introduce a novel plug-and-play rotator to consistently align point clouds to canonical orientations, resolving rotation and sign ambiguities. Additionally, DD3D employs a point-wise generator capable of synthesizing high-resolution point clouds from low-resolution training samples, making the approach flexible and scalable. Experiments conducted on shape classification and segmentation tasks demonstrate that DD3D consistently outperforms existing baselines. The method proves effective, scalable, and memory-efficient, significantly advancing dataset distillation for 3D data.\n\n## update after rebuttal\nThe authors tried to address my concerns, and I encourage the authors to fulfill their commitment by including the promised updates in the final manuscript. My final rating is \"Weak Accept\" (leaning towards accept).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed review and the recognition of our contributions.\n\n&nbsp;\n### ``Experimental Designs Or Analyses``\n\n> **Q1: The approach demonstrates relatively limited performance on fine-grained tasks.**\n\nA1: We propose two strategies to further improve the performance of DD3D.\n1. Improving the number of CPC. We report the performance of DD3D with CPC=100, and find that it can achieve comparable performance with the full dataset.\n\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| GM | 81.74 | 84.17 | | GM | 57.52 | 62.82 |\n| DD3D | 83.91 | **86.68** | | DD3D | 61.96 | **65.51** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n2. Combined with knowledge distillation (KD). Recent DD methods suggest that leveraging KD can significantly improve the performance of DD. We can also adopt this strategy to achieve lossless performance.\n\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n|DD3D | 83.91 | 86.68 | |DD3D | 61.96 | 65.51 |\n|DD3D+KD | 86.55 | **88.75** | |DD3D+KD | 65.06 | **66.84** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n> **Q2: This highlights potential limitations in its ability to capture detailed geometric features.**\n\nA2: The goal of DD is to make the synthetic data informative, making the synthetic samples deviate from the distribution of real data. As a result, the synthetic dataset may have some artifacts and overlook some detailed geometries. **The same results can also be observed in vision dataset distillation.**\n\nNotably, ``Figure 4`` (lines 385-395) illustrates that synthetic data generated by GM has much noise and isolated points, making the data unrealistic. On the other hand, synthetic data generated by DD3D is coherent and captures the global geometric shapes.\n\n> **Q3: The proposed method focuses on shape-level tasks and does not easily extend to large-scale scene-level applications.**\n\nA3: We conduct a semantic segmentation experiment on the S3DIS dataset, which contains more than 80M points in the training data and has more noise than shape-level datasets.\n\nWe use Area 5 as the test set and other areas as the training set, and report the overall accuracy (OA) and instance mean IoU of different methods. We can observe that DD3D consistently outperforms GM by a large margin, demonstrating its effectiveness in large-scale scene-level tasks.\n| S3DIS (OA / mIoU) | GM | DD3D | Full |\n| :-- | :--: | :--: | :--: |\n| CPC=1 (0.06%) | 53.09% / 0.3674 | **57.45% / 0.4043** | 73.78% / 0.5786 | \n| CPC=10 (0.6%) | 61.72% / 0.4438 | **64.05% / 0.4624** | 73.78% / 0.5786 |\n\n> **Q4: Including comprehensive benchmarks against state-of-the-art rotation-invariant methods.**\n\nA4: We compare DD3D with some SOTA rotation-invariant methods, including RISurConv, TetraSphere, and SGMNet (suggested by Reviewer gVCR). The results are shown below.\n| ModelNet40 | CPC=10 | CPC=50 | Full |\n| :-- | :--: | :--: | :--: |\n| DD3D | 58.14 | **71.27** | 80.45 |\n| RISurConv | 55.81 | OOM | **95.60** |\n| TetraSphere | 54.72 | OOM | 90.50 |\n| SGMNet | **58.33** | 70.48 | 80.77 |\n\nWe have the following observations:\n1. Both RISurConv and TetraSphere perform well in the full dataset training. However, they are not suitable for the DD task, as they all need to build a k-NN graph to learn rotation-invariant representations. During distillation, the synthetic point clouds are dynamically optimized, resulting in different nearest neighbors. As a result, we need to **re-calculate the k-NN graph** in each iteration, which significantly increases the time and space overhead.\n2. DD3D and SGMNet are model-agnostic, which can be easily combined with different point cloud models. Here we use the 3-layer PointNet. Their results are better than RISurConv and TetraSphere, indicating that complex models do not always lead to better distillation performance. In contrast, simple models may be more suitable for DD.\n\n&nbsp;\n### ``Supplementary Material``\n\n> **Q5: The Appendix B is incomplete with only Rotator, there is no Generator. Also, the Algorithm 2 is different with the provided source code. In addition, the provided source code is incomplete too, without instructions to reproduce.**\n\nA5: We will revise our code to make it more complete and easier to read. Specifically, \n- Add a PyTorch-style pseudo algorithm of the Generator.\n- The implementation of Algorithm 2 is rooted in ``CINR.py/InvariantWrapper``. We also provide a simpler vision of the rotator, which removes the hidden dimension of SIREN to reduce the model complexity.\n- Add a Readme file to illustrate the reproduction process.\n\n&nbsp;\n### ``Essential References Not Discussed``\n\n> **Q6: There are several references regarding rotation-equivariant and invariant features that are worth discussing.**\n\nA6: Thanks for pointing out these important related works. We will cite and discuss them in the revision.", "claims_and_evidence": "The authors prove mathematically that random rotations weaken the principal components of real data, negatively impacting distillation performance. This strongly motivates the inclusion of a rotation-aware component (\"rotator\") in their proposed method.", "methods_and_evaluation_criteria": "The authors provide thorough theoretical analyses, formally demonstrating that matching rotation-invariant features is essential for successful 3D point cloud distillation.", "theoretical_claims": "The paper provides a robust mathematical foundation, clearly explaining why the proposed dataset distillation (DD3D) approach works effectively.", "experimental_designs_or_analyses": "The approach demonstrates relatively limited performance on fine-grained tasks such as shape classification and part segmentation. This highlights potential limitations in its ability to capture detailed geometric features. Furthermore, the proposed method focuses on shape-level tasks and does not easily extend to large-scale scene-level applications.\n\nMoreover, while the paper compares DD3D with various distillation methods, its claims could be substantially strengthened by including comprehensive benchmarks against state-of-the-art rotation-invariant methods as I will mention below.", "relation_to_broader_scientific_literature": "The paper can promote research on dataset distillation for point clouds, based on 2D image studies.", "essential_references_not_discussed": "There are several references regarding rotation-equivariant and invariant features that are worth discussing:\n\n[1] Zhang et al., RISurConv: Rotation Invariant Surface Attention-Augmented Convolutions for 3D Point Cloud Classification and Segmentation, ECCV’24\n\n[2] Hao et al., RIGA: Rotation-Invariant and Globally-Aware Descriptors for Point Cloud Registration, TPAMI’24\n\n[3] Melnyk et al., TetraSphere: A Neural Descriptor for O (3)-Invariant Point Cloud Analysis, CVPR’24.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "It is good to show experiments with large-scale scene and also compare with recent rotation-invariant methods as I mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AIbICwFA6o", "reviewer": "Reviewer_gVCR", "summary": "The paper addresses the problem of distilling large 3D point cloud datasets into a much smaller set while preserving model performance. The paper identifies two key challenges unique to point clouds: random orientation and variable resolution. To tackle these issues, the authors propose a novel framework called DD3D, which introduces a plug-and-play point cloud rotator that aligns each point cloud to a canonical orientation, making the distilled data rotation-invariant. It also uses a point-wise generator to produce point clouds from noise, allowing flexible output sizes (arbitrary resolutions) while training on lower resolutions. The rotator and generator are optimized jointly using a gradient-matching distillation objective so that models trained on the small synthetic set achieves performance close to ones trained on the full set. The paper demonstrates DD3D on 3D shape clasification and part segmentation, and shows competitive performance through this dataset distillation process.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful feedback and insightful questions.\n\n&nbsp;\n### ``Other Strengths And Weaknesses``\n\n> **Q1: Comparing DD3D to a scenario where a rotation-invariant model is used could help delineate the benefits of the proposed approach. Any discussion in this direction is welcomed.**\n\nA1: As suggested, we compare the performance between DD3D, SGMNet [1], and Li et al [2] in the randomly rotated ModelNet40 dataset. Specifically, DD3D and SGMNet are model-agnostic methods, and we use our 3-layer PointNet as the backbone. The results are shown below.\n| ModelNet40 | CPC=10 | CPC=50 | Full |\n| :-- | :--: | :--: | :--: |\n| DD3D | 58.14 | **71.27** | 80.45 |\n| SGMNet | **58.33** | 70.48 | 80.77 |\n| Li et al | 43.82 | OOM | **89.4** |\n\nWe have the following observations:\n1. DD3D has a similar performance to SGMNet, demonstrating the effectiveness of the proposed rotator. Generally, DD3D leverages the fact that eigenvectors are rotation-equivalent to alleviate the influence of random rotations, i.e., $(PR)(R^\\top U)=PU$. SGMNet uses the outer product to eliminate rotations, i.e., $(PR)(PR)^\\top=PP^\\top$. Compared to SGMNet, DD3D only needs to maintain a $3 \\times 3$ matrix $U$ rather than an $N \\times N$ matrix $XX^\\top$, which is more efficient. \n2. While Li et al’s work has the highest accuracy in the full dataset, it does not perform well in the distillation task. This observation is consistent with a recent work in DD [3] that networks with complex components may affect the performance of DD. \n3. Li et al’s work is out-of-memory in the CPC=50 setting, due to the quadratic complexity of fastest sampling. Each time the synthetic point cloud is optimized, the fastest sampling needs to be recalculated, which significantly increases the time and memory overhead of the distillation process.\n\nBased on the above observation, we have the following discussion:\n1. **Distillation network is important**. The SOTA point cloud models often need to build a k-NN graph or apply fastest sampling on the point clouds, which is not suitable for DD as it costs too much time and space.\n2. **Model-agnostic v.s. Model-specific**. Li et al’s work is model-specific and achieves the highest accuracy. However, it is infeasible to transfer it to some simple models, like PointNet. On the other hand, SGMNet is model-agnostic and can be used for different models, which is more suitable for DD.\n\n> **Q2: The framework adds extra components (the rotator and generator) to the distillation pipeline, which increases the method’s complexity.**\n\nA2: Compared to the gradient matching process, the rotator and generator only slightly increase the computational complexity. To verify this conclusion, we use `line_profiler` to record the time used in rotator, generator, and gradient matching, respectively, and report their percentages. The results below indicate that the rotator and generator only slightly impact the computational overhead:\n\n| CPC | Rotator (%) | Generation (%) | Matching (%) |\n| :---: | :---: | :---: | :---: |\n| 1     | 0.5 | 3.9 | 95.6 |\n| 10   | 0.3 | 2.7 | 97.0 |\n| 50   | 0.1 | 1.0 | 98.9 |\n\n> **Q3: The authors could elaborate on the point-wise generator’s architecture and training process.**\n\nA3: Thanks for the great advice. We will add more description of the rotator and generator of DD3D in the revision. We briefly introduce the training process of the generator below.\n\n- The point-wise generator $\\mathbb{R} \\rightarrow \\mathbb{R}^3$ aims to map each sampled noise into a point. Therefore, we can adjust the amount of input noise to control the resolution of synthetic point clouds.\n- During training, we sample fewer points from the real data and calculate their gradient as supervision to update the generator. During inference, we can sample more noise to ensure geometric details.\n\n> **Q4: A deeper understanding of its training procedure or potential failure cases (e.g., ambiguous alignments for symmetric shapes) would be useful.**\n\nA5: We will add more visualization in the revision to illustrate how the rotations affect the training process of synthetic datasets.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidences.", "methods_and_evaluation_criteria": "The proposed methods make sense for the problem.", "theoretical_claims": "I have checked the theoretical claims in the main paper and saw no obvious errors.", "experimental_designs_or_analyses": "The experimental designs are valid.", "relation_to_broader_scientific_literature": "The paper could be significant for efficient 3D network training and provide insights in this field.", "essential_references_not_discussed": "The references are adequate.", "other_strengths_and_weaknesses": "Strengths\n- This work is the first to focus on dataset distillation for 3D point clouds​, an extension of data distillation methods beyond images. It clearly pinpoints unique challenges in 3D that were not handled by prior 2D distillation methods​. The problem is well-motivated by practical needs (reducing memory/training costs on huge 3D datasets).\n- The paper provides a theoretical analysis to justify the approach. While intuitive, the paper proves (under simplifying assumptions) that the gradient-matching distillation objective is equivalent to preserving the data variance, and that random rotations can weaken the principal components (variance) of the data, harming distillation​. This analysis highlights why matching rotation-invariant features is important.\n- The paper backs up its claims with comprehensive experiments on multiple datasets and tasks, including 3D point cloud classification and segmentation. Notably, the authors tested performance on cross-architecture generalization, and show general applicability of the distillation scheme.\n\nMinor Weaknesses\n- Since some point cloud models are inherently rotation-invariant (e.g. [1, 2], and should be many more recent methods), comparing DD3D to a scenario where a rotation-invariant model used (instead of adding an external rotator) could help delineate the benefits of the proposed approach. Any discussion in this direction is welcomed.\n- The framework adds extra components (the rotator and generator) to the distillation pipeline, which increases the method’s complexity. The paper would benefit from more details on these components. For instance, the authors could elaborate on the point-wise generator’s architecture and training process, e.g., do they simply adjust the amount of input noise, how is the generator supervised across different point counts, etc. Likewise, the rotator module is a key piece of the pipeline- a deeper understanding of its training procedure or potential failure cases (e.g., ambiguous alignments for symmetric shapes) would be useful. Without these details, it’s hard to assess the overall robustness under different input settings.\n\n[1] Li, Xianzhi, et al. \"A rotation-invariant framework for deep point cloud analysis.\" IEEE transactions on visualization and computer graphics 28.12 (2021): 4503-4514.\n[2] Xu, Jianyun, et al. \"Sgmnet: Learning rotation-invariant point cloud representations via sorted gram matrix.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.", "other_comments_or_suggestions": "See Other Strengths And Weaknesses.", "questions_for_authors": "See Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9w4aeFq1P1", "reviewer": "Reviewer_bmGW", "summary": "The paper introduces DD3D, a dataset distillation method tailored for 3D point clouds, addressing challenges of orientation diversity and varying resolutions. It proposes a rotation-invariant feature matching approach, a point cloud rotator for canonical alignment, and a point-wise generator that efficiently produces multi-resolution point clouds. Extensive experiments demonstrate DD3D’s effectiveness in shape classification and part segmentation, showing strong cross-architecture and cross-resolution generalization while reducing memory usage.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and suggestions.\n\n&nbsp;\n### ``Other Strengths And Weaknesses``\n\n> **Q1: Comparison between dataset distillation,  knowledge distillation, and semi-supervised learning.**\n\nA1: Dataset distillation (DD) and knowledge distillation (KD) are two orthogonal directions in efficient deep learning. DD aims to compress data, and KD is used to compress models. A recent DD method [1] suggests that incorporating KD into DD can alleviate the performance gap between the real and synthetic data. To verify this, we report the performance of **DD3D** and **DD3D+KD** in the ModelNet40 and ScanObject datasets below.\n\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n|DD3D | 83.91 | 86.68 | |DD3D | 61.96 | 65.51 |\n|DD3D+KD | 86.55 | **88.75** | |DD3D+KD | 65.06 | **66.84** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\nWe can see that with the help of KD, the performance of synthetic datasets is close to the real datasets, indicating the potential of combining DD and KD. Moreover, semi-supervised learning is also a potential application of DD. Leveraging the knowledge of a few labeled samples to compress the massive unlabeled data is also a promising direction.\n\n[1] Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective. NeurIPS 2023.\n\n> **Q2: Performance improvements over other methods not specifically designed for 3D tasks seem relatively modest.**\n\nA2: The advantages of DD3D over other methods not specifically designed for 3D tasks, e.g., GM, DM, and TM, concentrate on addressing the rotation and resolution issues.\n\n1. For the randomly rotated datasets, DD3D outperforms other methods by a large margin. We report the results in Table 4 of the main text below. We can see that the performance of GM and DM drops quickly without the help of the rotator of DD3D, demonstrating the effectiveness of DD3D in distillating 3D datasets.\n| ModelNet40 | Random | GM | DM | DD3D |\n| :-- | :--: | :--: | :--: | :--: |\n| PointNet | 14.75 | 9.47 | 10.16 | 17.91 |\n| PointNet + PCA | 60.77 | 53.55 | 55.57 | 62.72 |\n| PointNet + Rotator | 70.13 | 68.92 | 69.31 | **71.27** |\n\n2. For the large-scale datasets, DD3D enables “low-resolution training and high-resolution generation”, which significantly reduces the space overhead during distillation. We report the space overhead of GM and DD3D below, from which we can observe that the memory cost of GM is 10x higher than DD3D, but its performance is still not as good as DD3D. Therefore, DD3D can handle large-scale datasets and larger CPC.\n| CPC (1/5/10) | Memory (MB) | Performance |\n| :-- | :--: | :--: |\n| GM | 120 / 1200 / 6000 | 53.38 / 72.11 / 75.45 |\n| DD3D | 140 / 230 / **630** | 53.82 / 73.54 / **76.31** |\n\n&nbsp;\n### ``Other Comments Or Suggestions``\n\n> **Q3: It seems that the link in the paper is not anonymous.**\n\nA3: We kindly clarify that this link is not related to our paper (our code is in the Supplementary Material) but to a related work on point cloud DD. **This link does not violate the double-blind reviewing policy.**", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "No", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n1, The paper is well-written and easy to understand.\n\n2, It is the first work to introduce dataset distillation for 3D point clouds.\n\n3, The rotation-equivariant design for point cloud distillation is intuitive and well-motivated.\n\nWeaknesses:\n\n1, While I am not an expert in this area, I am familiar with other data-efficient methods such as knowledge distillation and semi-supervised learning, and dataset distillation appears to demonstrate promising results in comparison.\n\n2, However, the performance improvements over other methods not specifically designed for 3D tasks seem relatively modest, raising concerns about the method’s overall impact.", "other_comments_or_suggestions": "It seems that the link in the paper is not anonymous.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1AJYtp2zOr", "reviewer": "Reviewer_yUbr", "summary": "In this paper, the authors presented DD3D, a novel framework for 3D point cloud distillation that aligns the rotation-invariant data distribution between real and synthetic data by transforming point clouds into a canonical orientation. Once trained, DD3D is capable of synthesizing point clouds at arbitrary resolutions, thereby significantly reducing memory consumption and enhancing scalability. Through extensive experiments on both shape classification and part segmentation tasks, the proposed method can achieve the superior performance compared to conventional dataset distillation methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the detailed and helpful comments. We reply to the comments in detail. Hope to address your concerns.\n\n&nbsp;\n### ``Methods And Evaluation Criteria``\n\n> **Q1: Potential impact of incorporating more recent rotation-invariant or rotation-equivariant models.**\n\nA1: Thanks for the great advice. We have tried several rotation-invariant or equivariant models for DD. However, there are two issues. \n\n1. The architectures of advanced rotation-invariant models are quite different, making it hard to transfer to rotation-variant models. \n2. These models often involve complex components. A recent survey on DD [1] shows that more complex models do not necessarily lead to better performance. See Q2 for a performance comparison.\n\nOn the other hand, the rotator of DD3D can alleviate the above two issues as it is lightweight and **model-agnostic**.\n\n[1] Dataset distillation: A comprehensive review. TPAMI 2024.\n\n> **Q2: How such integration might influence the performance or generalizability of the proposed approach?**\n\nA2: We compare DD3D with some SOTA rotation-invariant methods, including RISurConv, TetraSphere (suggested by Reviewer vcTR), and SGMNet (suggested by Reviewer gVCR). The results are shown below.\n| ModelNet40 | CPC=10 | CPC=50 | Full |\n| :-- | :--: | :--: | :--: |\n| DD3D | 58.14 | **71.27** | 80.45 |\n| RISurConv | 55.81 | OOM | **95.60** |\n| TetraSphere | 54.72 | OOM | 90.50 |\n| SGMNet | **58.33** | 70.48 | 80.77 |\n\nWe have the following observations:\n1. Both RISurConv and TetraSphere perform well in the full dataset training. However, they are not suitable for the DD task, as they all need to build a k-NN graph to learn rotation-invariant representations. During distillation, the synthetic point clouds are dynamically optimized, resulting in different nearest neighbors. As a result, we need to **re-calculate the k-NN graph** in each iteration, which significantly increases the time and space overhead.\n2. On the other hand, the rotator of DD3D and SGMNet are model-agnostic, which can be easily combined with different point cloud models. Here we use the 3-layer PointNet as the backbone, and their results are better than RISurConv and TetraSphere. This indicates that the complex models do not always lead to better distillation performance. In contrast, simple models may be more suitable for DD.\n\n> **Q3: Performance drop compared to using the full dataset and limitation for high-accuracy applications.**\n\nA3: We fully understand your concerns about the accuracy. For applications requiring high accuracy, we recommend two ways to improve the accuracy of DD3D.\n\n1. Improving the number of CPC. We report the performance of DD3D with CPC=100 in ModelNet40 and ScanObject, and find that it can achieve comparable performance with the full dataset.\n\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| GM | 81.74 | 84.17 | | GM | 57.52 | 62.82 |\n| DD3D | 83.91 | **86.68** | | DD3D | 61.96 | **65.51** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n2. Combined with knowledge distillation (KD). Recent DD methods suggest that leveraging KD can significantly improve the performance of DD. We can also adopt this strategy to achieve nearly lossless performance.\n\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n|DD3D | 83.91 | 86.68 | |DD3D | 61.96 | 65.51 |\n|DD3D+KD | 86.55 | **88.75** | |DD3D+KD | 65.06 | **66.84** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n> **Q4: Potential degradation when scaling to larger or more complex tasks.**\n\nA4: We conduct semantic segmentation experiments on the scene-level S3DIS dataset, which contains more than 80M point clouds for training. Generally, the scene-level task is more challenging than the object-level task, as it includes more data and noise. We can observe that DD3D consistently outperforms GM by a large margin, demonstrating its effectiveness in large-scale scene-level tasks.\n| S3DIS (OA / mIoU) | GM | DD3D | Full |\n| :-- | :--: | :--: | :--: |\n| CPC=1 (0.06%) | 53.09% / 0.3674 | **57.45% / 0.4043** | 73.78% / 0.5786 | \n| CPC=10 (0.6%) | 61.72% / 0.4438 | **64.05% / 0.4624** | 73.78% / 0.5786 |\n\n&nbsp;\n### ``Experimental Designs Or Analyses``\n\n> **Q5: Compare the performance and feature characteristics of models trained on both aligned and unaligned orientations.**\n\nA5: ``Table 4`` in the main text shows the performance of models trained on both aligned and unaligned orientations. We can see that the performance of GM and DM drops quickly without the help of the rotator of DD3D, demonstrating the effectiveness of DD3D in distillating 3D datasets.\n\n&nbsp;\n### ``Other Comments Or Suggestions``\n\n> **Q6: The authors should revise several typos and grammatical errors in the paper.**\n\nA6: We will carefully polish our paper and revise typos.", "claims_and_evidence": "The proposed method opens up new opportunities for efficient training of point cloud models and broadens the applicability of dataset distillation techniques to unstructured 3D data, compared to existing distillation methods have primarily focused on structured data such as images, videos, and text.", "methods_and_evaluation_criteria": "One of the core idea is that rotation invariance positively contributes to the effectiveness of distillation. In this context, I am curious about the authors’ perspective on the potential impact of incorporating more recent rotation-invariant or rotation-equivariant models, and how such integration might influence the performance or generalizability of the proposed approach.\n\nWhile some performance drop compared to using the full dataset is expected in dataset distillation, it remains a notable limitation for high-accuracy applications. Despite the improvements over existing methods, concerns remain about potential degradation when scaling to larger or more complex tasks.", "theoretical_claims": "The paper provides solid mathematical justifications that effectively support the proposed approach and offer clear insights into why the techniques work as intended.", "experimental_designs_or_analyses": "The authors conduct comprehensive evaluations of DD3D across diverse datasets and scenarios, demonstrating that it consistently outperforms traditional distillation methods.\n\nTo reinforce the author’s argument, it would be beneficial to compare the performance and feature characteristics of models trained on both aligned and unaligned orientations. Such a comparison would provide clearer insights into the effect of rotation on feature learning and model robustness.", "relation_to_broader_scientific_literature": "The authors present a theoretical analysis showing that matching rotation-invariant features is essential for effective 3D point cloud distillation. They demonstrate that random rotations weaken the principal components of real data, leading to degraded distillation performance, which strongly justifies the use of the proposed rotator.", "essential_references_not_discussed": "Not applicable.", "other_strengths_and_weaknesses": "I mentioned all comments including reasons and suggestions in each section and I recommend that the author will provide my concerns in the rebuttal.", "other_comments_or_suggestions": "The authors should revise several typos and grammatical errors in the paper. (not critical)", "questions_for_authors": "Rather than posing additional questions, I would like the authors to carefully review and address the concerns I have raised above in detail.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qB7nGaesRh", "reviewer": "Reviewer_z87V", "summary": "This paper studies the dataset distillation for point cloud. This paper claims this is the first study on dataset distillation for point cloud, aiming at the two challenges: diverse orientations and resolutions in 3D space. To overcome these issues, this paper 1) proposes a plug-and-play point cloud rotator to align the point cloud to a canonical orientation; 2) devises a point-wise generator\nto produce point clouds at various resolutions based on the sampled noise amount.  The experimental results demonstrate the proposed method achieves higher score than previous methods.\n\n\n### Update after rebuttal\n\nThe authors addressed my concerns. After reading other reviews, I keep my initial rating \"weak accept\".", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive advice and the opportunity to address your concerns.\n\n&nbsp;\n### ``Other Strengths And Weaknesses``\n\n> **Q1: The task model is also trained on the original dataset. How does DD significantly reduce the computational cost?**\n\nA1: We would like to clarify that the task model is not trained on the original datasets. Generaaly, gradient matching employs a bi-level optimization framework:\n- Inner-loop: Optimize task model using **synthetic data**.\n- Outer-loop: Fix the task model and match the gradients between a batch of real and synthetic data.\n\nAfter distillation, the synthetic data contains the gradient information of real data, thereby converging the models with fewer iterations, i.e., batches.\n\n> **Q2: Do we need to further train another task model on the distilled dataset?**\n\nA2: Yes, another task model needs to be trained from scratch. Although the task model is optimized in the inner loop, its performance is far from ideal, as the synthetic data is not fully trained.\n\nOne benefit is that well-trained synthetic data can be used to train different task models. For example, we can use PointNet during distillation and train more complex models, such as DGCNN or Point Transformer, after distillation. A cross-architecture performance is reported in ``Table 3`` of the main text.\n\n> **Q3: For other objects, how to divide the parts? The division strategy impacts the performance and how?**\n\nA3: The parts of objects are defined by the labels of the real datasets. For example, the ShapeNetCore dataset contains 16 object classes and 50 part classes in total. There is a segmentation dictionary to store the correspondence between objects and parts, like {'Airplane': [0, 1, 2, 3], 'Bag': [4, 5], ...}.\n\nIn summary, **the division strategy is fixed and depends on real data**, which will not have different impacts on the performance of DD3D.\n\n> **Q4: Performance of DD3D**\n\nA4: We understand your concerns about the performance of DD. We propose two strategies to mitigate the performance gap between models trained on real and synthetic data\n\n1. Improving the number of CPC. We add the performance of DD3D with CPC=100 in ModelNet40 and ScanObject, and find that it can achieve comparable performance with the full dataset.\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| GM | 81.74 | 84.17 | | GM | 57.52 | 62.82 |\n| DD3D | 83.91 | **86.68** | | DD3D | 61.96 | **65.51** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n2. Combined with knowledge distillation (KD). Recent DD methods suggest that leveraging KD can significantly improve the performance of DD. We can also adopt this strategy to achieve nearly lossless performance.\n| ModelNet40 | CPC=50 | CPC=100 | | ScanObject | CPC=50 | CPC=100 |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n|DD3D | 83.91 | 86.68 | |DD3D | 61.96 | 65.51 |\n|DD3D+KD | 86.55 | **88.75** | |DD3D+KD | 65.06 | **66.84** |\n| Full | 88.05 | 88.05 | | Full | 66.96 | 66.96 |\n\n> **Q5: Comparison with state-of-the-art point cloud analysis models.**\n\nA5: The performance of DD3D is behind the SOTA point cloud analysis models, as we only adopt the vanilla 3-layer PointNet as the distillation network for DD3D, which makes a good trade-off between effectiveness and efficiency.\n\nNotably, in the image distillation task, DD methods often use a 3-layer ConvNet as the distillation network [1]. We report the performance of vision DD (Tiny-ImageNet) and point cloud DD (ModelNet40) below.\n\n| Tiny-ImageNet | 3-layer ConvNet | ResNet-18 |\n| :--: | :--: | :--: |\n| IPC=50 | 37.6% | 59.8% | \n\n| ModelNet40 | 3-layer PointNet | PointNext |\n| :--: | :--: | :--: |\n| CPC=50 | 88.1% | 94.00% |\n\nWe can observe that the performance of the 3-layer ConvNet is far from the ResNet-18 model. But it still promotes the development of vision DD. On the other hand, the performance of the 3-layer PointNet is close to the SOTA models, like PointNext.\nWe believe that our research will also benefit the 3D DD field, and future works will extend DD to the SOTA point cloud analysis models.\n\n[1] Dataset Condensation with Gradient Matching. ICLR 2021.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "do not check carefully.", "experimental_designs_or_analyses": "no issues.", "relation_to_broader_scientific_literature": "Dataset distillation has not yet been studied in the community. This paper, as the first attempts, might be helpful for the community.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\nThis paper makes the first attempt for point cloud dataset distillation, which might provide some insights for the community.\n\nThe proposed method seems to be efficient in comparison with previous methods.\n\nWeaknesses:\n\n=== I am not familiar with this task. ===\n\n- During the dataset distillation process, the task model is also trained on the original dataset. Therefore, how does DD significantly reduce the computational cost of training neural networks from scratch (LINE09, PAGE01)?\n\n- After the distillation process, do we need to further train another task model on the distilled dataset?\n\n- For part segmentation (L338), \"each\nshape is divided into multiple parts. For example, an airplane\ncan be divided into its fuselage, wings, engines, and\ntail.\" For other objects, how to divide the parts? The division strategy impacts the performance and how?\n\n- The performance is low, which impacts the practical application. For example, for modelnet40, current SOTA has achieved over 94% accuracy, while the highest score is 83% in this paper. For part segmentation, the performance lags far behind.", "other_comments_or_suggestions": "-", "questions_for_authors": "-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received positive reviews from all the reviewers. Please include all the promised updates in the final version.
Decision: Accept (poster)