MIRACLE 3D: Memory-efficient Integrated Robust Approach for Continual Learning on 3D Point Clouds via Shape Model Construction
Authors: Hossein Resani
Behrooz Nasihatkon
Venue: ICLR 2025 Poster (2025)
Keywords: Continual learning
Point cloud processing
3D Shape model
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: A novel framework for Memory-Efficient and Privacy-Preserving Continual Learning in 3D object classification, using compact shape models and Gradient Mode Regularization to improve accuracy and reduce memory usage.

Abstract: In this paper, we introduce a novel framework for memory-efficient and privacy-preserving continual learning in 3D object classification. Unlike conventional memory-based approaches in continual learning that require storing numerous exemplars, our method constructs a compact shape model for each class, retaining only the mean shape along with a few key modes of variation. This strategy not only enables the generation of diverse training samples while drastically reducing memory usage but also enhances privacy by eliminating the need to store original data. To further improve model robustness against input variations—an issue common in 3D domains due to the absence of strong backbones and limited training data—we incorporate Gradient Mode Regularization. This technique enhances model stability and broadens classification margins, resulting in accuracy improvements. We validate our approach through extensive experiments on the ModelNet40, ShapeNet, and ScanNet datasets, where we achieve state-of-the-art performance. Notably, our method consumes only 15% of the memory required by competing methods on the ModelNet40 and ShapeNet, while achieving comparable performance on the challenging ScanNet dataset with just 8.5% of the memory. These results underscore the scalability, effectiveness, and privacy-preserving strengths of our framework for 3D object classification.

Published as a conference paper at ICLR 2025
MIRACLE3D: MEMORY-EFFICIENT INTEGRATED Ro- BUST APPROACH FOR CONTINUAL LEARNING ON POINT CLOUDS VIA SHAPE MODEL CONSTRUCTION
Hossein Resani Behrooz Nasihatkon K. N. Toosi University of Technology hossein, resani @mmail.gmma......mmm com nas i masihatkon@kntu.ecc
ABSTRACT
In this paper, we introduce novel framework for memory-efficiett and privacy- preserving continual learning in 3D object classification. Unlike conventional memory- approaches in continual learning that require storing numerous ex- emplars, our method constructs a compact shape model for each class, retaining only the mean shape along with few key modes of variation. This strategy not only enables the generation of diverse training samples while drastically reducing memory usage but also enhances privacy by eliminating the need to store orig- inal data. To further improve model robustness against input variations- an is- sue common in 3D domains due the absence of strong backbones and limited training data we incorporate Gradient Mode Regularization. This technique en- hances model stability and broadens classification margins, resulting in accuracy improvements. We validate our approach through extensive experiments on the ModelNet4O, ShapeNet, and ScanNet datasets, where we achieve state-of-the-ar performance. Notably, our method consumes only 15% of the memory required by competing methods on the ModelNet40 and ShapeNet, while achieving compa- rable performance the challenging ScanNet dataset with just 8.5% of the mem- ory. These results underscore the scalability, effectiveness, and privacy-preserving strengths of our framework for 3D object classification.
INTRODUCTION
Continual learning is an approach in machine learning where models are designed to learn and adapt incrementally from new data over time without forgetting previously acquired knowledge. One of the central challenges this context catastrophic forgetting- the tendency of neural networks to lose previously learned information when adapting to new tasks or data. While significant progress has been made in addressing catastrophic forgetting in 2D images, its application to 3D point clouds still relatively underdeveloped and presents distinct challenges. Unlike the structured grids of 2DD images, 3D point clouds consist of irregularly spaced points, requiring specialized techniques such PointNet (Qi al., (017) and graph neural networks (Wang al., 2019) for effective feature extraction. Moreover, the 3D domain lacks large-scale datasets similar to ImageNet (Deng et al.) 2009) limiting the ability of models learn robust features. These challenges are exacerbated in raplay-based continual learning methods, where traditional exemplar selection techniques, such as Herding (Rebuffi al. 2017 Welling] 2009). struggle with the irregular and multimodal nature of 3D feature spaces. This makes it particularly challenging to retain previously learned features while acquiring new information, heightening the risk of catastrophic forgetting. Addressing these issues in continual learning for 3D point clouds is crucial for enabling adaptive, resilient systems in real- world applications such as autonomous driving, robotics, and augmented reality, where models must continuously learn from evolving stream of sensory data while retaining their ability to accurately recognize past patterns and structures.
In continual learning, common strategy involves replay- based methods (Rebuffi al. 2017 Wu jet al., 2019: Belouadah & Popescul 2018) Chen et al., 2020). where small subsets of data from each task, called exemplars, stored and later replayed alongside current data during training on

---

Published as a conference paper at ICLR 2025
Input samples
Feature space
exemplars
AAA
Encoder
Memory
centroid
(a) Herding strategy
Input samples
Mean shape
Shape Model
Construction
Memory
of
(b) Our method
Figure I: Comparison of the overview of (a) previous methods and (b) our method. Previous meth- ods depend herding strategy (Rebuffi et al. 2017 Welling 2009), which is less effective for 3D point clouds.
new tasks. These exemplars are important because they help mitigate catastrophic forgetting by allowing the model maintain connection to past knowledge while adapting new information. However, traditional approaches in 2D and 3D continual learning that require storing raw input data from previous tasks introduce significant challenges, especially in the context of privacy and memory efficiency. Storing raw data can raise legal concerns under regulations like the European GDPR, where users have the right to request the deletion of their personal data, making compliance difficult if original data must be retained for learning purposes. This issue becomes even more critical when working with sensitive information, such as medical imaging, where the potential for privacy violations is high. Moreover, storing exemplars over multiple tasks leads to an increasing memory burden, which impractical, especially for real-time applications where computational resources are limited. While some 2D continual learning approaches attempt to address this by storing low-dimensional features instead full images (Iscen et al. 2020) or using low-fidelity images (Zhao et al., 2021), this issue remains relatively underexplored in 3D continual learning.
To address these challenges, we propose a novel exemplar generation method focusing on memory efficiency, privacy preservation, and backbone independence. Our approach utilizes the geomet- ric properties of input point clouds to create a compact shape model, storing only the mean shape along with few modes of variation. This not only preserves privacy but also significantly reduces memory requirements. Additionally, we introduce Gradient Mode Regularization to enhance robust- ness against input variations within the shape space. Our approach is also trckbone-independe allowing easy integration with different architectures. Our contributions can be summarized as:
Memory-Effirieee and Privacy-Preservee Shape Generation: Using shape model to represent an entire class objects leads to highly compact representation. As a side effect, it avoids the storage of original data.
Gradient Mode Regularization: Penalizing the gradient of the loss along the principal modes of shape variation enhances the representational capabilities of samples drawn from shape models.
State-of-the-Arr Performance: Achieving state-of-the-art performance on three major point cloud benchmarks: ModelNet40 (Wu et al. 2015), ShapeNet (Chang al. 2015), and ScanNet (Dai et al. 2017). In the final incremental session, achieves improvements of 4.9%, 2.1 %, and 1.8% ModelNet40, ShapeNet, and ScanNet, respectively, while using only 15% of the memory compared to the sfate-of-the for ModelNet40 and ShapeNet, and just 8.5% of the memory for ScanNet.
2

---

Published as a conference paper at ICLR 2025
2
RELATED WORK
2.1 SHAPE REPRESENTATION VIA STATISTICAL SHAPE MODELS
Shape representation involves methods that describe the geometry and structure of an object, en- abling efficient analysis, manipulation, and comparison. Statistical Shape Models (SSMs) are widely recognized approach for shape representation, providing low-dimensional, parametric model of complex objects. Typically, SSMs are constructed by applying Principal Component Anal- ysis (PCA) to set of objects in dense correspondence. This yields a linear model in which shapes are represented as points in a low-dimensional affine vector space. While non-linear and multi-linear variants exist, PCA-based SSMs remain the most commonly used in practice due to their simplicity and effectiveness. (Cootes et al.
1995). SSMs are extensively used in computer vision, particularly for modeling human-related Teatures such as faces, bodies (Ambellan et al. 2019). bones (Raja-
mani et al., 2004] Sarkalkan et al. 2014), and organs (Weiherer et al. [023 Their applications
span across numerous fields, including face recognition (Blanz & Vetter 2003] Feng et al. [02T]
Li et al.] 2023; Zielonka et al. (222)) and body reconstruction, as well as various areas in medicine (Fouefack et al.] 2020 Lüthi et al. 2017: Weiherer et al.) 2023), forensics, cognitive science, neu- roscience, and psychology (Egger et al. 2020a). Recent advancements (Loiseau et al. 2021 Raju ict al.] 2022), explore new methodologics for improving shape representations, including geomet- ric deep learning techniques and improved handling of non-linearities. In this study, we develop compact shape model for each class by preserving only the mean shape and a select number of key variation modes.
2.2 3D CONTINUAL LEARNING
Despite significant progress in 2D continual learning, 3D continual learning remains relatively un- derdeveloped. Several recent work aim to address challenges such as catastrophic forgetting and data sparsity. Chowdhury et al. (2021) introduced method leveraging knowledge distillation and semantic word vectors to mitigate the forgetting of prior training. Zhao & Lee (2022) addressed class-incremental 3D object detection by using a static teacher for pseudo annotations of old classes and dynamic teacher to continually learn new data. Zamorski et al. (2023) proposed Random Com- pression Rehearsal (RCR), which employs a compact, autoencoder- model to compress and store critical data from previous tasks. Camuffo & Milani (2023) developed a continual learning approach for semantic segmentation in LiDAR point clouds, tackling coarse-to-fine segmentation challenges and data sparsity issues. Chowdhury et al. (2022) proposed using Microshapes orthogonal basis vectors to describe 3D objects-to to help align features between synthetic and real data, reducing domain gaps and enhancing robustness against real-world noise. I3DOL (Dong et al. (02I) uses an ceaptive-geomettr approach to handle irregular point clouds and an attention mechanism to fo- significant geometric structures, aiming to minimize forgetting. It also introduces fairness compensation strategy to balance training between new and old classes. Later, InOR-Net (Dong et al. 2023) improved on this by enhancing geometric feature extraction and incorporating dual fairness strategy to effectively manage class imbalances and prevent biased predictions. However, both methods still suffer from inefficiencies, as they rely on the herding approach (Rebuffi et al.) 2017 Welling 2009), which struggles to effectively capture diversity in 3D point clouds-a crucial issue highlighted by Resani et al. (2024). Addressing this limitation by developing an exemplar selection approach specifically designed to handle the unique characteristics of 3D data remains an important and underexplored research area.
CL3D Resani et al. 2024) demonstrates that the herding approach (Rebulfi et al. 2017: Welling 2009) commonly used in the 2D domain, is unsuitable for the 3D domain. introduces novel exemplar selection strategy for 3D continual learning, utilizing spectral clustering and combining input, local, and global features, achieving state-of-the-art performance. While the use of raw input features offers independence from network-specific dependencies, the CL3D method still relies on network-derived features for local and global clustering. Moreover, storing original data in memory raises concerns regarding privacy preservation, which serves major motivation for our work. In our approach, we propose a novel exemplar generation strategy that avoids storing original data, thereby ensuring privacy preservation.
3

---

Published as a conference paper at ICLR 2025
Shape Construction
Sample Generation
New Samples
X
Memory
+
oo
Point Cloud Network
Logg Lxn
No
Cr (nL-(BCo, (ON,)
1x
Shared
Session
Session t+1
Point Cloud
LaD
Network
Novel Class
Figure 2: Overview our method. For each old class, a compact shape model is created using the mean shape and few key modes of variation derived from SVD, which are stored in memory. the next session, new samples are generated by applying perturbations to the mean shape using these modes. These generated samples, along with novel class data, are processed through shared point cloud network, trained using cross-entropy, knowledge distillation, and Gradient Mode Regu- larization to enhance robustness and prevent forgetting.
REPLAY-BASEL CONTINUAL LEARNING SETTING
Consider a sequence of disjoint tasks = (D), ..... D"). and let C (ci...., m be the set of target classes in task D We assume that the classes between tasks are disjoint, i.e., C n for i of The goal of continual learning is progressively train model, where at each session only the training samples from the current task D = {(X*, Y") consisting of point clouds X and their corresponding labels y accessible. During testing. the model trained task D is expected to predict outputs not only for the current task but also for all prior tasks D) ..... Dt-1
3.1 EXEMPLAR MEMORY MANAGEMENT
In continual learning, memory management for storing exemplars typically follows two strategies (Zhou et al. 2023). The first is to maintain a fixed number of samples per class, ensuring consistent representation as new tasks are added, though leading to a linearly increasing memory size. The second is to impose fixed overall memory cap, which stabilizes memory usage but may reduce class representation as more tasks accumulate. Here, we adopt the first strategy.
PROPOSED METHOD
continual learning model for 3D point cloud classification. By storing only the mean shape and key variations for each class, we sig- nificantly reduce memory usage. Gradient Mode Regularization further enhances robustness during incremental sessions. An overview of MIRACLE3D is shown in fig. 2 and we provide further details in the following sections.
Our approach develops a memory-efficient,
4.1 PRELIMINARIES
In morphable shape models (Egger al. 2020b), samples of object class represented as mean shape plus linear combination of several basis shape modes. Consider a set of point clouds X1, ......... X......... .m each being a sample from specific object class. These point clouds are assumed to be pre-registered and resampled, so that each contains exactly n points, with all points correspondence. Hence, each point cloud sample can be expressed by an nx3 matrix X, E Rax3 The shape model comprises the mean shape X = 1 Σn X; E RRx3 principal modes of variation
V1, V2, V2...., Vę E R and their corresponding scales 0 02, ..,, we vectorize the mean-centered samples, resulting in y; = vect - X) E R3 The modes V; are then obtained as the first & left singular vectors of the matrix Y yy, Y2........ : Vml R3axm which are subsequently reshaped into n 3 matrices. The scalars σ; are the associated singular values. In many cases, the samples X, assumed to be normally distributed. Thus,
R. To derive these modes,
shape can be
4

---

Published as a conference paper at ICLR 2025
generated as:
X x Σ Go
where R are sampled from standard normal distribution.
4.2 PROTOTYPES AS SHAPE MODELS
The main idea of our approach is to represent the samples of an object class e using a compact shape model. This shape model is defined by three main components: the mean shape X set of modes of variation Vi, Vg...., V E R2x3 and the associated scaling factors of. , E IR, where the superscript indicates that the model belongs class c. Here, & is kept small to maintain memory efficiency while capturing the essential variability within the class. The modes of variation, together with the scaling factors, capture the key dimensions along which the shape of class samples can vary, allowing for a compact representation of the shape diversity. When novel classes are introduced in the continual learning scenario, we need to generate samples from previously learned classes to mitigate catastrophic forgetting. To accomplish this, we generate synthetic samples from the old class e using
== X a Σ
6 o Vg.
This equation is similar to the more general form presented in ① but includes adjustment: scalar coefficient a < 1. This coefficient is deliberately chosen to be small (around 0.2) in order to restrict the generated samples
high-probability region of the shape space. By using a smaller value of a, we ensure that the generated shapes represent the most typical and probable configurations of the class. For each old class c, we generate n, synthetic samples {Zş )= These generated samples serve as proxy for the original data of the old classes, and they are used alongside the novel class samples to retrain the model, thereby preventing it from forgetting previously acquired knowledge. Note that the samples are not stored in memory but are generated on demand. This process allows the model to maintain its performance across both old and novel classes, ensuring a more balanced and memory-efficied continual learning capability. The results of shape modeling, along with examples of generated point cloud samples, are presented in fig.
4.3 LOSs FUNCTION
Our loss function defined as
nx
((0)
Σ Σ (Ccx(0,X),.)) (x.(0.X,, +
(6 2j,
(3)
OECas
=;=c
where, / is the network parameters, Cnew and Cota are the sets new and old classes, respectively, LCE is the cross-entropy loss, CXD is the knowledge distillation loss (Li & Hoiem 2017), and LFL the focal loss (Lin et al., 2017)
Cn, 2j. -o, -1 (o(02,)))))) log log(P.(00,z)),
(0, Z.) is the output softmax probability of the network for class As suggested in Resani et al. (2024). we apply focal loss to tackle the class imbalance issue, given that the number of samples n, for the old classes is generally significantly smaller than the number of samples for the new classes.
where P0(
4.4 GRADIENT MODE REGULARIZATION
Regularizing the gradient norm of a neural network's output with respect to its inputs is the idea of Double Backpropagation (Drucker & Le Cun 1991), aimed at reducing sensitivity and promoting stability in model learning (Varga et al. 2017). ). This technique helps control output changes in response to input variations, thus enhancing robustness against adversarial attacks (Lee al. 2022
5

---

Published as a conference paper at ICLR 2025
Ross & Doshi Velez 2018: Finlay & Oberman 2021 and improving generalization (Rame al. ).22), particularly with small training datasets. Our idea here to perform gradient regularization in the directions of the major modes of variation Vç-
Consider a sample zj derived from a shape model using equation 2 When a small perturbation is introduced along the direction of each mode of variation Vž. it is crucial ensure that the resulting shape maintains a low loss value. To ensure this, for the old class samples 2/, we replace the focal loss (0, 2 c) in (3) with regularized loss:
Le (0, Cn.
5 +
(0,, c)/22 v9)2
where (:.-) denotes the inner product, (f, 22.c))..0./ c)/02j represents the gradient the loss function with respect to the network input Zj, and is balancing hyperparameter. The added regularization term encourages the directional derivative of the cost function to be close to zero in the direction of the modes Vy, which helps ensure that the modified shape remains consistent with the original shape distribution after small perturbations along these modes. fig. iillustrate our Gradient Mode Regularization concept. In the absence of Gradient Mode Regularization, variations in input and small perturbations can lead to decreased efficiency, pushing the model away from the low-error region of previously learned classes. By applying Gradient Mode Regularization, our approach preserves larger margin from the low-error region of old classes, thereby enhancing robustness and stability against such changes.
5 EXPERIMENTS
We adopt the experimental setup used by 13DOL (Dong et al.) (021). InOR-net (Dong jet al. 2023), and CL3D (Resani al. 2024), including the datasets, backbone architecture, and the number of incremental sessions. To the
0
0 *
Low omor for oid task
best of our knowledge, these are the only exist- ing studics that focus on exemplar-bar based con- tinual learning for point clouds, making them the most suitable for comparison with our re- sults.
2
region
Sub-quimal region
No Panalty ('orgeting)
Sub-qrimal upolile
Dan uptien
Figure 3: Illustration of Gradient Mode Reg- ularization. The figure illustrates the parameter spaces Of and 05, highlighting the low-error re- gions for the old and novel classes, respectively. Without any regularization, model updates (blue arrow) may lead to catastrophic forgetting. Ad- ditionally, the absence of regularization can re- sult in sub-optimal updates (red arrow), making the model sensitive small perturbations, even within the shape space. In contrast, our method (green arrow) employs Gradient Mode Regular- ization to steer updates toward more reliable gion for the old classes, providing a larger margin than previous methods. This approach improves the model's 's stability and robustness against per- turbations during continual learning.
Datasets. We conduct our evaluation using three datasets: ModelNet40 (Wu et al., 2015). ShapeNet (Chang et al. 2015), and ScanNet (Dai et al. 2017). ModelNet40 contains 40 classes of clean 3D CAD models. In align- ment with Dong al. (2021) and Dong et al. (202)), we utilize 10 incremental sessions, each adding four new classes. Our method stores the mean shape of each class alongside two modes of variation, resulting in three compo- nents stored per class, This results in total memory usage of 120 samples caaples--rerrsenttiiie an 85% reduction compared to methods I3DOL, and Neoo--ee-hiighighting the memory effi- ciency aspect of our approach.
For the ShapeNet dataset, we use 53 categories, consistent with 13DOL and InOR-net Dong jtt al. (2023). We follow their configuration of nine incremental sessions, with each session introducing six new classes, except for the final ses- sion which introduces five classes. Since the ShapeNet dataset is already aligned, no alignment preprocessing is required, and only resampling is performed.

---

Published as a conference paper at ICLR 2025
Table Accuracy comparison on the ModelNet40 (Wu al. 2015). ShapeNet (Chang al. 2015). and ScanNet (Dai et al. 2017) datasets. The table reports the average accuracy across all sessions (Avg.). the accuracy of the last session (Last), and the total number exemplars stored memory (M). Notably, unlike other approaches, our method does not store any original samples. The term "Join"" represents learning classes simultaneously (upper bound), while "/7" indicates sequen- tial learning of sessions without any forgetting mitigation (lower bound). Best results are highlighted in bold and second-bes results are underlined. The table is an extended version of Dong et al. (2023)
ModelNet40
ShapeNet
ScanNet
Method
Aaeg
A
M
Aavg
A
M
Aavg
Atast
M
Joint (Upper bound) FT (Lower bound)
94.3
88.5
all
93.3
89.3
all
93.0
91.0
all
28.0
9.2
26.9
6.2
0
30.1
8.2
0
2D Approaches Applied on 3D Point Clouds
LwF
Hoiem, 2017
60.3
31.5
63.4
39.5
53.1
38.1
iCaRL
Reburt 2017
68.9
39.6
800
69.5
44.6
1000
56.0
36.3
600
DeeSIL
Belouadah Popescu, 2018
72.1
43.7
800
71.7
47.2
1000
63.1
43.7
600
EEIL
Castro et all 2018
75.0
48.1
800
74.2
51.6
1000
65.1
45.7
600
IL2M
Bolouadah & Popescu 2019
81.1
57.6
800
77.6
61.4
1000
66.7
48.3
600
DGMw W Ches
2020
77.2
53.4
800
73.8
49.2
1000
63.6
43.8
600
DGMa
Chen al. 2000
76.8
51.5
800
73.4
48.7
1000
63.7
44.7
600
BiC Wa
2019
48.5
66.8
800
78.8
64.2
1000
66.8
48.5
600
RPS- Net Rajasegarnn et al.
2019
81.7
58.3
800
78.4
63.5
1000
67.3
49.1
600
3D Specific Methods
I3DOL
Dong al.,
85.3
61.5
800
81.6
67.3
1000
70.2
52.1
600
InOR- Neet
let Done al. 2023)
87.0
63.9
800
83.7
69.4
1000
72.2
54.8
600
CL3D
85.1
69.7
120
80.8
67.6
120
71.8
55.4
120
MIRACLE3D (Ours)
88.1
74.6
120
84.9
71.5
120
72.9
57.2
120
The ScanNet dataset consists of 17 classes collected from real indoor scene scans. We use the FilterReg (Gao Tedrake, 2019 to register the samples before resampling them. Our approach stores three components per class. We conduct nine incremental sessions, each introducing two new classes, with the final session adding just one class.
Evaluation Metric and baselines. In table [ We report the average accuracy after the final ses- sion, denoted as Ajsst- along with the average incremental accuracy, denoted as Aavg. which repre- sents the average of accuracies after all sessions (including the initial one). For a more complete analysis, include the scenario where the network has access to the entire dataset from previous tasks (joint training), which serves as an ideal upper bound. Conversely, as a lower bound, we pro- vide results for the scenario involving complete forgetting (denoted as FT in the tables), where the model updates its parameters exclusively based on new tasks. Furthermore, we include the results of well-known 2D approaches applied to 3D point clouds demonstrate their shortcomings this context, emphasizing the necessity for dedicated 3D-specific methods.
Implementation Details. MIRACLE3D is implemented in PyTorch and trained on a Tesla GPU with a batch size of 32. We use PointNet Qi et al. (2017) the backbone for fair comparison with prior work (Dong et al. 2021 2023), though our approach not tied to any specific architecture. The Adam optimizer is used with 50 epochs per incremental session. Focal loss addresses data imbalance between old and novel classes, and the distillation factor is set to O.I. We generate 10 samples per class during training, which are not stored in memory.
7

---

Published as a conference paper at ICLR 2025
0gV2
X
2- ogVa
Mean shape
First two modes variation
Generated Samples
Figure 4: Results of Shape Modeling: Mean shape (left), first principal modes of variation (center), and examples of generated point cloud samples (right).
COMPARISON
debonsseettt comparison datasets including ModelNet40 dataset (Wu al., 2013). ShapeNet dataset (Chang et al., 2013) and ScanNet (Dai et al.] 2017). In the ModelNet40 dataset, our proposed method, M/RACLE3D, outperforms all competing approaches in terms of both average accuracy (Awg)) and the accuracy of the last session (Auss), achieving 88. 15.1% % and 74.6%, respectively. These results indicate sigmificant improvement over other 3D-specific methods like 13DOL and InOR-Net, which achieved lower average accuracies of 85.3% and 87.1%. respectively. Our memory usage (M) is notably efficient, requiring only 120 exemplars, compared to the 800 or 1000 exemplars used by other methods.
Results.
The ShapeNet dataset (Chang et al., 2015) includes more classes compared to ModelNet40 and ScanNet, making it more complex scenario. On this dataset, MIRACLE3D achieves an average accuracy of 84.9% and last session accuracy of 71 71.5%, outperforming other 3D-specific methods.
The NeanNet dataset (Dai et al. 2017), being a real-world point cloud dataset, introduces further challenges due to inherent noisiness, incompleteness, and other complex characteristics typical of real-world scans. Despite these challenges, MIRACLE3D achieved average accuracy of 72.9% and last session accuracy 57. 97,2% outperforming all other approaches evaluated, including CL3D and InOR-Net. which achieved average accuracies of 70.0% and 72.2%, respectively. Moreover, the adapted 2D approaches, such as RPS-Net, fell behind significantly, indicating that methods designed specifically for 3D point clouds are necessary for effective learning in real-world scenarios. Impor- tantly, MIRACLE3D remains memory efficient with only 120 exemplars compared to the larger memory footprints of other methods.
5.2
ABLATIONS AND ANALYSIS
5.2.1 EFFECT OF GRADIENT MODE REGULARIZATION
The effect of Gradient Mode Regularization is evaluated determine its impact on model perfor- mance in our setting. As shown in table 2 incorporating this regularization led to improvements in both average accuracy (Awyg) and last session accuracy (Atss). Specifically, Gradient Mode Regu- larization helped stabilize the model by reducing its sensitivity to input variations along key modes of shape variation, thereby enhancing robustness against the inherent variability of 3D point clouds.
Table 2: Comparison of average accuracy (Awug) and last session accuracy (Aust) with and without Gradient Mode Regularization on the ModelNet40, ShapeNet and ScanNet datatsets.
ModelNet40
ShapeNet
ScanNet
Method
Auyg
Ajast
Auvg
A
Aavg
A Ajast
W/o Gradient Mode Regularization
87.5
72.3
84.1
69.4
72.3
56.1
W Gradient Mode Regularization
88.1
74.6
84.9
71.5
72.9
57.2
8

---

Published as a conference paper at ICLR 2025
5.2.2 OPTIMAL NUMBER OF VARIATION MODES
A core focus of our approach optimizing memory efficiency, aiming to achieve minimal mem- ory usage while outperforming state-of-the-art methods. Through extensive experimentation with different numbers of modes of variation, as shown in table 3 we found that configurations with or 4 modes consistently offered superior performance compared to other configurations. However, given our emphasis on minimizing memory consumption, we chose to store only modes of vari- ation along with the mean shape. It's important to note that the optimal number of modes can vary depending on the variability within object classes and the quality of the shape models.
Table 3: Comparison of average accuracy (Aavg) and last session accuracy (Alast) using different combinations of shape components on the ModelNet40 dataset.
Mean Shape
1st
2nd
3rd
4th
Aarg (%)
Last (%)
81.3
66.8
84.5
69.5
88.1
74.6
87.6
72.1
88.1
74.8
5.2.3 NUMBER OF GENERATED SAMPLES
We conducted experiments to determine the optimal number of generated samples as shown in ta- ble 4 We found that 10 samples yielded the best performance, surpassing other configurations. It is important to note that these samples are not stored in memory; rather, they are generated on-the- fly solely for training purposes. When we increased the number of samples from 10 to 20, did not observe any improvement in performance. The lack of improvement may be attributed to an increase in the number of degraded samples within certain classes (as will be briefly discussed in the discussion), which could negatively impact the learning process.
Table 4: Comparison of average accuracy (Aavg) and last session accuracy (Anaa) with varying num- bers of generated samples on the ModelNet40 dataset.
No. of generated samples
Aavg (%)
Atast ( (%)
5 samples
85.9
69.8
10 samples
88.1
74.6
20 samples
87.3
72.3
5.2.4 PERFORMANCE ACROSS OTHER BACKBONES
We adopted PointNet Qq 2017) as the backbone for our model to ensure fair comparison with existing work in 3D continual learning (Dong et al. 2021 2023): Resani et al. (2024)). A key advantage of our approach, however, lies in backbone independence. Since our model operates directly in the input space, it can be seamlessly integrated with other backbones. To demonstrate this, we conducted additional experiments using more recent backbones, specifically DGCNN (Wang al. (2019) and PointMLP-clice (Ma et al. (2022)). as shown in table 5 One of the reported metrics, the Forgetting Rate (FR), measures the performance gap between joint training (the upper bound) and the model's actual performance. Our results show that because method does not rely on backbone-specific features, the forgetting rate remains relatively consistent across different architectures. These findings highlight that our method achieves similar effectiveness regardless of the backbone, reinforcing the backbone independence of our approach.

---

Published as a conference paper at ICLR 2025
Table Comparison of performance across different backbones in terms of average accuracy (Ours Awg), last session accuracy (Ours Ausa), and forgetting rate (FR). Joint training results are included as the upper bound. The relatively consistent FR across different backbones highlights the backbone independence of our model.
Backbones
Ours (Aavp)
Ours (Atast
Joint (Awg)
Joint (Alast
(%)
PointNet QQi Ot al. 2017
88.1
74.6
94.3
88.5
6.2
DGCNN Wang al. 2019
89.1
79.8
95.4
92.1
6.3
PointMLP Peelle -eile (Ma cl al. 2022
89.3
82.3
95.8
93.0
6.5
5.2.5 RANDOMNESS EFFECT IN SHAPE MODEL.
The scalar coefficient a introduced in eq. acts as a control variable that adjusts the level of variation in the generated samples. We deliberately set samples remain within a high-probability region of the shape space. By using a smaller value for a, the generated shapes are more likely to represent the most typical and probable configurations for each class, Through our experimentation in table 6 we found that an optimal value for this hyperparameter in our setting is a 0.2, which effectively balances the trade-off between diversity and representativeness.
small value ensure that the generated
Table 6: Comparison average accuracy (Auvg) and last session accuracy (Auss) with three different levels of randomness: low (0.1, 0.2), medium (0.5), and high (1).
Randomness weight
Anvg ( (%)
Auust ( (%)
0.1
87.9
73.2
0.2
88.1
74.6
0.5
85.8
68.9
1
83.6
65.4
6 DISCUSSION
Our MIRACLE3D model demonstrates impressive capabilities; however, certain challenges persist, particularly due the variability within some classes, which complicates smooth morphing between dissimilar samples. Integrating clustering methods could help by grouping more similar samples to- gether, as suggested by Resani al.] (2024). Additionally, since our approach is based on point clouds, ensuring registration accuracy is crucial. Our current affine methods occasionally produce nonsensical and inaccurate shapes, although even these contribute to mitigating forgetting. This indicates that adopting nonlinear deformable registration could yield better results. Another promis- ing direction is developing a direct neural network-based shape model (Loiseau et al.) (021)) which presents challenges in terms of memory efficiency. To address these challenges, techniques like knowledge distillation (Hinton et al. 2015) could help reduce storage requirements for neural rep- resentations, making this approach more viable.
7 CONCLUSION
In this paper, we presented MIRACLE3D, a novel memory-efficient, privacy-preserving continual learning approach tailored for 3D point cloud classification. By constructing compact shape model that captures the mean and key modes of variation for each class, our method significantly reduces memory requirements while avoiding the need to store raw data. Additionally, we introduced Gradi- ent Mode Regularization to enhance model robustness, effectively mitigating catastrophic forgetting even under challenging input variations. A key strength of our method lies in its backbone indepen- dence, as it operates directly in the input space, enabling seamless integration with various 3D deep learning architectures. Our experiments, conducted across the ModelNet40, ShapeNet, and ScanNet datasets, demonstrated that MIRACLE3D achieves state-of ofte-offthhe-rr performance, outperforming isting 3D-specific continual learning methods with just a fraction the memory, regardless of the backbone used.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Felix Ambellan, Hans Lamecker, Christoph von Tycowicz, and Stefan Zachow. Statistical shape models: understanding and mastering variation in anatomy. Springer, 2019.
Eden Belouadah and Adrian Popescu. Deesil: Deep-shallow incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0-0, 2018.
Eden Belouadah and Adrian Popescu. I12m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
Volker Blanz and Thomas Vetter. Face recognition based on fitting a 3d morphable model. IEEE Transactions on pattern analysis and machine intelligence, 25(9):1033-1074, 2003.
Elena Camuffo and Simone Milani. Continual learning for lidar semantic segmentation: Class- incremental and coarse-to-fine strategies sparse data. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Patlern Recognition, pp. 2447-2456, 2023.
Francisco M. Castro, Manuel Marin- JamennJ Nicolas Guil, Cordelia Schmid, and Karteek Ala- hari. End-to-end incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
A. X. Chang, A. Funkhouser, I.. J. Guibas, Hanrahan, Q. Huang, 7. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, I. Yi, and F. Yu. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:
2015.
Xiuyi Chen, Fandong Meng, Peng Li, Feilong Chen, Shuang Xu, Bo Xu, and Jie Zhou. Bridg- ing the gap between prior and posterior knowledge selection for knowledge knowledge-grounded dialogue generation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.). Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP)), pp. 3426- 3437, Online, November 2020. Association for Computational Linguistics. doi: 10. 18653vvl 2020.emnlp-main.275. Utt/ttpp:: http://aaaaaaaaaaaaca/ee org/ 20m.1.000..m.......mmmmmmmmmmmm. main. 275
Townim Chowdhury, Mahira Jalisha, Ali Cheraghian, and Shafin Rahman. Learning without forget- ting for 3d point cloud objects. In Advances Computational Intelligence: 16th International CorkkCoonerence on Artificial Neural Networks, /WANN 2021, Virtual Event, June 16-18, 2021, Proceedings, Part 16, pp. 484-497, Springer, 2021.
Townim Chowdhury, Ali Cheraghian, Sameera Ramasinghe, Sahar Ahmadi, Morteza Saberi, and Shafin Rahman. Few-shot class-incrementee learning for 3d point cloud objects. In European Conference on Computer Vision, pp. 204-220 Springer, 2022.
Timothy F Cootes, Christopher Taylor, David H Cooper, and Jim Graham. Active shape models- their training and application. Computer vision and image understanding, 61(1):38 59, 1995.
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nicßsner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 245-255. leee, 2009.
Jiahua Dong, Yang Cong, Gan Sun, Bingtao Ma, and Lichen Wang. 13dol: Incremental 3d object learning without catastrophic forgetting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 6066 6066-6074, 2021.
Jiahua Dong, Yang Cong, Gan Sun, Lixu Wang. Lingjuan Lyu, Jun Li, and Ender Konukoglu. Inor- net: Incremental 3-d object recognition network for point cloud representation. IEEE Transac- tions on Neural Networks and Learning Systems, 2023.
Harris Drucker and Yann Le Cun. Double backpropagation increasing generalization performance. In JJCNN-91 -e-atlle International Joint Conference on Neural Networks, volume 2, pp. 155-150. IEEE, 1991.
11

---

Published as a conference paper at ICLR 2025
Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3d morphable face models past, present, and future. ACM Transactions Graphics 396)),9(5)11-3 2020a.
Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3d mor- phable face models-pass present, and future. ACM Transactions on Graphics (ToG), 39(5): 1-38, 2020b.
Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the-wild images. ACM Transactions on Graphics (ToG), 40(4))1-13, 2021.
Chris Finlay and Adam M Oberman. Scaleable input gradient regularization for adversarial robust- ness. Machine Learning with Applications, 3:100017 2021.
dean-Rassaire Fouefack, Bhushan Borotikar, Tania S Douglas, Valérie Burdin, and Tinashe EM Mutsvangwa. Dynamic multi-object gaussian process models. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 755-764. Springer, 2020.
Wei Gao and Russ Tedrake. Filterreg: Robust and efficient probabilistic point-set registration us- ing gaussian filter and twist purameterization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11095-11100 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015.
Ahmet Iscen, Jeffrey Zhang, Svetlana Lazebnik, and Cordelia Schmid. Memory-effiiient incremen- tal learning through feature adaptation. In Computer Vision-ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pp. 699-715. Springer, 2020.
Sungyoon Lee, Hoki Kim, and Jaewook Lee. Graddiv: Adversarial robustness randomized neu- ral networks via gradient diversity regularization. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2): 2651, 2022.
Chunlu Li, Andreas -orel-Forster, Thomas Vetter, Bernhard Egger, and Adam Kortylewski. Robust model-based face reconstruction through weakly- superviee outlier segmentation. P Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 372-381, 2023.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-290 2017.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Romain Loiseau, Tom Monnier, Mathieu Aubry, and Loic Landrieu. Representing shape collections with alignment-aware linear models. In 2021 International Conference on 3D Vision (3DV), pp- 1044-1053 IEEE, 2021.
Marcel Lüthi, Thomas Gerig, Christoph Jud, and Thomas Vetter. Gaussian process morphable mod- els. IEEE transactions on pattern analysis and machine intelligence, 10(8):1860-111 2017.
Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: simple residual mlp framework. arXiv preprint arXiv:2202.07123, 2022.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
Kumar T Rajamani, Johannes Hug, Lutz Peter Nolte, and Martin Styner. Bone morphing with statistical shape models for enhanced visualization. In Medical Imaging 2004: Visualization, Snage-Guided Procedures, and Display, volume 5367, pp. 122-130. SPIE, 2004.
12

---

Published as a conference paper at ICLR 2025
Jathushan Rajasegaran, Munawar Hayat, Salman H Khan, Fahad Shahbaz Khan, and Ling Shao. Random path selection for continual learning. In H. Wallach, H. Larochelle, A. Beygelzimer. F. d'Alché-Buc, E. Fox, and R. Gameett (eds.), Advances in Neu- ral Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL. 8 https::/prroeediingg neuri ps cc pacc/pape____ httpr//eeeeeeappe///1//////
737e9e Paper pf pdf
Ashwin Raju, Shun Miao, Dakai Jin, Le Lu, Junzhou Huang, and Adam P Harrison. Deep implicit statistical shape models for 3d medical image delineation. In proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 2135-2143, 2022.
Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of-distribution generalization. In International Conference on Machine Learning, pp. 18347- 18377. PMLR, 2022.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Hossein Resani, Behrooz Nasihatkon, and Mohammadreza Alimoradi Jazi. Continual learning in 3d point clouds: Employing spectral techniques for exemplar selection. arXiv preprint arXiv:2409.08388 2024.
Andrew Ross and Finale Doshi-Veled, Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Nazli Sarkalkan, Harrie Weinans, and Amir A Zadpoor. Statistical shape and appearance models of bones. Bone, 60:129-140, 2014.
Dániel Varga, Adrián Csiszárik, and Zsolt Zombori. Gradient regularization improves accuracy of discriminative models. arXiv preprint arXiv:1712.09000 2017.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael Bronstein, and Justin M Solomon. Dynamic graph enn for learning point clouds. ACM Transactions Graphics (tog), 38(5): 1-12, 2019.
Maximilian Weiherer, Andreas Eigenberger, Bernhard Egger, Vanessa Brébant, Lukas Prantl, and Christoph Palm. Learning the shape of female breasts: an open-access 3d statistical shape model of the female breast built from 110 breast scans. The Visual Computer, 39(4):1597-1616 2023.
Max Welling. Herding dynamical weights to learn. Proceedings of the 26th annual international conference on machine learning, pp. 1121-1128, 2009.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), June 2019.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
Maciej Zamorski, Michat Stypułkowski, Konrad Karanowski, Tomasz Trzciński, and Maciej Zieba. Continual learning 3d point clouds with random compressed rehearsal. Computer Vision and Image Understanding, 228:103621 2023.
Hanbin Zhao, Hui Wang, Yongjian Fu, Fei Wu, and Xi Li. Memory--fficient dlass-incrementa learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 33(10))5966-597) 2021.
Na Zhao and Gim Hee Lee. Static-dynamic co co-teaching for inasemnncremltall 3d object detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 2022.
13

---

Published as a conference paper at ICLR 2025
Wei Zhou, Qi-Wei Wang, Zhi Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Deep ilass-incrementall learning: survey. arXiv preprint arXiw:2302.03648, 2023.
Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European conference on computer vision, pp. 250-269, Springer, 2022.
14


Reviews:
[{"review_id": "3BXO7fnTkj", "reviewer": "Reviewer_HZdC", "summary": "MIRACLE 3D proposes to learn shape models from point cloud data for more efficient representation of data in continual learning scenarios. The authors use the shape model mean + variations to learn the data for a continual setting and then perform gradient regularization to promote stability of gradients to keep a balance for old and new classes. The compact representation leads to significant reduction in memory footprint. The method is evaluated on 3 popular datasets- ModelNet40, ScanNet, and ShapeNet where it performs well on classification.", "questions": "* Learning a new task + exploring if the order of tasks matter in point cloud data might be interesting to see", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper is mostly well written and the method is clearly explained\n* Significantly reduced memory footprint compared to exisiting methods as it uses shape models to represent the data\n* An ablation is performed and the paper also explores optimal number of modes and number of samples to generate for continual learning", "weaknesses": "* The paper talks at length about continual larning in a setting which has multiple tasks but the proposed method is only shown for classification where new classes are learned. This makes the pitch of the paper seem confusing with the results. Moreover, the case for learning new classes is much weaker compared to learning new tasks (+ classes). Can the authors explore learning new tasks such as part segmentation andnobject retrieval? \n* The case for using a shape model to reduce memory footprint for continual learning as the main contribution is weak in my opinion \n* The proposed gradient mode regularization leads to minimal improvement in accuracy for classification. It could be that the extra stability is particularly helpful for learning new tasks in which case such a scenario should be tested. Metrics other than accuracy could help to make a more copelling case.\n\nPost rebuttal:\nSee reply.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PoQrdppKf1", "reviewer": "Reviewer_DwN5", "summary": "The authors propose an exemplar generation method for memory-efficient and privacy-preserving continual learning in 3D object classification. Additionally, Gradient Mode Regularization is introduced to enhance robustness against input variations.", "questions": "see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed method achieves state-of-the-art results on widely-used datasets.\n- Compared with previous methods, MIRACLE3D is memory-efficient.", "weaknesses": "- Lack of ablation study. Only the Gradient Mode Regularization is evaluated on ModelNet. \n- How does Geometry-awareness benefit for this task? \n- The authors claim MIRACLE3D is backbone independent. However, all the experiments are conducted only on pointnet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3c4mbQmsfP", "reviewer": "Reviewer_d8L5", "summary": "The paper also draws on ideas from 2D continual learning approaches, particularly techniques that leverage low-fidelity images to enable memory-efficient learning. By borrowing these concepts, MIRACLE 3D adapts low-dimensional representation strategies to 3D point cloud data, tackling the unique challenges posed by 3D continual learning. It enables continual learning for 3D object classification with a focus on memory efficiency and privacy preservation.", "questions": "[1] Could the authors display the mean shape along with a few primary modes of variation for selected classes? This visualization could clarify the model’s representational capacity and robustness for each class and would make it easier to evaluate how well the shape model captures intra-class diversity.\n\n[2] Can the authors evaluate MIRACLE 3D on datasets with greater intra-class variation, such as SemanticKITTI or nuScenes, to assess scalability? This would provide a stronger validation of MIRACLE 3D’s scalability and potential adaptations for applications with high variability, giving a more comprehensive view of its robustness in real-world settings.\n\n[3] Would the authors conduct a more thorough ablation study focusing on key hyperparameters, specifically the number of modes and gradient regularization strength?\n\n[4] Could the authors explore additional regularization or data augmentation techniques to improve robustness against noise and occlusions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The approach does not require the storage of original point cloud data, thus enhancing privacy.\nBy storing only a shape model (mean shape and modes of variation), the method achieves significant memory savings, reportedly using only 15% and 8.5% of the memory compared to competing methods on ModelNet40 and ScanNet, respectively.\nThe framework uses statistical shape models to capture the intrinsic geometric properties of 3D point clouds, which is particularly effective given the unstructured nature of 3D data.", "weaknesses": "The method’s reliance on storing mean shapes and a few modes of variation per class assumes limited intra-class variation, which may not hold in real-world applications with highly diverse categories. As the number of classes grows, even a few modes per class could lead to increased memory demands and possibly a drop in classification performance if the shape diversity is too restricted.\nThe paper lacks a deep exploration of how key hyperparameters, such as the number of modes of variation, RBF kernel parameters, or the coefficient in Gradient Mode Regularization, affect performance. Given the complexity of continual learning in 3D, tuning these parameters is likely essential to the model’s performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kw1xyrH29n", "reviewer": "Reviewer_9uF5", "summary": "This paper introduces a novel continual learning approach for 3D object classification. The key idea is to retain only the mean shape and a low number of modes for each of the previous classes, so that in the following tasks in the continual learning pipeline, samples from the previous classes can be conveniently generated instead of being saved in memory for replay.\nThis allows reducing the required memory at training time but also enhances privacy since there is no need to store the original old 3D shapes.\nFurthermore, the authors propose a new regularization technique to steer the network parameters toward a better optimal solution and avoid catastrophic forgetting. The authors asses their contributions on three benchmarks based on the popular ModelNet40, ShapeNet, and ScanNet datasets, achieving state-of-the-art performance.", "questions": "I encourage the authors to work on the listed limitations, as they can me simply solved with some additional experiments/ablations and some writing.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. I consider the proposed method easy and, at the same time, highly effective. In my view, this is the main strength of the proposed approach.\n2. The proposed approach shows great performance improvement over previous works.", "weaknesses": "1)The main weakness I can think of in this paper is on the presentation/writing side. \nFirst of all, the listed contributions on page 2 are not really contributions, at least not all of them. For example, Geometry-Awareness and Backbone Independence are more properties than real contributions. I believe the authors should spend a consistent effort in rewriting this section and clearly state their two contributions, which are the memory-efficient and privacy-preserving shape generation and the Gradient Mode Regularization.\nSimilar considerations can be done about the related works. In my opinion, in some sections such as Sec. 2.1, 2.2, 2.4, it is not clear where the paper stands compared to previous works. Please make an effort to emphasize why these papers are related and why they differ. \n\n2)A second problem, but still related to the previous one, is the lack of visualizations that can greatly improve the understanding of the paper. For example, why not show the generated samples while varying some parameters such as alpha and sigma? I would like to see these shapes to also have a qualitative idea of the proposed approach.\n\n3)Lack of some details/ablations. How are the samples for each class selected? Can the authors provide some ablations regarding this aspect? Also, the contribution of the Gradient Mode Regularization seems to be marginal. It is okay as the paper is not entirely focused on this contribution; however, it would be important to show that it also works in other datasets and not only for ModelNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)