ICLR 2025

Published as a conference paper at ICLR 2025
PRIVACY-AWARE LIFELONG LEARNING
Ozan ¨Ozdenizci1, Elmar Rueckert1, Robert Legenstein2
1 Chair of Cyber-Physical-Systems, Montanuniversit¨at Leoben, Austria
2 Institute of Machine Learning and Neural Computation, Graz University of Technology, Austria
{ozan.oezdenizci,elmar.rueckert}@unileoben.ac.at
robert.legenstein@tugraz.at
ABSTRACT
Lifelong learning algorithms enable models to incrementally acquire new knowl-
edge without forgetting previously learned information. Contrarily, the field of
machine unlearning focuses on explicitly forgetting certain previous knowledge
from pretrained models when requested, in order to comply with data privacy reg-
ulations on the right-to-be-forgotten. Enabling efficient lifelong learning with the
capability to selectively unlearn sensitive information from models presents a crit-
ical and largely unaddressed challenge with contradicting objectives. We address
this problem from the perspective of simultaneously preventing catastrophic for-
getting and allowing forward knowledge transfer during task-incremental learn-
ing, while ensuring exact task unlearning and minimizing memory requirements,
based on a single neural network model to be adapted. Our proposed solution,
privacy-aware lifelong learning (PALL), involves optimization of task-specific
sparse subnetworks with parameter sharing within a single architecture. We ad-
ditionally utilize an episodic memory rehearsal mechanism to facilitate exact un-
learning without performance degradations. We empirically demonstrate the scal-
ability of PALL across various architectures in image classification, and provide
a state-of-the-art solution that uniquely integrates lifelong learning and privacy-
aware unlearning mechanisms for responsible AI applications.
1
INTRODUCTION
Lifelong learning algorithms enhance the ability of machine learning models to incrementally ac-
quire new skills or integrate new knowledge over time from sequentially o...