ICML 2025

Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device
Learning
Le-Trung Nguyen 1 A¨el Qu´elennec 1 Van-Tam Nguyen 1 Enzo Tartaglione 1
Abstract
On-device learning has emerged as a promising
direction for AI development, particularly be-
cause of its potential to reduce latency issues
and mitigate privacy risks associated with device-
server communication, while improving energy
efficiency. Despite these advantages, significant
memory and computational constraints still repre-
sent major challenges for its deployment. Draw-
ing on previous studies on low-rank decompo-
sition methods that address activation memory
bottlenecks in backpropagation, we propose a
novel shortcut approach as an alternative. Our
analysis and experiments demonstrate that our
method can reduce activation memory usage,
even up to 120.09× compared to vanilla training,
while also reducing overall training FLOPs up to
1.86× when evaluated on traditional benchmarks.
The code is available at https://github.com/Le-
TrungNguyen/ICML2025-ASI.git.
1. Introduction
Recent significant advances in deep learning have allowed
artificial intelligence to penetrate deeply into human activi-
ties, enhancing the quality of life for users more than ever
before. Deep learning has found applications in a wide
variety of fields, such as healthcare through tumor image de-
tection (Hu et al., 2018) or finance through quantitative trad-
ing (Ozbayoglu et al., 2020). Most notable are applications
of large language models (LLMs), such as GPT (Radford
et al., 2018) or Gemini (Gemini Team et al., 2023).
In that regard, one of the current prominent research
branches is on-device learning, where models are fine-tuned
directly on edge devices, eliminating the need for commu-
1LTCI,
T´el´ecom
Paris,
Institut
Polytechnique
de
Paris,
France.
Correspondence
to:
Le-Trung
Nguyen
<le-trung.nguyen@telecom-paris.fr>,
Enzo
Tartaglione
<enzo.tartaglione@telecom-paris.fr>.
Proceedings of the 42 nd International Conferen...