Accelerated Training on Low-Power Edge Devices
Authors: Mohamed Aboelenien Ahmed
Kilian Pfeiffer
Osama Abboud
Ramin Khalili
Heba Khdr
Joerg Henkel
Venue: Accepted by TMLR ()

Abstract: Training on edge devices poses several challenges as these devices are generally resource-constrained, especially in terms of power.  
State-of-the-art techniques at the device level reduce the GPU frequency to enforce power constraints, leading to a significant increase in training time. To accelerate training, we propose to jointly adjust the system and application parameters (in our case, the GPU frequency and the batch size of the training task) while adhering to the power constraints on devices. We introduce a novel cross-layer methodology that combines predictions of batch size efficiency and device profiling to achieve the desired optimization. Our evaluation on real hardware shows that our method outperforms the current baselines that depend on state of the art techniques, reducing the training time by up to $2.3\times$ with results very close to optimal. Our measurements also indicate a substantial reduction in the overall energy used for the training process. These gains are achieved without reduction in the performance of the trained model.

Published in Transactions on Machine Learning Research (10/2025)
Accelerated Training on Low-Power Edge Devices
Mohamed Aboclenien Ahmed
mohameed.ahmedee @kit. edu
Karlsruhe Institute of Technology
Kilian Pfeiffer
kilian.pfeiffer edu
Karlsruhe Institute of Technology
Osama Abboud Huawei Research Center Munich
Ramin Khalili
numin..boollieehuwwee.ccm
Huowei Research Center Munich
Heba Khdr Karlsruhe Institute of Technology
hebo.kldr Okit. eduu
Jörg Henkel Karlsruhe Institute of Technology
henkel edu
Reviewed OpenReview: https: httpe://openrevieeu openreview.
net/ forum? cGy041,BEm
Abstract
Training edge devices poses several challenges these devices generally resource- constrained, especially in terms of power. Stale-of-the-arl techniques at the device level reduce the GPU frequency to enforce power constraints, leading to significant increase in training time. To accelerate training, we propose to jointly adjust the system and application parameters (in our case, the GPU frequency and the batch size of the training task) while adhering to the power constraints on devices. We introduce a novel cross-layer methodology that combines predictions of batch size efficiency and device profiling achieve the desired optimization. Our evaluation on real hardware shows that our method oulperforms the current baselines that depend on state of the art techniques, reducing the training time by up to 2.3x with results very close to optimal. Our measurements also indicate substantial reduction in the overall energy used for the training process. These gains are achieved without reduction in the performance of the trained model.
1 Introduction
Recent trends are moving toward the deployment of deep learning models on edge devices due to the need for real-time mecksinn-making, reduced communication cost, and privacy compared to offloading computation cloud-based servers [Wagg et al. (2018); ; He et. al. (2018); Chen & Ran (2019); Ganesh et al. (2022) Low-lateney inference is essential in applications such as autonomous vehicles, robotics, surveilancee, and smart agriculture. To meet. these requirements, edge devices are equipped with compact GPUs optimized for computer vision and deep learning inference tasks Maghazeh et al. (2013); |Barba-Guaman et al. (2020): Saddik et al. (2021) ;; Fernández-Sanjurjo et al. (2021) Zhang et al. (2018): Ganesh et al. (2022). Such devices are often power constrained, running on batteries with limited power outputs.
Many applications on edge devices involve sensitive or private information that cannot be shared due to regulatory or ethical concerns, making challenging maintain accurate models without access to this data. Training on the edge offers adaptability to the data on these devices without transmitting it, reducing exposure risk. Transfer learning is thereby applied for on-device training, where model is pre-trained with

---

Published in Transactions on Machine Learning Research (10/2025)
Power Limit 5 W
(128, 307 MH2)
(64, 460 MHz)
Power [W]
Time (T+][s]
1,000
6
500
6
921
921
768
768
4
GPU frequency
614
GPU frequency [Ml1a]
614
16
16
460
Batch size
64
307
[MHla]
Batch size
32
64
307
128
153
128
153
(a) Power
(b) Time
Figure 1: Peak power and time for single training pass over a set of samples across different. batch size and GPU frequency combinations. The gray plane represents the power limit in the left figure, and the black dots in both figures are the feasible combinations that can be utilized under that constraint. The black circle represents the operating point with maximum feasible frequencies for the batch sizes of 128, which will be selected by the stato-of-the-art techniques. The green circle represents an operating point at batch size 64, that could be selected when the frequency and the batch size are jointly selected to accelerate training under the power constraint. Selecting this operating point accelerates training by 31.9% (see Time curves).
large (public) dataset and finc-tuned on-device the target domain Cai et al. 2020 Chiang al. (0023) Lin et al. (0022 Yang et al. (2023). While this offers less training time compared to training from scratch, transfer learning is still computationally demanding task utilizing higher power compared to inference.
To accelerate training, usually large batch sizes for mini-batch gradient descent are used. However, increasing the batch size increases the power consumption which is limited at edge devices. To enforce the power constraint, the circuit-level control will select the frequency level that satisfies this constraint considering the worst-case computation scenario. Intuitively, this will decelerate training: in the state of the art, the system-level and application-level (training) parameters are set independently from each other, and this is sub-optimal will be demonstrated in the following example.
Figure [ shows the power and time required to train a ResNet18 He et al. 2016) model on Nvidia Jetson Nano with 4096 of CIFAR10's training samples, while using different combinations of the GPU's frequency and batch size. Following the state the art, and for a power constraint of W, the system would select the GPU's frequency f = 307 MHz and the batch size b = 12 When we look the training time, this selection is sub-optimal. In particular, we could choose smaller batch size (b = 64), allowing the GPU to operate at higher frequency (f 460 MHz), and thus reducing the training time by 31.9%. This shows that to accelerate the truining on low-power edge devices, both the GPU frequency (a system porameter) and the batch size (an application perameter) should be jointly selected.
Motivated by this observation, we introduce, for the first time, a cross-layer approach that simultaneously optimizes both the GPU frequency and the batch size used for training, aiming to accelerate the process while adhering to device power restrictions. Note that the example in Fig, iillssrates the impact of batch size on single training pass. However, modifying the batch size in training (in contrast to the inference process) influences the number of iterations needed to reach the targeted accuracy, and consequently the overall training lime. This presents additional dimension to the problem, which we will consider in our approach as detailed in Section 3 Furthermore, our cross-layer optimization is capable of leveraging operating points within the design space that previous works have not considered. This approach facilitates a marked decrease in the necessary training time to accuracy.
We select. the maximum batch size which is allowed by the memory the device.
2

---

Published in Transactions on Machine Learning Research (10/2025)
Our solution incorporates offine profiling of Neural Network (NN) training the device, where we assess power and lime demands for different. combinations of batch sizes and GPU frequencies. Furthermore, we estimate how batch size correlates to the number of iterations required to attain a desired accuracy by training on a proxy dataset on a server. By merging these insights, we identify the combination that minimizes overall training time on the device while respecting its specified power constraints.
In summary, we make the following novel contributions:
We motivate and demonstrate the importance of the joint optimization of GPU frequency and batch size to accelerate on-device training under power constraint, shown across representative models (CNNs and Transformers) and tasks (image classification and next character prodiction).
We introduce a new cross-layer approach that performs the discussed joint optimization, taking into account the effect of batch size the overall training time, aiming accelerate on-device training while complying with the device's power limitations.
Our evaluation on real hardware under different power constraints for image classification lask on SVHN Netzer et al. (2011) and CLNIC Darlow al. (2018) datasets, using models ResNet18, MobileNet 22 Sandler al. (2018), and Efficient VIT Liu al. 2023 and next character prediction task using a transformer network shows significant improvement in the training speed by an average gain of 1.5x and peak of 2.3x compared to baselines. We observe reduction in the total energy used for the training at the device.
We provide a comprehensive sensitivity analysis that demonstrates the robustness of our solution towards the proxy dataset selection five image classification datasets and three next character prediction datasets, validating the practicality of our approach and ensuring its effectiveness.
2 Background and Related Work
This section discusses the control parameters (i.e., GPU frequency and batch size) used in this work and the state-of-the-art techniques. We do not consider training on the CPU for the GPU-equipped devices, as the GPU is an order magnitude faster (and energy efficient) while consuming nearly the same average power (see Appendix A.1 for some related discussion).
2.1 Voltage Frequency Scaling (VFS)
VFS a system-level technique used for power management in processors Cochran et al. (2011); ; Lee et al. (011). It adjusts the voltage and frequency at runtime, providing a trade-off between performance and energy efficiency Le Sueur & Heiser (2010); Guerreiro et al. (2019). Reducing each of the processor's fre- quency f and voltage V values can result in a cubic reduction in the dynamic power, i.e., Paynamic & fV², Within a device, the chip manufacturer provides discrete set of frequencies to operate with, where fi, E 21, max]. The voltage is function of the operating frequency f, and will be updated automati- cally for any given frequency. To enforce power constraints on processors, the industry standard technique is have an upper bound ʃ (and V) for the device to use under a power limit. VFS also applied to the CPU's frequency, but this beyond the scope of our work as we train on the GPUs.
VFS has been studied in Nabavinejad et al. (2019) Liu & Karanth 2021 p for improving performance and energy efficiency for inference only. Tang et al. (2019) conducted study the impact of GPU VFS performance and energy for both training and inference. However, the study mainly focused on the computational perspective without addressing accuracy and training speed till convergence. In contrast, we show how changing the frequency and batch size due to power limits, accompanied by the difference in iterations reach accuracy, can lead to different optimal configurations training.
2The selected models and tasks align with existing literature for training on edge devices Ponzina 2023 Chol & Sobelman 2022): Jia al. (2024): PPeiffer et (2023), and Jetson Nano and TX2NX represent typical edge GPU platforms.
3

---

Published in Transactions on Machine Learning Research (10/2025)
2.2
Batch Size
One of the state-of-the-art techniques for training NNs the minibatch gradient descent, where gradients are computed using the samples drawn in mini-batch to approximate the overall gradient, cnabling iterative updates of the model parameters. The latency for processing a batch is influenced by the GPU's ability to parallelize computations and its memory bandwidth. Increasing the batch size generally decreases the latency of processing data samples by parallelizing more computations on multiple processing units. Nevertheloss, this increases the number of computational operations on the GPU, and hence the power consumption.
The impact of batch size on accuracy and convergence speed is explored from multiple aspects. /oyal et al. (2017): Krizhevsky| 2014);; Zhang et al.] (2019) studied the interdopendencies between batch size and other hyperparameters such as learning rate, weight decay, and optimizers. [Smith et al. (2018) proposed to increase the batch size during training rather than decreasing the learning rate to leverage the regularization effect employed by larger batch sizes. McCandlish et al. (2018) proposed a gradient noise scale metric to predict the most efficient batch size that maximizes the thronghput in the next training step. This approach is tailored for distributed learning, where a batch of data is split across multiple devices. These aforementioned works have not considered the power efficiency of batching on the device. In contrast, the works in Nabavinejal et al. 2001 2022) consider the impact of the batch size on power consumption of inference operations. In particular, a binary search approach is employed in Nabavinejad et al. (2022) to find an appropriate batch size that accelerates inference and then the GPU frequency is adjusted accordingly to satisfy power constraints on GPU servers. Most importantly, this approuch only considers inference where the statistical impact of the butch size does not erist. You et al. (2023) aimed at optimizing the energy efficiency of periodic training jobs (continuously re-executed for incoming data flow) on powerful GPU clusters. To achieve their optimization (which is diffcrent from the one we target in this paper), they propose to adjust the batch size, set power limit on the server, and depend on circuit-level control to select the frequency that satisfies that power limit. In particular, they obtain Pareto-optimal combinations of batch sizes and power limits that optimize for energy and performance through profiling the whole training job on the server, since this job will be repeated for new data flows. However, this solution can be applied on powerful GPU clusters and not on edge devices with limited resources.
In summary, none of the state of the art has addressed the joint selection of the batch size and GPU frequency to accelerate training at edge devices under power constroints.
3 Problem Statement
We consider the following scenario: for a specific training task, an edge device requests a pre-trained NN model M with its weights 0 from a server in order to fine-tune it on local data D till reaching a given accuracy threshold. Importantly, the edge device has a power constraint Pimxx, which should not be exceeded during the training process. Our goal in this paper is to ménômize the training (fine-tuning) time at edge devices under their given power constraints.
We introduce T, as the training time required to apply training using a fixed number of samples As shown in Fig, [ the joint selection of b and f will help reduce T, under a power constraint. However, the ultimate optimization goal is to minimize the total training time required to reach a target accuracy, which we label TTaco- A set of parameters, ie., frequency and batch size (f. b), that are optimal for training a fixed number of samples (Tx) might not be necessarily optimal for the training to accuracy (TTace).
We display in Fig. 2 the training time to reach an accuracy of 78% ( (Tacc)) for ResNet18 using two batch sizes of bj = 8 and b2 = 32 under three different power constraints (i.e., P1 = 4.5 W, P2 = W, and P3 = 7 WW). We notice that for the three power constraints, selecting by allows to utilize a higher frequency than by. For each batch size, we select the highest frequency that satisfies the power constraint, and measure T, and TTace- We observe that using b2 (the higher batch size) always leads to a lower T.. However, selecting the same batch size over by leads to a longer TTacc for P1 and P2, and shorter TTace for Ps.
This shows the complexity of the targeted problem. In particular, TTace does not only depend on T,, but it also depends on the number of times of processing to reach target accuracy (Ns..)- ) In this example, Na
4

---

Published in Transactions on Machine Learning Research (10/2025)
TT (b)
TT (ba)
Ts
8
32
3,000
P3
P
200
The
2,000
Tels
153
307
460
614
768
921
GPU Frequency [MHz]
1,000
Pr
P3
P
0
0
153
307
460
614
768
921
Pi
P2
P3
GPU Frequency [MHz]
Feasible Operating Points(b, f) for the power constraints
The training time fixed number of samples T3 and the total training time Tiaee to reach accuracy threshold of 78% using two batch sizes and 32, while considering the maximum feasible GPU frequencies under three power constraints; P1 = 4.5 W, P2 5 and P3 7W. We observe that for P. and P2 selecting 8 will lead to lower TTace while for P3 selecting b = 32 is better. This is in contrast with our observation for Tg, where selecting b 32 is the best option in all cases.
Figure 2:
for by equal 15 while Ne..e for bj equal 10. These values and the effect of power constraint on the feasible frequency highly influence the optimal batch size to minimize TT In summary, there is clear indication how select the optimal operating points (f,b) to achieve the target goal. We formulate our optimization problem as follows:
min bEB.JEF subject P(b, M) Pmax
Tacc f.
(1)
where B is the set of feasible batch sizes, F is the set of available GPU's frequencies, and P(b, M) is the required power to training M using b and f. We rewrite TTace the multiplication of T, and Nonce thus have:
TT (b, f,M, (. x Nowco M,,,
(2)
is selected, s.I. bmax S ID], where bmax the largest batch size that can fit inlo the memory of the devices. This detached formulation enables our proposed optimization method, presented in Section 4 In particular, the first factor Т, does not depend the training data D, nor on the accuracy threshold. The second factor Nawce is independent of the GPU frequency of the device.
4 Awwer-Aware Training
We propose optimization approach that co-selects b and to minimize TTace edge device under Pmax- Following the problem split that we propose in Equation 2 our solution consists of two main parts. The first one considers device specifics, and involves measuring time and power for training a given model M, i... (b, f. .)) and P(b, M) for every b E and f f The second part is responsible for estimating the efficiency of batch sizes, i.e., their impact on Nemce- This part does not depend on device characteristics and is computationally expensive, therefore we consider for pre-training and sending of the model.
be performed on the server that is responsible
The workflow for our approach is follows: The device first sends request to the server for pre-trained neural network model (M) with specific architecture and input size. The server responds with the requested model and its pre-trained weights. Our proposed power-aware training then begins. The device profiles the model in terms of time and power as will be described in Section 4.1 while the server estimates N for various batch sizes, as will be discussed in Soction 4.2. and sends these estimations the device. Basod on the profiling and estimations, the device selects the best combination of batch size and frequency minimize TT An overview presented in Figure]
5

---

Published in Transactions on Machine Learning Research (10/2025)
Edge Device
GPU Server
Profiler
Train for few iterations
Store in lookup tables
and record Ptb. D. T2(b,
LUT poweell []- P(b, 0 LUT umd f- T,bb
Proxy Dataset Trainer Train with different batch sizes proxy dataset and approximate NB(B5)
Configuration Selection (Section f", b
P
h
Nomn. N,
Set GPU freq
Train
Dy by PgP6
Figure Overview of our proposed cross-layer approach that accelerate training under power constraint though the joint selection of batch size b and GPU frequency f.
4.1 Profiling
The power and time for training device also depend on the NN's architecture M, in addition to the frequency and batch size. Besides, changing the input shape, such the image size in vision tasks, can lead to different time and power requirements even when using the same NN architecture. Furthermore, the time and power are independent of the actual training data. Therefore, profiling T, (b, J, M) and P(b, ʃ, M) given the task's imput shape can be performed before full data acquisition, as long as the input dimensions of the data samples are known.
Processing few mini-batches is adequate to obtain accurate profiling- For each batch size and frequency combination, we set the GPU frequency to f and process few mini-batches of size b. The power sensor values are monitored to extract the peak power. Furthermore, the processing time is recorded, and then the average processing time for a mini-batee batch is calculated. This average processing time is then scaled to the processing time for s samples, denoted as Tg. T, (b, f, M) and P(b, f. M) are then stored in two lookup (ables, denoted as LUT Time and LUT-M Power
The proposed profiling strategy considers hardware and NN structures. Also, the time and power values are not affected by the network weight updates; thus, profiling is applicable before receiving pro-trained weights if M is available at the device and is needed to be perdomed only once. In future and more practical setting, we could assume such profiling to be provided by the device manufacturer and for a pre-defined set of tasks and models. This can be further supported by periodic validation of current values with those recorded in the LUTS. a deviation is detected, specifically relative performance change betweon operating points (e.g., due memory or cache contention), a re-profiling procedure is initiated accommodate the change, requiring only a few batches per operating point, that is much cheaper than full training. This process can be further optimized by interpolating the deviations across operating points, thereby enabling efficient updates to the LUTss with minimal computational overhead.
Furthermore, if Pmaz is known at the device, the profiling can be performed more efficiently as follows. Given that B and F are sorted, and power consumption increases with both and f, we begin by profiling the largest with the minimun f, incrementing f until the maximum feasible value under Pmaz is reached. Next, we move the second largest b, starting from the highest feasible frequency found for the previous batch size, and repeat the process until the smallest batch size is profiled. This profiling is equivalent most to 1.9% and 6% of the training times performed on ResNet18 and MobileNet in Section 5.2

---

Published in Transactions on Machine Learning Research (10/2025)
4.2 Estimation of Nuke for Batch Sizes
(,,,M,D) As discussed in Section 3 batch sizes have different efficiency in terms of Nemc (b,, M, D). Estimat-
ing Neace (b;, M, D) for every b, E B depends also on the training data that is only available at the device and M's pre-trained weights. To solve Equation (]) the exact number of samples processed to reach target accuracy for each batch size can be replaced with the relative ratio between batch size rb, (i.e., normalized to the maximum No Nance) as follows:
Na (be, M, D) Tb maxN (6, D) BEB
(3)
With this simplification, we adjust our focus to estimate the relation vector between batch sizes such that r == (ro,, Pby......... Tomae)) where rb. E (0. However, this is still a complex task to solve given the non-linear training dynamics of deep learning; especially as the convergence speed of every batch size changes across training, making impossible to estimate with few probes of multiple batch sizes given the training state. Obviously, training till convergence for multiple batch sizes is computatiooaall-expenss be conducted on the device. These all make on-device estimation of r inaccurate, if not infeasible.
lask and cannot.
We thus propose to estimate r on powerful GPU server. Particularly, M is trained on the server with multiple batch sizes until convergence (reaching the target accuracy) using a proxy dataset Ds since the server does not have access to D. Each batch size run starts from the same pre- are-trained weights and yields its corresponding Nauce on Ds. This systematic exploration allows us to comprehensively assess the long- horizon impact of the batch size on the model's convergence while leveraging the computational capabilities of the server, along with the datasets available on it and augmentation techniques. Since M, which would be used on the edge device, is already designed for specific task type and pre-trained a public dataset, a proxy dataset Ds should share the same task type (e.g., image object classification) and similar input shapes. Training network M on Ds, despite having different objectives, allows us to estimate the relationship between batch sizes and their relative examples to accuracy on D. Thus, we can finally have a mapping such that ID, R rn- By estimating the batch size relation vector r on the server, any edge device aiming to train M to utilize this vector. In Section 5.2] we provide evaluation for two different devices, namely Nvidia Jetson Nano and Nvidia Jetson TX2NX, utilizing r.
4.3 Batch Size and Frequency Selection
The device profiling and estimation of batch size efficiently are performed in an offline manner and at the design time. In contrast, the configuration selection is performed at runtime, described below. Given power constraint Pmax; and the power measurements stored for training a specific M in LUTA Power, we construct a set of feasible combinations C' consisting of every feasible batch size with its corresponding highest (and fastest) frequency satisfying Punax as follows:
C - (i. E (1,...,, [B]],
(4)
ji + max e [L...., [FI,, LUTA
Fower (1, < P Pmax)
The processing time for the feasible combinations is then extracted from LUT imate training time is computed by multiplying the time for every by (and frequency) by the corresponding ri element from the relation vector r (i.e., estimated at design time). The configuration ft) that minimizes the approximate total training time is selected. Finally, we set the GPU frequency to f and then start the training using batch size b². We provide the configuration selection in Algorithm | The selection part is Onn²) in the worse case scenario (O(IB) x [F])). Since B and F are sorted, and for each b, the power/time over F are also sorted, the selection reduces to On log n) via binary search and is negligible to training time.
Time- Following this, an approx-
5 Results
In this section, we evaluate the training time and energy consumption for finetuning vision and text tasks on Nvidia Jetson Nano and TX2NX, Additionally, we assess the effectiveness of our approach by conducting sensitivity analysis the selection of the proxy dataset.
7

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Batch size and GPU frequency selection
Require:
Power Limit maxx Neural Network M, List of feasible batch sizes B, List of GPU frequencies F, Dataset at server D
1: Server:
2: r + Proxy(B, Ds)
See section (Section 4.2)
Device:
Let LUT Time
6: Let LUTPOWE
E [
Power
7: for E 0, 1,
[B] do
8:
b b-BBi B
9:
for j j 0....... [F do
10:
f [
11:
time, power Profile(b, f. M)
On Device (Section 4.1
12:
LUT M
Timel. [i.]] t time
Ai LUT Power [. < power
13:
14: // On-device Configuration Selection:
15: TTace = LUTT X r
Element- multiplication D Based on Equation (4)
16: C+ GETCOMBIMTIONS(B... LUT.M Power: Pmaax)
17: TTE t TTaool
18: (b idx,f idx) argmin(TTR.
19: b" f" + Blb Flf idx]
Selected batch size and GPU frequency
5.1 Experiment Setup
Datasets and models: We evaluate our approach for the image object classification and next-word pre- diction tasks. For image classification, model is pre-trained with the full CIFAR100 Krizhevsky et al. (2009) and to be trained on subsets (i.c., quarter) of SVHN and CINIC datasots on the devico. We use a subset of CIFARIO Krizhevsky el al. (2009) as the proxy dalaset on the server. For all datasets, the imput images are of size 3 32 x 32. Each dataset (subset) is divided into training and testing sets with an 80/20 split, with target accuracy evaluated
the test split. We evaluate ResNet18, MobileNet and Efficient ViT--(I) models, trained with Adam optimizer Kingma & Ba (2015). A summary of the experimental settings is provided in Table [ For the next. character prediction task, we evaluate 6 layer transformers with 6 at- tention heads per attention block, 256 embedding dimensions, and sequence length of 64. We use AdamW Loshchilov & Hutter (2017) as an optimizer. We pre-train the model on WikiText-2 dataset Mority et al. (2016), utilize liny sha shakespeare Karpathy (2015) as a proxy dataset, and Lrain it on some Jane Austin and Charles dickens novel. We use 90% of a dataset for training and the rest for testing. For all datasets, we fix the vocabulary include only words with English letters, digits, punctuation, spaces, and new lines. We set the target character level accuracy for Austin and Dickens at 62% and 61%, respectively.
Batch size and lcarning rate: The choice appropriate learning rate and batch size are often inter- twined, as they impact each other's effectiveness. Larger batch sizes provide more stable gradient est estimales, potentially permitting the use of higher learning rates. Therefore, to preserve the performance of deep mod- cls with different batch sizes, we apply learning rate scaling (i.e., square root scaling Krizhevsky (2014) for Adain and A For ResNet18, the batch sizes ranged from 4 to 128, consisting excl exclusively of powers of two. The initial learning rate of 5 x 10-4 is used for the largest batch size of 128 (with learning rates scaled for other batch sizes). The same setup was also applied to Mobile and Efficient VII; however, the batch size 128 was omitted due to memory constraints. For transformers, we similarly consider batch sizes of to 128 with learning rate of 10-3 for the batch size of 128.
We used the for works of Jane Austin's from nitk package Bird al. 2009 downloaded Dickens works from project Gutenburg. More details provided Appendix

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Experimental setup for image classification datasets and models used.
Dataset Dataset size
Target Acc.
ResNet18
MobileNetV2
Efficient ViT
Cifar10
12500
87%
84.5%
84% 91.5%
SVHN
18315
92%
90%
CINIC
22500
76%
74%
72%
Hardware and power limits: We evaluate our method on Nvidia Jetson Nano with GB memory on three scenarios. In the first and second scenarios, the power limits at the device are set to P = 4.5 W and P max = 7 W. In the third, the device operates without any power limits, denoted as N/A. To show that our solution and our results are not device specific, we also provide evaluation on another device (i.e., Nvidia Jetson TX2NX). We use PyTorch 1.10 Paszke et al. (2019) for Jetson Nano and TX2NX. For pretraining and proxy datasets' training we use NVIDIA A6000 GPU with Pytorch 2.1.
Comparison basclines: To highlight the impact of joint selection of GPU frequeney and batch size, we compare our approach to the following baselines that depend state of the art techniques:
Baseline 1: The state of practice is to use the largest b that can fit into memory Goyal et al. (2017); Camelo Cretu (2023). where the latter use edge GPUs. For the three power limits, we use 307 MHz, 614 MHZ, and 921 MHz as upper-bound operating GPU frequencies for the device.
These are determined based on profiling training of different models and selecting the frequencies that assure power limit is satisfied irrespective of what model or is used for the training.
Baseline 2: We select the value of b that minimizes Ne on the proxy dataset, but we use the same GPU frequencies in Baseline 1, no joint optimization is applied.
Fastest configuration: This baseline serves as an upper bound where optimal / and b are jointly selected. Furthermore, this baseline validates the importance of jointly selecting f and 6, as evidenced by its performance gap compared to other baselines under power constraints. Since the optimal configuration is not known prior to training, model on the given dataset with all batch sizes to get Na substitute in Eq. 2 and finally select the best b and f configuration on the device.
determine il by exhaustively training the given
We repeat every experiment with different five seeds for image classification tasks and three seeds for next character prediction and record the mean and standard deviation.
5.2 Training Time Evaluation
Table 2 and Table 3 report the training time comparison of our approach and the three baselines on Nvidia provides additional comparison for image classification task on Nvidia Jetson TX2NX.
Jetson Nano. Table 4
Firstly, we start with the evaluation of image classification tasks reported in Table For ResNet18 training, our method consistently outperforms baseline 1, achieving 1.5 2.3x reduction in training time. It also outperforms bascline with 1.4 1.7x less training time. These two baselines are only able to explore a small subset of the design space, potentially missing the optimal configuration. Finally, compared to the fastest configuration, our method selects the same configurations in most of the evaluations, while in the case of a different selection, minor performance decline (of 3%) is observed. Training of MobileNet V2 has three distinct characteristics: lower peak power, better parallclization for larger batch sizes, and different r. Our method adapts to that by selecting larger batch sizes (and frequencies) than those selected for ResNet18. Compared with baseline 1, our method performs better with minor differences in training time since both select large batch sizes and maximum utilizing the power. In low-power, the gain from our approach increases as the selected batch size enables higher frequencies, reducing training time by up to 1.4x. Baseline 2 misses the opportunity to use higher frequencies, especially MobileNet V2 consumes less power.

---

Published in Transactions on Machine Learning Research (10/2025)
Table 2: Training time comparison with baselines and upper bound over three different power limits (i.e., Pmaae: P maxx' and N/A) on Jetson Nano, showing peak gain of 2.3x. Recorded times are in seconds. CIFARIO is used as proxy dataset in our proposed approach. (Evaluation time is excluded)
Model
Method
SVHN
CINIC
Pal
på
N/A
pl max
p2
N/A
ResNet18
Buseline
34347
8204-12770
6187.11637
18154:14109
1038 1349
7829:11772
ResNet18
Baseline 2
11345=1616
7055=1005
5817# 828
12436+1443
7733
897
6377# 740
ResNet18
Our
8477-1207
4607.L 725
4170. 656
029211078
44541 788
4032 113
ResNet18
Fastest coufiguration
8477=1207
4607 725
41704 656
9030:1199
44541
788
4032 713
MobileNetV2
Baseline
8708-2396
5107+1405
3990++108
8109+1121
5226+ 657
4082+ 513
MobileNet
Baseline
8082= 912
4871 550
3912# 441
$428: 692
5079 417
40802 335
MobileNet
Ours
5940.L 916
3912.1441
3912.1 441
61941 509
40801: 335
4080 335
MobileNetV2
Fastest coufiguration
5040.1 916
3912.1 441
3912.1 441
6194.1 509
4080 335
4080.L 335
VifiiientViT
Baseline
13033-1233
7839+741
6369+6002
8100+564
4800++339
3900++275
EfficieutVil VIT
Baseline
14539=663
92691-422
78431337
92231584
58801372
49751315
EilicientViT
Ours
11036:5003
6925-1655
6369++602
7001+1444
4240=299
3900+-275
VificientViT
Fastest configuration
11036=503
6925=655
6369:602
7001#444
4240::299
3900-2275
Table 3: Training time comparison with baselines and upper bound for different power limits (i.c., P P
p2 max: and N/A). Recorded times are seconds. Tiny Shakespeare is used as proxy dataset our approach.
Model
Method
Austen
Dickens
P1 max
p2 max
N/A
Pl
max
p2
N/A
Transformer
Baseline 1
81361152
4607+299
3556:129
1234714399
6991=2249
5397=192
Transformer Baseline 2
9425+1131
6081+85
5193+172
14278-+2228
12911++532
7867=125
Transformer Ours
7232:1011
39821126
3556:1229
10956:175
60431215
5397:192
Transformer Fastest configuration
7232:1101
3982--256
3556:1229
10956:+175
6013-2215
5397=192
Table 4: Training time comparison with baselines and upper bound over three different power limits (i.e.,
PL maxx p2 max: and N/A) on Nvidia Jetson TX2NX. Recorded times are in seconds. CIFARIO is used proxy dataset in our proposed approach. (Evaluation time is excluded)
Model
Method
SVHN
CINIC
Pl max
p2 max
N/A
P1 max
N/A
max
ResNel18
Baseline
7343119443
3944-1043 1043
3021-1799
929211210
499111129
3823.1865
ResNei18
Baseline
733711943
4771=679
4086-1582
8042-1933
52301
44791520
ResNet18
Ours
4283+708
2648+417
2431+382
5463+889
2560+453
2350+416
ResNet18
Fastest configuration
4283+708
2648-411
2431+-382
5046++893
2510+ 625
2294+571
Ultimately, our method selects the fastest configurations when training MobileNet V2 in all power-limits scenarios. Similarly, our method results in less training times compared to baselines 1 and 2 under p1 and p2 when finetuning Efficient VIT, while selecting the fastest configuration in all scenarios.
Next, we examine the next character prediction training. As shown Table 3 our method improves the training time by 1.15x and 2.14x compared to bascline and 2, while choosing the same configurations the fastest configuration.
In Table compare our solution with the baselines for 1e training Nvidia Jetson TX2NX, where the device receives for ResNet18 training from the server. For training on SVHN, our method outperforms both the baselines 1 and 2, reducing the training time by peak of 1.7x and 1.8x, respectively. Furthermore, it selects the same configuration as the optimal one. Similarly, when training on the CINIC dataset, baselines and require up to 1.94x and 2.04x more training time compared to ours. Our method did not select the exact optimal configuration in this scenario; however, selects near-optimal configuration that results in only 2% 8.2% more training time compared to optimal.
10

---

Published in Transactions on Machine Learning Research (10/2025)
Baseline
Baseline
B
Ours
Fastest Configuration
-10*
ResNet18/SVHN
-10*
ResNet18/CIn CINIC
Energy [ s)
Energy [W ss]
4
2
P
P
N/A
PI
N/A
max
max
max
MobileNetV2/SVII
MobbleeeL22/CINIC
-10
-10*
Energy [W s]
Energy [Wa]
3
2
P
max
maxx
N/A
PI
P
N/A
Efficient ViT/SVHN
ViTicientVVT//C /CINCC
-10*
-10
Energy [W s)
Energy [W
2
Pmax
P
N/A
P1
N/A
Transformers/ Austen
Transformers/Diceen
-10
-10
Energy [W s.
Energy [W s)
3
4
2
P1 max
P max
N/A
P1 max
N/A
max
Power Constraint
Power Constraint
Figure 4: Energy consumption comparison between our approach and baselines during training under three power scenario, using ResNet18, MobileNet V2, and Efficient VIT on the SVHN and CINIC datasets, and a transformers network on Austen and Dickens datasets, performed on Nvidia Jetson Nano.
In summary, our method outperforms existing baselines, reducing training time to accuracy across various model architectures, tasks, and hardware, with an average speedup 1.5x to baselines and 2 under power constraints pl Plan and p2 maa.
5.3 Energy Evaluation
We investigate the energy efficiency of our proposed approach for the image object classification and next character prediction tasks Figure
For the image classification task, it is noticeable that method always outperforms baseline since it takes less training time given the power limits. In addition, selections made by baselines 1 and 2 can lead to up to 2x and 1.4x more energy usage compared to our method, respectively. It is important to state that minimizing the training time does not always mean minimizing energy consumption. An example is comparing the energy consumption for p2 on ResNet18 with the no power limit N/A. Despite having lower training time N/A (as shown in Table 2 training under Plax is more energy efficient since the used frequencies for p2 provide a better tradeoff between performance and power. The same applies for Baseline 2 when training MobileNet and Baseline Vificient under p2
max: that uses lower
11

---

Published in Transactions on Machine Learning Research (10/2025)
Target Dataset
Target
Dataset
Target Dataset
Cilarto
SVHN
CINIC FMNIST STL
Clfcc0
SVHN
CINIC
FMNIST
STL
Cilarto
SVHN
CINIC
FMNIST
STL
Ciari0
Clari0
Cifar10
Proxy Dutaset
SVHN
SVHN
9
0
SVHN
g
0
10.1
CINIC
0
0
CINIC
0
0
CINIC
0
0
0
10.1
FMNIST
FMNIST
FMNIST
2.5
2
0.1
p?
N/A
Target Dataset
Target Dataset
Target Dataset
Shakes.
Austen
Dickens
Shakes.
Austen
Dickens
Shakes.
Austen
Dickens
Shakes
Shakes.
0
Shakes.
0
Proxy Dataset
Austen
0
Austen
Austen
Dickens
Dickens
Dickens
P
P2 maaE
N/A
Figure 5: Matrices showing the mean percentage increase in training time of our approach to the fastest configuration for image classification and next-character prediction datasets on Nvidia Jetson Nano across three power constraints. The top matrix corresponds to training MobileNetV2 for image classification tasks, and the bottom to Training a transformer network for nexi-character prediction tasks. Each entry reflect the mean percentage increase in training time when selecting configuration given the proxy-dataset's relation vector (row) instead of the optimal configuration (column) for the target data. A value of 0% denote selected, resulting in time increase. The results indicate that the
cases where the same configuration proposed hot is not sensitive to the selection of proxy dataset, with minimal impact on the training time when the selected configuration differs from the fastest configuration for the target dataset.
frequency with the same batch size of our approach. For the next character prediction task, our approach uses 9% less energy for pl and nearly the same energy consumption for pa Paax while training in less time (the configuration selection for P compared to the baseline In addition, method records 1.25x less training energy for the baseline 2.
5.4 Proxy Dataset Analysis
We conduct a sensitivity analysis for the selection of the proxy dataset for image classification and next character prediction tasks by testing all possible proxy and target dataset combinations. We report the mean percentage increase (averaged over runs; Section 5.1 1 in training time relative to the fastest configuration.
For the image classification task, add two additional datasets namely, Fashion MNIST Xiao et al. (2017) (FMNIST) and STL CCaates et al. (2011). The tasks include object, digit, and fashion classification, with dataset sizes ranging from 5K to 22.5K samples. We provide in Fig. confusion matrices showing the percentage time increase for the choice proxy dataset selection compared to the fastest configuration for different datascts for training MobileNetV2 and the transformer network for next character prediction on Jetson Nano. For MobileNetV2, the results show that our approach selects the correct configurations in most of the cases with only few instances where the optimal configuration is not selected in no power
12

---

Published in Transactions on Machine Learning Research (10/2025)
constraint scenario. The same observation is also applied to ResNet18 as shown in Figure 7 (Appendix A.2 For the next character prediction, the optimal configuration is selected regardless of the selected proxy dataset. To sum up, this analysis shows that our approach is not sensitive to the selection of proxy dataset, demonstrating its practicality.
6 Conclusion
In this work, we propose power-aware training method aimed at accelerating training on cower-constrained GPU devices. Our results show that great savings can be achieved in terms of training time and energy consumption, when carefully and jointly selecting the system and application parameters for training, with- out scarifying the training model quality. The proposed solution applicable to wide range of models (including, but not limited to, CNNS and transformers). Our solution can be employed leverage nearby renewable energy sources by adapting GPU frequency and batch size to the available green energy at edge devices to reduce carbon foolprint. A limitation of this work is that we assume that the power constraint on a device does not change through the training period, which may not hold for long-duration tasks that can take multiple days. For future work, we want to extend our study to the varying power constraints and distributed learning settings where devices with different constraints contribute to the training.
Acknowledgments
This research was supported in part by the EU Horizon project EXIGENCE under Grant Agreement No. 101139120.
References
Luis Barba-Guamaan Guaman, Jose Eugenio Naranjo, and Anthony Ortiz. Deep learning framework for vehicle and pedestrian detection in rural roads on an embedded gpu. Electronics, 9(4):589, 2020.
Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit. O'Reilly Media, Inc.', 2009.
Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. Tinytl: Reduce memory, not parameters for efficient on-device learning. Advances Neural Information Processing Systems, 33:1 11115--12999 2020.
Icaro Camelo and Ana- Maria Cretu. Evaluating compact convolutional neuraal neural networks for object recognition using sensor data on resource-conssrrained devices. Engineering Proceedings, 58(1), 2023. ISSN 2673-4591.
URL httpp://wwwwt mdpi. cotp://33////334//5555/
Jiasi Chen and Xukan Ran. Deep learning with edge computing: review. Proceedings of the IEEE, 107 (8):1655-667, 2019. doi:
29219777
Hung-Yuu Yuch Chiang, Natalia Frumkin, Feng Liang, and Diana Marculescu. Mobiletl: on-device transfer learning with inverted residual blocks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 7166-7174, 2023.
Kyubaik Choi and Gerald E Sobelman. An efficient enn accclerator for low-cost edge systems. ACM Transactions on Embedded Computing Systems (TECS), 21(4): 20, 2022.
Adam Coates, Andrew Ng, and Honglak Lec. An analysis of singlo-layer networks in unsupervisod feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and slatistics. pp. 215-20122 223. JMLR Workshop and Conference Proceedings, 2011.
Ryan Cochran, Can Hankendi, Ayse K. Coskun, and Sherief Reda. Pack & cap: adaptive dvfs and thread packing under power caps. In Proceedings of the 44th Annual IEEE/ACK International Symposiun on Microarchitecture, MICRO-44, pp. 175-185, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450310536.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Luke N Darlow, Elliot Crowley, Antreas Antoniou, and Amos J Storkey. Cinic-10 is not imagenet cifar-10. arXiv preprint 181: 0.00505 2018.
Mauro Bermandez-Sanjurjj Manuel Mucientes, and Victor Manuel Brea. Real-time multiple object visual tracking for embedded gpu systems. IEEF Internet of Things Journal, 8111):117--11888 2021.
Prakhar Ganesh, Yao Chen, Yin Yang, Deming Chen, and Marianne Winslett. Yolo-ret: Towards high accuracy real-time object detection on edge gpus. In Proceedings of the LEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3267-3277, 2022.
Priya Goyal, Piotr Dollár, Ross Girshick, Picter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangging Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint 1rXi;:1766. 02677, 2017.
João Guerreiro, Aleksandar Mlie, Nuno Roma, and Pedro Tomás. Dyls-aware application classification to improve gpgpus energy efficiency. Parullel Computing, 83:03-117, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pottern recognition, pp. 770-778, 2016.
Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Ame: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV), pp. 784-88 784-800, 2018.
Chentao Jia, Ming Hu, Zekai Chen, Yanxin Yang, Xiaofei Xie, Yang Liu, and Mingsong Chen. Adaptivefl: Adaptive heterogeneous federated learning for resource-constrained aiot systems. In Proceedings of the 61st ACM/IEEE Design Automation Conference, pp. 1 6, 2024.
Andrej Karpathy. char-rnn.
chapst 2015.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Alex Krizhevsky. One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv:1404.5907 2014.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Etienne Le Sueur and Gernot Heiser. Dynamic voltage and frequency scaling: The laws of diminishing returns. In Proceedings of the 2010 international conference on Power aware computing and systems, pp. 1-8, 2010.
Jungseob Lee, Vijay Sathisha, Michael Schulte, Katherine Compton, and Nam Sung Kim. Improving throughput of power-conse constrcon gpus using dynamic Eattagefrreuencyy and core scaling, In 2011 In- ternational Conference on Parallel Architectures and Compilation Techniques, pp. 111-120, 2011. doi: 10.1109/PACT.20 22111.1....
Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. On-device training under 256kb memory. Aduances in Neural Information Processing Systems, 95:22941-22954, 2022.
Siqin Liu and Avinash Karanth. Dynamic voltage and frequency scaling to improve energy-efficinnc of hardware accelerators. In 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), pp. 232-241. IEEE, 2021.
Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, and Yixuan Yuan. Efficientvit: Memory efficient vision transformer with cascaded group attention. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 14400-14400, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. Xie preprint 2017.
14

---

Published in Transactions on Machine Learning Research (10/2025)
Arian Maghazeh, Unmesh D. Bordoloi, Petru Eles, and Zebo Peng. General purpose computing low-pows cow-owwer embedded gpus: Has it come of age? In 2013 International Conference on Embedded Compuler Systems: Architectures, Modeling, and Simalation (SAMOS), pp. 1 1-10, 2013. doi: 10.1109/SAMO00 AAA.AM00.200...2.00000
Sam McCandlish, Jared Kaplan, Dario Amodei, and OpenAl Dota Team. An empirical model of large-batch training. Xrvi preprinl or arXiu:1812.06163 2018.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint orXix:6609. 07844, 2016.
Scyed Morteza Nabavinejad, Hassan Hafez-Kolaht and Sherief Reda. Coordinated dvfs and procision control for deep neural networks. IEEE Computer Avchilecture Letters, 18(2):136 140, 2019.
Seyed Morteza Nabavinejad, Sherief Reda, and Masoumeh Ebrahimi. Batchsizer: Power- Power-performance trade- off for dnn inference. In 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 819 824, 2021.
Seyed Morteza Nabavinejad, Sherief Reda, and Masoumeh Ebrahimi. Coordinated batching and dvfs for dnn inference on gpu accelerators. IEFE Transactions Parallel and Distributed Systems, 33(10)22496-2500 2022.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, Spain, 2011.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Kilian Pfciffer, Ramin Khalili, and Jörg Henkel. Aggregating capacity in through successive layer training for computaationally-cnsstrrnee devices. Advances in Neural Information Processing Systems, 36:35386 35402, 2023.
Flavio Ponzina, Marco Rios, Alexandre Levisso, Giovanni Ansaloni, and David Atienza. Overflow-free com- pute memories for edge ai acceleration. ACM Transactions on Embedded Computing Systems, 22(5s):1 23, 2023.
Amine Saddik, Rachid Latif, Mohamed Elhoseny, and Abdelhafid El Ouardi Real-time evaluation of dif- ferent indexes in precision agriculture using a heterogeneous embedded system. Sustainable Computing: Informatics and Systems, 30:100506, 2021.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Lhang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 4510-4520, 2018.
Samuel L. Smith, Pieter-Jan Kindermans, and Quoe V. Le. Don't decay the learning rate, increase the batch size. In International Conference on Learning Representations, 2018
Zhenheng Tang, Yuxin Wang, Qiang Wang, and Xiaowen Chu. The impact of gpu dvfs on the energy and performance of deep learning: An empirical study. In Proceedings of the Tenth ACM International Conference on Future Energy Systems, pp. 315 325, 2019.
Robert J Wang, Xiang Li, and Charles Ling. Peleo: real-time object detection system on mobile devices. Advances in neural information processing systems, 31, 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist novel image dataset for benchmarking machine learning algorithms. Xiv preprint 1 077447777 2017.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Yuedong Yang, Guihong Li, and Radu Marculescu. Efficient on-device training via gradient filtering. In Proceedings of the IEEF/CVE Conference on Compuler Vision and Pollern Recognition, pp. 3811-3820, 2023.
Jie You, Jae-Won Chung, and Mosharaf Chowdhury. Zeus: Understanding and optimizing GPU energy consumption of DNN training. In 20th USENIX Symposium Networked Systems Design and Implemen- lation (NSDI 23), pp. 119 139, Boston, MA, April 2023. USENIX Association. ISBN 978-1-939133-93-5
URL Sttps://wwrr /ww. usenix http://feerrcceeaaaaaaaeeeaaaaaa
Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George Dahl, Chris Shallue, and Roger B Grosse. Which algorithmic choices matter at which batch sizes? insights from noisy quadratic model. Advances in neural information processing systems, 32, 2019.
Xingyao Zhang, Chenhao Xie, Jing Wang, Weidong Zhang, and Xin Fu. Towards memory friendly long-short term memory networks (Istms) on mobile gpus. In 2018 51st Annual (EEE/ACM International Symposium on Microarchitecture (MICRO), pp. 162-174, 2018.
16

---

Published in Transactions on Machine Learning Research (10/2025)
A Appendix
A.1 Training edge CPU
In this section, we briefty discuss training on an edge device's CPUs (i.e., Jetson Nano) and the reasons of why to focus only training on GPUs such devices. In Fig. we show the processing time and power for training over 50K samples with batch size 32, where both CPU and GPU are using the maximum frequency. We observe that the GPU is order of magnilude faster compared to the CPU, while nearly consuniig similar average power, Similarly, the GPU would be nearly an order of magnitude more energy efficient on these devices compared to the CPU. Thus, we focus in our work on edge GPUs as training more time and energy efficient.
II T(s)
Paog
10
10a
8
Time [s]
6
Power/W] [
4
10a
2
0
CPU
GPU
Figure ResNet18 time for training 50K samples (T:) and average power (red triangles) of Cifar10 on Jetson Nano over CPU and GPU.
A.2 Proxy dataset analysis extension
We provide a continuation for the sensitivity analysis for the proxy dataset selection. In Fig. 7 provide the confusion matrices for selecting different proxy datasets to train ResNet18. The difference for ResNet18 compared to MobileNetV2 (in Figure 5)is that the suboptimal configurations are selected for the lowest power constraint, while for other cases the selected configurations match the optimal.
Target Dataset CINIC
Target Dataset
Target Dataset
ClarlD SVHN
FMNIST
STL
Cilac10 SVHN CINIC FMNIST
STL
Clar10 SVHN CINIC FMNIST STL
Clar10
0
0
2.9
27.3
11
B
0
CMar10
0
0
0
0
Procy Dataset
SVHN
0
2.9
27.3
SVHN
0
0
SVHN
0
0
CINIC
PMNNSTT
1.8
10.2
FMNIST
FMNIST
STL
0
2.9
27.3
STL
D
STL
0
0
p1 maz
p2 max
N/A
Figure: 7: Matrices of mean time increase percentages to the fastest configuration for image classification datasets across three power constraints for ResNet18 training on Jetson Nano.
17

---

Published in Transactions on Machine Learning Research (10/2025)
A.3 Text datasets details
For Jane Austen's dataset part, we extracted it from the nitk package (gutenberg corpus) where the text consists of 3 novel namely Emma, Persuasion, and Sense and Sensibility. For Dickens, we used eight novels namely The Pickwick Papers, Pictures from Italy, A Tale of Two Cities, A Story of the French Revolution, The Chimes, Mugby Junction, The Haunted Man and the Ghost's Bargain, and The Mystery of Edwin Drood. We filter the licensing text, author history, etc. from the training and testing text.
A.4 Configuration selection visualization
In this section, we provide additional visualization for our method. Figures X and g illustrate the configura- tion candidates from our method under power constraints, together with the cases of no joint optimization. Specifically, this includes the power constraint cases of P and P for ResNet18 in Figure and trans- formers for the next character prediction in Figure g Our approach finds wider range of operating points in the scarch space, effectively adapting to different training workloads. For instance, the next-character prediction task less power-inlensive, which allows exploring broader set of operating points. Additionally, it selects the best configuration to minimize TTeee
-10
-10
10
1.5
Power[[]] [W]
1.5
6
TTace
62
aT
b1
62
4
0.5
0.5
6*
4
32 64 128
4
8
16
32 64
128
4
8
16
32 64 128
Batch Size
Batch Size
Batch Size
Figure Visualization the the power and resulting training times to accuracy for the configuration candidates of our method when finetuning ResNet18 on SVHN under power constraints of pl Phas (red) and pa Paaz (green) respectively. We highlight baselines 1 and (b1, b2), corresponding the cases without joint optimization, together with b" denoting the point selected by method.
Our method explores more operating points in the search space while selecting the best configuration.
-104
-10
10
2
1.5
[W]
Power| [W]
TTace
1.5
aac S
6
62
b2
4
0.5
0.5
8
16
32
64
128
8
16
32
64
128
8
16
32
64
128
Batch Size
Batch Size
Batch Size
Figure 9: Visualization of the the power and resulting training times to accuracy for the configuration candidates of our method when finetuning transformer on Austen dataset under power constraints of P (red) and p (green). We highlight baselines and (b1, b2), corresponding to the cases without joint optimization, together with b*, denoting the point selected by our method. Our method explores more operating points in the search space while selecting the best configuration.
18

---

Published in Transactions on Machine Learning Research (10/2025)
SVHN
CINIC
100
80
ResNet18
MobileNetV2
VIT
75
Accuracy
Accuracy
90
70
85
65
60
16
32
64
128
4
Batch size
Batch size
Figure 10: Maximum test accuracy across different batch sizes for the image classification task, showing stable performance with only marginal variation.
A.5 Effect of batch Size on model quality
To complement our main results, we analyze the impact of batch size on training quality. Specifically, we train the image classification models under different batch sizes and show the mean of maximum accuracy achieved over three runs test set in Fig. TO. The max accuracy remained consistent across different batch sizes, showing only marginal differences without signs of increased overfitting. Therefore, the proposed configuration selection approach can select the batch size and frequency combination that improves the training speed given a device's power constraint and capabilities without compromising model quality. This is consistent with prior work Goyal et al. (0177) showing that larger batch sizes can used without loss in accuracy.
A.6 Power measurement details
We read the power sensor values during training with a sample rate of 1s. For the Nvidia Jetson Nano, the power measurements are available read from using sysfs nodes. For example, on the jetson nano, the sysfs nodes are available in the path
19


Reviews:
[{"review_id": "0qBNzPvsiM", "reviewer": "Reviewer_PcJG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new method to accelerate neural network training on edge devices under power constraints by jointly optimizing GPU operating frequency and training batch size. Traditionally, only GPU frequency was limited to meet power constraints, which caused a large increase in training time. This study introduces a cross-layer control method spanning the hardware layer (GPU frequency) and the application layer (batch size), and accelerates training by adjusting both simultaneously within the device's power cap. Specifically, it predicts the efficiency of each batch size (the number of iterations needed to reach a target accuracy) using a proxy dataset, and profiles processing time and power for each batch size-frequency pair on the device, then selects the combination that minimizes total time to the target accuracy. As a result, the proposed method achieves up to a 2.3x reduction in training time (1.5x on average) over current baselines, and shows on real devices that total energy consumption is also reduced. It also achieves these improvements without degrading the final model accuracy. In addition, sensitivity analyses on five image classification datasets and three character-level prediction datasets verify the robustness and practicality of the method with respect to proxy data selection.\n\nMain strengths: The paper addresses an important problem of on-device learning on edge devices. The proposed method is an interesting approach that integrates hardware control and machine learning parameter tuning, and achieves large performance gains through the joint optimization of GPU frequency and batch size, which had not been explored before. Experiments are conducted comprehensively on real low-power GPU devices such as Jetson Nano and TX2NX, and the effectiveness is shown across multiple models and tasks, including image classification (ResNet18, MobileNetV2, EfficientViT) and natural language (character-level prediction with a Transformer), suggesting good generality. It also considers practically important points such as reducing energy consumption and maintaining accuracy, supporting the effectiveness and usefulness of the proposal.\n\nMain weaknesses: On the other hand, the novelty is a relatively simple optimization idea (combining elements that had been tuned separately in existing techniques), so it remains a systems-side engineering method rather than a theoretical breakthrough in machine learning algorithms. In other words, while the contribution is important for applications, the academic innovation is somewhat incremental. In addition, the method assumes that the device's power budget remains constant during training, and it cannot directly handle cases where the battery level changes over long training or where the power constraint itself changes. Furthermore, because estimating the optimal batch size requires pretraining with a proxy dataset, the selection of that dataset and the additional cost are practical issues in real deployments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims in the paper are supported by sufficiently convincing evidence. The authors claim that the proposed method \"speeds up training by 1.5x on average\" and \"reduces energy consumption,\" and this is backed by detailed experiments on real devices. For example, on Jetson Nano with ResNet18 for image classification, the method reaches the target accuracy 1.5x faster on average than existing methods (Baseline 1: frequency limiting under a power cap with maximum batch size; Baseline 2: selecting a batch size that converges faster, etc.), with a peak speedup of 2.3x. These results are clearly shown in tables and graphs. Consistent improvements are also observed for other models and tasks such as MobileNet and Transformer, indicating effectiveness not limited to specific cases. For energy consumption, the proposed method uses less than the baselines, and in some cases it is reported to require about half the energy of prior methods. Therefore, the paper's claims are clearly supported by abundant experimental results and analysis, and the evidence is reliable.", "audience_yn": "Yes", "audience_explanation": "There will be readers within TMLR's audience who are interested in this work. In recent years, machine learning on edge devices (on-device learning) and energy-efficient AI have become highly active research areas. This study fits exactly in that area and addresses a practical and somewhat urgent problem of improving training efficiency under resource constraints; the results (significantly shorter training time and improved energy efficiency) will be useful information for researchers and engineers interested in edge AI and embedded machine learning.", "requested_changes": "Please further clarify the positioning of the proposed method and the differences from prior work in the introduction and related work. In particular, compared with existing methods for similar problems (e.g., those that control only GPU frequency or that adjust batch size for inference), emphasizing what this work achieves for the first time will make its novelty clearer to readers. From the current text it can be inferred that \"joint optimization of batch size and frequency has no precedent,\" but based on the early example that shows a 31.9% speedup in configurations that prior methods could not explore, please explicitly summarize the uniqueness of your method.", "broader_impact_concerns": "There are no particular serious ethical or social concerns for this study. Rather, by improving the efficiency of on-device learning, the method may contribute to privacy protection (training finished on the device without sending data to the cloud) and to power savings, so positive impacts are expected.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "lCCYdw1nvX", "reviewer": "Reviewer_vngf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel cross-layer methodology to accelerate the training of deep learning models on power-constrained edge devices. The authors argue that conventional methods, which typically reduce GPU frequency to meet power limits, lead to significantly longer training times. Their proposed solution is to jointly optimize the GPU frequency (a system parameter) and the training batch size (an application parameter). The methodology involves two main phases: offline profiling of the neural network's power consumption and training time on the target device for various frequency and batch size combinations, and estimating the relationship between batch size and the number of iterations required to reach a target accuracy. This estimation is done on a powerful server using a proxy dataset. By combining these two pieces of information, the system can select the optimal frequency-batch size pair that minimizes the total training time while respecting the device's power constraints.\n\n1. It clearly shows that independently setting the GPU frequency and batch size is suboptimal for training on power-constrained edge devices.\n\n2. The authors propose a practical method that combines device-specific hardware profiling with server-based training analysis (using a proxy dataset) to find the best operating point (batch size and GPU frequency).\n\n3. The evaluation on real hardware (Nvidia Jetson Nano and TX2NX) demonstrates a reduction in training time by up to 2.3x and an average of 1.5x compared to baseline approaches, without compromising the final model's performance.\n\n4. The method is validated across various models (ResNet18, MobileNet V2, EfficientViT, Transformers), tasks (image classification, next character prediction), and datasets, showing its robustness and practicality. The authors also analyze the impact on total energy consumption, noting significant reductions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This version of paper has been improved a lot. The authors have revised their performance claims and corrected data in the results tables, making the latest version more accurate and trustworthy. \n\nThis version provides a much more detailed and specific summary of the contributions, giving the reader a clearer understanding of the work's scope from the outset. The contributions are explicitly detailed, mentioning the exact models and tasks: \"as shown across representative models (CNNs and Transformers) and tasks (image classification and next character prediction)\". It also explicitly names the models and datasets used in the evaluation summary bullet point. This level of detail is a significant improvement.\n\nBy correcting the main performance metric (2.4x=>2.3x) and a key data point in the results, the authors have substantially improved the paper's trustworthiness.", "audience_yn": "Yes", "audience_explanation": "Low-power ML should be in the scope of interests of TMLR.", "requested_changes": "I don't have further requests here.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fct1anze5f", "reviewer": "Reviewer_rhhh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work focuses on addressing the challenge of efficient deep learning model training on power-constrained edge devices.\n\nIts key contribution is a cross-layer methodology that jointly optimizes system parameters (GPU frequency) and application parameters (batch size). This approach, which combines offline device profiling with server-side batch size efficiency estimation, dynamically selects the best configuration to minimize training time while adhering to power limits.\n\nExperiments show this method reduces training time by up to 2.3x and lowers energy consumption compared to baselines, without sacrificing model performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper is well written and is relatively easy to understand. The authors provided performance evaluations to support their claim. However, I have a few questions:\n\n- Since this method utilizes an estimation method for deciding the batch size. It is surprising to see in Tables 2 and Table 3 that the performance of the Joint optimization is actually as good as the optimal one. Is there any case where the proposed method will not give us the optimal? \n\n- Even though it is beneficial for training speed to jointly optimize both batch size and GPU freqency, more analysis should be provided in terms of the other effects of using different batch sizes, such as max accuracy achievable, generalization of the model, overfitting, etc.\n\n- It is unclear how to select a proxy dataset in a real training scenario, and how the proxy dataset is used to estimate the real dataset for server-side batch size efficiency estimation is not too clear.", "audience_yn": "Yes", "audience_explanation": "Training on power-constrained edge devices is a relevant topic for the community. Jointly optimizing GPU frequency and batch size, can be a topic that benefits sustainable ML.", "requested_changes": "- For figure 8 and figure 9, it would be beneficial to also show the curves without join optimization and just optimize for the best batch size, so the readers can better understand the benefit of the join optimization.\n\n- address the concerns mentioned above", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)