The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones for Progress Towards More Efficient Model Inference
Authors: Zeyu Yang
Wesley Armour
Venue: ICML 2025 poster (2025)
Keywords: Energy Efficiency
Green AI
Efficient and Scalable Vision
Sustainability
Primary area: general_machine_learning->evaluation

Abstract: Deep learning has achieved significant success but poses increasing concerns about energy consumption and sustainability. Despite these concerns, there is a lack of understanding of their energy efficiency during inference. In this study, we conduct a comprehensive analysis of the inference energy consumption of 1,200 ImageNet classification models—the largest evaluation of its kind to date. Our findings reveal a steep decline in accuracy gains relative to the increase in energy usage, highlighting sustainability concerns in the pursuit of marginal improvements. We identify key factors contributing to energy consumption and demonstrate methods to improve energy efficiency. To promote more sustainable AI practices, we introduce an energy efficiency scoring system and develop an interactive web application that allows users to compare models based on accuracy and energy consumption. By providing extensive empirical data and practical tools, we aim to facilitate informed decision-making and encourage collaborative efforts in the development of energy-efficient AI technologies.

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones for Progress Towards More Efficient Model Inference
Zeyu Yang'
Wesley Armour'
Abstract
becoming major concern. Google reported that the energy consumption of machine learning workloads constituted 10-15% of its total energy usage from 2019 to 2021, with training accounting for 40% and inference for 60% (Patter- son et al., 2022). Similarly, Meta observed a power capacity distribution of 10:20:70 among experimentation, training, and inference in their Al infrastructure (Wu et al., 2022).
Deep learning has achieved significant success but poses increasing concerns about energy consump- tion and sustainability. Despite these concerns. there is lack of understanding of their energy efficiency during inference. In this study, we con- duct a comprehensive analysis of the inference energy consumption of 1,200 ImageNet classifi- cation models- largest evaluation of its kind to date. Our findings reveal steep decline in accuracy gains relative to the increase in energy usage, highlighting sustainability concerns in the pursuit of marginal improvements. We identify key factors contributing to energy consumption and demonstrate methods to improve energy effi- ciency. To promote more sustainable Al practices, introduce energy efficiency scoring system and develop an interactive web application that al- lows users to compare models based on accuracy and energy consumption. By providing extensive empirical data and practical tools, we aim to fa- cilitate informed decision decision-making and encourage collaborative efforts in the development of energy- efficient Al technologies.
Moreover, the electricity consumption of these tech giants has been rising steadily. Google's 's electricity usage increased by an average of 21 % per year over the past decade, grow- ing from 3.7 TWh in 2013 to 25.3 TWh in 2023 (Google, 2019; 2024). Meta's electricity consumption grew by an average of 32% per year over the past five years, from 4.9 TWh in 2018 to 15.0 TWh in 2023 (Meta, 2024). Data centers globally were estimated to consume about 1 1% of global electricity and contribute 0.3% of greenhouse gas emissions in 2018 (Masanet et al., 2020; Jones et al., 2018). Furthermore, the widespread adoption of autonomous ve- hicles could require as much electricity as all current data centers combined (Sudhakar al., 2022).
The high energy consumption associated with AI leads to several negative consequences. Economically, it results in higher capital costs for purchasing computing hardware and increased operating expenses for electricity and cooling. Environmentally. it produces a large carbon footprint, ex- acerbating climate change. Additionally, the substantial computational demands impede further development and innovation in AI. The soaring costs of purchasing rent- ing computing power close the door to many researchers, leaving only large tech companies as major players in the field. This centralization contradicts the open-source ethos that has traditionally driven Al and software development. Moreover, high energy consumption hinders the deployment of AI in edge scenarios where battery life and thermal de- sign power (TDP) are constrained (Yang et al., 2022; Pereira et al., 2023).
1. Introduction
Over the past decade, AI has achieved remarkable capa- bilities in various fields. However, these accomplishments have come at the cost of significant computational demands. AI research has traditionally prioritized achieving the high- est possible accuracy, often disregarding considerations of model size, complexity, and data requirements.
As the field matures and more Al products and services tran- sition into commercial deployment, computational cost is
'Department of Engineering Science, University of Oxford, Oxford, UK. Correspondence to: Zeyu Yang <zeyu, yangu.yange @ ang.ox.ac.uk>.. Wesley Armour wes.armour@oorr..oxx
Despite these concerns, there is a lack of comprehensive understanding of the energy consumption during inference and the energy efficiency of different models. In this study, we address this gap by measuring the inference energy con- sumption of 1,200 ImageNet classification models-a scale that is orders of magnitude larger than any previous work.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
Our research aims to answer the following questions: How much additional cost are we incurring for marginal increases in accuracy? What are the contributing factors to energy consumption in Al models? Do current acceleration tech- niques improve energy efficiency? How to trade-off between energy consumption and accuracy?
accuracy regardless of computational cost norm in the field. They also advocated for reporting model's FLOP count as a standard practice in publications. Van Wyns- berghe (Van Wynsberghe, 2021) introduced the Sustainable Al movement to promote changes throughout the Al life cycle - including training. fine-tuning, implementation, and governance towards ecological integrity and sustainable development.
Our key contributions are as follows:
Extensive Dataset: We provide a comprehensive dataset of inference energy consumption metries for 1,200 ImageNet classification models, enabling better understanding and comparison of model efficiencies.
Li et al. et al., 2016) were among the first investigate energy efficiency on GPUs, testing both the training and inference of AlexNet. OverFeat, VGG, and GoogleNet on NVIDIA K20m and TITAN X GPUs. They identified the energy consumption of different CNN layers and analyzed the impact of hardware settings such as batch size, hyper- threading, ECC, and DVFS on energy efficiency.
Insights into Energy Consumption Factors: We identify factors contributing energy consumption, correct existing misunderstandings. and assess the impact of throughput improvements on energy consumption.
Canziani et al. (Canziani et al., 2016) conducted compara- tive analysis of more than a dozen models, including variants of Inception, VGG, and ResNet, evaluating metrics such as accuracy, memory usage, inference time, and power con- sumption on an NVIDIA Jetson TX1. Their work aimed to guide efficient DNN design for practical applications by highlighting the trade-off's between accuracy and compu- tational requirements. Yao et al. (Yao et al., 2021) tested three CNNs VGG16, ResNet50, and Inception-V3. on three GPUs: NVIDIA Tesla M40, P4, and V100. They highlighted the impact of different configurations and opti- mizations, including quantization and the use of TensorRT and Tensor Cores, on energy consumption, providing in- sights for more energy-efficien deployment of CNNs in nigh-performance computing environments. Overall, the number of models evaluated in these works is limited, and the GPUs used are outdated by today's 's standards.
Two new ways to quickly estimate the energy consump- tion of model: 34% error just from model param, gmacs and activation count, and less than 1 1% error from throughput and TDP of GPU.
Energy Efficiency Scoring System: We introduce a scoring system to rank models based on their energy efficiency, providing a standardized metric for evalua- tion and informed eecision-making
Interactive Web Application: We develop web app that allows users to visualize and compare models based on energy efficiency and other metrics, promot- ing conegyyconscious choices within the community.
Our study empirically confirms that current deep learning models are significantly less energy-propon pronotpprpootionall than ideal- ized hardware expectations suggest, aligning with insights from (Barroso & Hölzle, 2007). The steep diminishing returns we identify reinforce the need for more energy- proportional model architectures and inference methods.
Henderson et al. (Henderson et al., 2020) proposed a stan- dardized framework for consistent reporting of energy and carbon emissions ML research, aiming to raise awareness, enable cost-benefit analyses, and promote energy-efficient practices in model development and deployment. One of their main arguments was that model's parameter count and FLOPs do not necessarily correlate with energy con- sumption. They tested over 20 models, including VGG, ResNet, MobileNet, and SqueezeNet. However, they did not specify which GPU they used. Most importantly, they ran all models with a batch size of one, which underutilizes any reasonably modern GPU and gives larger models an unfair advantage.
Our findings directly support the goals outlined by (Schwartz et al., 2020), advocating for Green AI principles that prioritize energy efficiency alongside accuracy. By demonstrating the marginal gains in accuracy versus exponential energy demands, our work underscores the urgency emphasized by the Sustainable Al move- ment ( (Van Wynsberghe, 2021) to embed sustainability criteria into Al model evaluation and selection.
2. Related Work
Desislavov et al. (Desislavov et al., 2023) conducted one of the most extensive analyses to date, examining 94 dif- ferent ImageNet classification models. They showed that efficiency gains from hardware advances and algorithmic improvements mitigate energy growth despite increasing model complexity. However, they estimated the energy con- sumption of model by dividing the model's FLOPs by the
Several researchers have called on the Al community to raise awareness of the energy consumption of Al models and their subsequent environmental consequences. Schwartz et al. (Schwartz et al., 2020) proposed the concept of Green AI, which emphasizes computational efficiency alongside model quality, as opposed to Red AI that prioritizes higher
2

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
GPU's FLOPs per second and multiplying by the GPU's TDP. This is a highly idealized and optimistic assumption, which, as we show in our results section, differs significantly from real-world scenarios.
2020) from the "Ampere" generation and the H100 PCIe 80GB (NVIDIA, 2023) from the "Hopper" generation. Both deliver state-of-the-art performance and efficiency in deep learning computations. Key hardware and software configu- rations can be found in the appendix.
Shifting focus away from computer vision, Samsi et al.(Samsi et al., 2023) benchmarked the inference energy and compute requirements of various configurations of the LLaMA model across GPU setups to highlight energy us- age patterns and identify optimization opportunities for re- source efficiency. Luccioni et al. (Luccioni et al., 2024) benchmarked 80 models across 10 specific tasks and 8 general-purpose models, providing a systematic comparison of energy consumption and carbon emissions in deployment. They emphasized the significantly higher costs of deploying general-purpont models compared to task-specific ones and urged careful consideration of these environmental impacts.
Inference Deployment Methods We evaluated the mod- els with two inference methods: standard PyTorch (Ansel et al., 2024) at FP32 precision and NVIDIA TensorRT at FP16 precision. Standard PyTorch serves as a non- optimized baseline, while TensorRT represents a production- level optimization for NVIDIA GPUs.
Accuracy Metries To thoroughly evaluate the models' accuracy, robustmess, and generalization, we used six val- idation/test datasets: the original ImageNet validation set (Russakovsky et al., 2015), along with five widely recognized datasets: ImageNet Real Labels (Beyer et al.. 2020), ImageNet V2 Matched Frequency (Recht et al., 2019), ImageNet Sketch (Wang et al., 2019), ImageNet Adversarial (Hendrycks et al., 2021b), and ImageNet Rendi- tion (Hendrycks et al., 2021a). These datasets assess stan- dard classification accuracy as well as performance across diverse visual domains and distribution shifts.
Beyond evaluating 50-100x more models, our work ad- dresses key limitations of prior studies, such as using batch size- = and TDP-based energy estimates. Our rigorous methodology yields accurate, realistic conclusions. We clar- ify deployment details often omitted in past work, showing PyTorcc can be up to 10x more mneryy-oonsuinng than TensorRT. Measurements were conducted under an industry- standard inference scenario to ensure real-world relevance. Prior studies also used outdated models and GPUs. By evaluating models up to 2024 on Hopper and Blackwell Generation GPUs, our study offers the most comprehensive analysis to date. Our open-source framework on GitHub' supports easy evaluation of new models and GPUs We invite the community contribute by utilizing our frame- work to assess their models and GPUs and contribute to our GitHub repo and web app.
Measurement Procedure We developed an automated script to evaluate all selected models. We iteratively in- creased the batch size, starting from and doubling until reaching the GPU's memory limit, ensuring maximum hard- ware utilization and fair comparisons across different model sizes. For each batch size, we performed two runs: warm- up to handle potential out-of-memory errors and prepare the GPU, followed by measured run. Inference was continued until more than 13 repetitions and 10 seconds of runtime were reached, ensuring sufficient data for both large and small models. More details can be found in the Appendix.
3. Methodology & Experimental Setup
Following the guidelines in (Yang et al., 2024), we measured GPU energy consumption using onboard power sensors via nvidia-smi. We recorded power usage and other GPU metries at 100 Hz, then logged the data for subsequent analysis.
Energy Measurement
Model Selection To comprehensively analyze energy ef- ficiency across diverse model architectures, we included all available pre-trained models from the Hugging Face PyTorch Image Models (Timm) library (Wightman, 2019). Timm is widely used for its extensive collection of state-of- the-art vision models encompassing convolutional neural networks, vision transformers, and hybrid architectures. We ultimately selected over 1,200 models in order to capture distinct energy-accurace trade-off's across various architec- tures, depths, and publication times. This breadth ensures our results are not simply redundant and that they capture the true diversity of existing approaches, covering a wide range of model sizes, complexities, and design philosophies.
Additional Metrics We collected key model statis- tics-such as parameter counts and FLOPs-using pt- flops (Sovrasov, 2018-2024) and torchinfo (Yep, 2020). We also recorded GPU utilization, VRAM usage, and tempera- ture to investigate how these metrics relate to model charac- teristics, performance, and energy consumption.
Result Integrity & Reproducibility We had exclusive machine access during experimentation, preventing interfer- ence from other processes. All experiments used the same A100 and H100 GPU cards for consistency. The servers were housed in a data center with controlled cooling, and GPU temperatures remained within operational ranges. We have made our source code available in a GitHub repository to facilitate replication and verification of our results.
Hardware Configuration All main experiments used two NVIDIA GPUs: the A100 PCIe 40GB (NVIDIA,
htt.uub..com//mmmeyaaaaaa DI- In-rreneeeee Reeeeeennneeeeeeeeee fffi ficency
3

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
90.0
90
80
Acciracy (1)
Accuracy 180
70
All models
80.0
MobileNeet V4
H100 TensorKT FP16
77.5
GENet
MLP
HGNet-v2
CNN
75.0
BEIT-V2
50
Transformer
EVA
Hybrid
72.5
EVA02
Fitted Growth
VGG
70.0
10a 10a 10a 10a
10*
10a
10a
10a
10*
Energy Consumption imjrlmagel
Energy Consumption (gmmg)
Figure 1. Energy consumption data plotted against the ImageNet accuracy for the H100 TensorRT FP16 inference setup. Models are categorized into general architectures, and the large black markers shows the center of the individual clusters.
Figure 2. Six highlighted models that largely forms the efficient frontier. ML veteran VGG is also highlighted for reference.
approximately doubles accuracy (from 40% to 80%). The second decade increases accuracy by only 7% (to 87%), and further tenfold increase gains just 3% (to 90%). with neg- ligible improvements thereafter. This pattern exhibits loga- rithmic growth on a logarithmic scale, indicating nested logarithmic relationship. We fit this growth function to the models on the efficient Pareto front of the H100 Ten- sorRT data (gray line in Fig. 1). Extrapolating suggests that achieving 100% accuracy would require about 207 MWh of electricity per image-enough power San Francisco for approximately 15 minutes.
Table Pearson Correlation Coefficient (PCC) and Spearman's s Rank Correlation Coefficient (p) of the energy consumption of models across different deployment setups: comparing same GPU with different software and different GPU with same software.
PCC p
A100- PT
H100 TRT
A100- TRT H100- PT
0.8553 0.9345 0.9939 0.9904
0.9840 0.9943 0.8299 0.9201
4. Results
In this section, we present our findings in four parts. First, we provide the overall energy consumption data of all tested models, analyzing observations and trends. Next, we in- vestigate the factors that contribute energy consumption, aiming to correct common misunderstandinns. We then ex- plore methods that improve energy efficiency and examine the relationship between other metrics. Finally, we discuss the trade-off between accuracy and energy consumption and introduce our interactive web app designed facilitate this and promote efficient and responsible Machine Learning.
When comparing architectures, MLPs generally yield lower accuracy at higher energy consumption, CNNs occupy the lower accuracy and energy range, and Transformers re- side in the higher accuracy and energy range. Hybrid CNN-Transformerss lie between these extremes. The large black markers in Fig. 1 show the centers of these clusters.
Efficient Frontier Models As shown in Fig.2. Models on the efficient frontier primarily come from just six fami- lies MobileNet-V V4 (Qin et al., 2025), GENet (Lin et al., 2020), HG (Contributors, 2023), BETT-V2 (Peng et al., 2022), EVA (Fang et al., 2023), and EVA02 (Fang et al., 2024) - ranging from lower to higher accuracy and energy consumption. The first three are CNN-based, while the latter three are Transformers, indicating no single archi- tecture uniformly maximizes energy efficiency.
4.1. Energy Consumption Results and Trends
Overall Results Fig. plots the energy consumption of each model against its ImageNet classification accuracy for the most efficient H100 TensorRT FP16 setup. The model distribution remains largely consistent across different in- ference methods (see Table 1), and data for the other three setups appear in the Appendix.
MobileNet-V4, V4, originally designed for eesource-consrring edge devices, remains highly efficient on the H100 80G, one of the largest GPU. However, as the model scales up, other families surpass its performance. Both GENet and HGNet-V2 are optimized for low-latency GPU inference via grouped and depthwise-separable convolutions, tuned network depth and width, residual bottlenecks, memory- efficient access patterns, and parallel-friendly operations. Although energy consumption correlates strongly with throughput (shown in later sections). it is notable that these latency-oriented designs also excel in energy efficiency.
These models are categorized by their architecture into Multi-Layer Perceptrons, Convolutional Neural Networks, Transformers, and hybrid TranSfranforrmr models. Note that the energy consumption axis is on logarithmic scale.
A prominent finding is the steep diminishing returns in ac- curacy as energy consumption grows. Energy usage spans four orders of magnitude (a factor of 10,000). In the first or- der of magnitude, tenfold increase in energy consumption
4

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
Moving higher accuracy and energy consumption, BEÎT- V2, EVA, and EVA02 not explicitly designed for efñi ciency. Rather than reducing energy for the same accuracy, they offer higher accuracy at equivalent energy levels totally another valid path toward more efficient progress.
80
70
Mixed Accuracy (%0
60
50
Performance on Other Datasets As the original Ima- geNet dataset has largely reached performance saturation, Fig.3 plots energy consumption against the average accuracy across the other five datasets described in the Methodology. Accuracy naturally decreases here because some of these datasets are more challenging. Notably, lower-energy CNNs experience larger performance drops, whereas higher-energy Transformers maintain stronger accuracy, consistent with the expectation that larger Transformer models offer greater robustness against domain shifts. Although the diminishing- return effect appears less pronounced than ImageNet, it still follows a
H100 TenserRT FP16
40
MLP
CNN
30
Transformrr
Hybrid
20
ImageNet-O Frontier
10a
10a
10a
10a
Energy Consumption (mjling))
Figure 3. Energy consumption plotted against average accuracy of the five other datasets for robustness analysis. The red line high- lights the pareto front models for the original ImageNet dataset.
pattern.
Accuracy
One might wonder the efficient frontier models for the original ImageNet lose their advantage on these additional datasets. The red line Fig.3 marks the Pareto front for ImageNet. We find that most frontier models remain effi- cient, except in the 60%-70% accuracy range, where several ResNeXt (Xie et al., 2017) models surpass them. This result likely arises from ResNeXt's relatively high accuracy on ImageNet Sketch, attributable to its wider "cardinality" ar- chitecture that enhance the out-of-distribution performance.
60
50-
RTX 3090
RTX 4090
40
90
Accuracy
To check whether the observed trends are generalizable to other GPUs, tested the 128 most effi- cient models on H100 on 4 other GPUs: RTX 3090, 4090, 5090 and mobile laptop GTX 1650 Ti. The results, shown in Fig. 4, strongly confirm the original trends observed on A A100 and H100, further validating the broad applicability of our findings.
Other GPUs
GTX 1650 TI
RTX 5090
101
105
10f
107
10a
105
10f
10"
Energy (uj)
Energy (uj)
Figure 4. 128 most efficient models H100 tested other
GPUs: RTX 3090, 4090, 5090, and a Laptop GTX 1650 Ti.
250
Pyforch
TensarRT
We found that the RTX 5090 offered limited efficiency gains.
Frequency
200
Across 128 models, the RTX 4090 achieved a 21.0% geo- metric mean energy reduction over the RTX 3090, while the RTX 5090 improved by only 5.7% over the RTX 4090. This slowdown in GPU generational energy efficiency gains high- lights the importance our research, as relying on newer GPUs being more efficient alone no longer sufficient.
150
100
50
10
20
10
20
30
40
Underestimation
Underestimation Factor
Figure 5. Error in naive estimation of energy consumption based on FLOPs compared with actual measurements on the A100 GPU, The geometric mean of the underestimation factor is 3.13 and 3.16.
4.2. Understanding of Energy consumption
We investigate the relationship between energy consumption and various model metrics, including the number of parame- ters, FLOPs, activations, and input image size. Additionally, aim to correct some misconceptions about the energy consumption of models.
the energy consumption model by dividing the model's FLOPs by the GPU's FLOP/second and multiplying by the GPU's TDP. We replicated this calculation and compared the estimated energy consumption with our actual measure- ments. Fig.5 shows the distribution of underestimation. On average, this method underestimates energy consumption by approximately three times and can underestimate by nearly 40 times in some cases.
Error of Naive Energy Estimation As mentioned in the related work section, (Desislavov al., 2023) estimated
5

---

The Hidden Joules:
Evaluating the Energy Consumption of Vision Backbones
Parameters, FLOPs, and Activations Intuitively, larger and more complex models demand more computation and thus consume more energy. However, (Henderson et al., 2020) concluded that "FPOs and Params have no strong correlation with Energy Consumption." In contrast, our re- sults reveal moderately strong linear correlation between parameter count and energy consumption, and very strong correlation for both FLOPs and activations (Fig.6). Build- fitted simple multiple linear
Energy Image (mg)
10*
10a
10
10a
ing on these correlations, regression (MLR) model to estimate energy consumption from Params, Gmacs, and activations, achieving Mean Absolute Percentage Error of 34.73%. Although this em- pirical model is not highly accurate, it offers a quick way to gauge energy efficiency from basic model statistics an improvement of nearly an order of magnitude over the naive FLOPs estimates mentioned above. Details of the fitted formula provided in the Appendix.
10'
10"
1010
10
Parameters
gmacs
10a
Energy Image (m))
10a
10a
10a
For models that accept variable input sizes, increasing the input size yields only negligible im- provements in accuracy but results in substantial increase in energy consumption. F illustrates the increase ac- curacy and energy consumption as the input size increases for subset of models that support variable input sizes.
Input Image Size
10c
107
100
10
10
10
10a
activations
Predicted Energy (m))
Figure 6. Relationship between energy consumption per image and number parameters, FLOPs, and activations (A100 with Py- Torch FP32). Pearson Correlation Coefficients are 0.6679, 0.8021, and 0.9023, respectively. The plot on the lower left shows the predicted energy using the emprical model the actual Energy consumption per image.
4.3. Efforts to improve energy efficiency
120
Accuracy 0.01 Input Size + 0.29 Energy 0.96 Input Size t 1.44
GPU Utilization We examine how GPU utilization affects inference energy consumption. Larger batch sizes process multiple inputs in parallel to increase hardware utilization, hence reducing per-sample overhead, thereby increasing efficiency. Fig.8A shows the energy consumption of Effi- cie on the A100 GPU with progressively larger batch sizes up to the GPU memory limit. Energy consumption drops significantly as batch size grows-until memory band- width and processing units become fully utilized. Beyond this point, further increases yield minimal improvements. In some extreme cases, CUDA libraries may adopt less effi cient strategies to fit larger batches into memory, ultimately increasing energy consumption.
Percentage Increase (%)
100
80
60
40
20
40
60
80
100
120
Input Size Increase (%)
Figure 7. Increase in accuracy and energy consumption as the input image size increases (A100 with TensorRT). The increase in accu- racy is minimal, whereas energy consumption is almost directly proportional with input size.
Energy consumption, Throughput, and TDP FBg.8B shows the throughput and latency achieved at varying batch sizes. Latency increases linearly with batch size, while throughput increases initially and then plateaus. We ob- served an inverse relationship between throughput and en- ergy consumption. Throughput is measured in images per second, and energy consumption is measured in joules per image. The product of these two gives the average power draw of the GPU during model execution:
Power (TDP), and the GPU maintains power draw not to exceed the TDP under heavy load.
To confirm this inverse relationship. we plotted through- put against energy consumption in Fig.8C, along with the maximum possible product of the two (the TDP line). As batch size increases. energy consumption decreases, while throughput increases. Although the relationship is not strictly linear, when batch size is small, the GPU is under- utilized, resulting in an average power draw below the TDP. As batch size increases, GPU utilization improves, and the throughput-energ. consumption combination approaches the TDP limit. The conclusion, although counterintuitive. is that higher average power draw leads greater energy
Imgs
Joules Joules
Watt Aug Drow
Tmg
(1)
The maximum power a GPU can draw is its Thermal Design
6

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
10"
PyTorch TensorkT
Pyforch
Decraaee in Energy (Facton)
Sneggy (Joules/ma)
10"
W10
TensorKT
TDP@250W
Throughpus suguhpu
200
10
10
10-
50
100
150
200
250
(A)
Batch Size
(B)
Energy Consumption (/image) Increase in Throughput (Factor)
Throughput (ingsise]
Figure 9. Relationship hetween energy consumption, throughput, and TDP: (A) Energy consumption
100
Latency (ms)
throughput for all mod- els, showing the TDP limit, and (B) Improvement in energy con- sumption versus throughput when using TensorRT instead of Py- Torch on the A100 GPU.
10a
Throughput
Latency
10
100
150
200
Batch Size
Energy Reduction Factor
PyTorch
Throughput (imgs(se)
TensorRT
TDP 250W
10
Energy Cons. (oules/mng)
10"
10a
102
10a
10
50
100
150
(D)
PyTorch TensorRT TDP @ 250W
Pytarch Energy per image (m))
Frequency
Draw(w)
Figure I0. Reduction energy consumption when using TensorRT instead of PyTorch on the A100. The geometric mean reduction factor with a geometric standard deviation of 1.53. the H100, the geometric mean reduction factor is 4.02 with a geometric standard deviation of 1.55.
200
150
50
10-1
Energy Cons. (oules/img)
Fig.9B illustrates the proportional relationship between increased throughput and decreased energy consumption when switching from PyTorch FP32 to TensorRT FP16. Since many inference accelerators focus on throughput, and the TDP ceiling enforces fixed power limit, any improve- ment in throughput directly translates into lower energy consumption.
Figure 8. Analysis of batch size effects on EfficientViT on H100: (A) Energy consumption at different batch sizes, (B) Throughput and latency, (C) Throughput vs energy consumption with the TDP limit indicated, and (D) Average power draw relative batch size.
efficiency due better GPU utilization.
This observation underscores the importance of making sure the inference setup for a particular model fully utilize the hardware of a particular GPU, ensuring a fair comparison between different models. Simply performing inference with a batch size of one (as mentioned in related works) would unfairly favor larger and more complex models, they naturally consume more GPU resources.
Energy Savings from TensorRT FP16 Fig.10 shows the reduction in energy consumption for each model when switching from ToTch FP32 to TensorRT FP16. On aver- age, energy consumption decreases by about fourfold. This improvement stems from lower computational and mem- ory overhead with FP16, as well as the layer/kernel fusion and kernel tuning provided by TensorRT, Notably, mod- els that were highly energy-intensifn under PyTorch greater gains (up to 10x), likely due to less efficient Py- Torch implementations. This also explains the relatively weak correlation between PyTorch and TensorRT energy consumption on the same GPU compared to using the same software on different GPUs.
Infer Energy Consumption from Throughput Fig.9A plots energy consumption against throughput for all models at their most efficient batch size on the A 100 GPU. Each model lies on the TDP line. One can accurately estimate a model's energy usage from its throughput alone with a Mean Absolute Percentage Error of 0.97% (c.g., when the GPU is unavailable, but throughput is reported paper).

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
to plot Test Datasets:
Options Effickency
A100 PCIE 10G6 O H100 PCIE 803B
imagenet oniginal
Score: * We gitldd Mameltan
Accuracydooule
X axi Scale: O Lineer
Log
integentee-eel
Deployment option:
C Imagench22 manched frquoncy
Senr Waightngg
50
Show:
* PyTorch O Tensoifl
etctch
imagenw-a
Acseptable Accuunicy
C imagenet dizan
imagentt-r
Mecimum Acceptablee Energy (md):
00006
Imagenel-s-clen
Accuracy (%)
66
50
commrrmer b0b Lo
celome
hgreeiz to as
calermer .126 wtd...md....... sei
50
conmence tema.
aprenvt
70
Accurnry (N)
COD.1.0 Aamb SEiR
224
WL.redumd.atte
botv2 base aetcc1o....
ACTUSSENV
65.392
SOCEEFT
26.913 DJ
serereotao10dd
Chronghouts
2213.267 s#g//
Latancy:
202.256 ma
senerellaatthd Loen.
00-
remerrotenn 21d 19sh..
hpnenvo jud anit
noneet Jarge be
50
repedd ind....
regrady
www. p9 19..
6 Faze
Energy (m3)
Energy (mJ)
Figure 11. Screenshot of the interactive web application. The top lets select inference setups, datasets, scoring metrics, and plotting options. The scatter plot shows energy consumption versus accuracy for all models based on these selections. Hovering over point reveals the model's details, and clicking opens its Hugging Face page. The red lines indicate user-defined accuracy and energy thresholds, while the background displays a contour map of the selected efficiency score. The bar plot on the right highlights top-perforning models under the chosen metric. The web app available at: https: /ttess///mmmmmeee Lmzeyuyang. gå huubioi DI- IwwerenceeEneeeeeeeeeeeeeeeeeeee Energy- ffteeggy-------eeeeeeeeeeeeeeele fefficccccy....oo
4.4. Trade-off between Model Accuracy & Energy
energy consumption):
propose two methods to evaluate the trade-off between energy consumption and achieved accuracy. standard "bang for the buck" calculation measures efficiency the ratio of accuracy to energy consumption (in percentage per joule). However, this metric may unfairly favor less accurate models. For instance, a trivial model that always outputs "goldfish" could achieve 0.1% accuracy on ImageNet- 1K while consuming negligible energy, yielding misleadingly high efficiency score despite being practically useless.
+ + (1 W) (100 - -4) A)) (2)
score = 100 -
N
where E the energy consumption, is accuracy, is the maximum energy consumption among all models (for normalization), and W is weight between 0 and 100. A weight of 100 prioritizes energy consumption entirely, while O prioritizes accuracy. Users can adjust W to suit specific requirements: for example, a medical diagnostic model might use W = 5 for high accuracy, whereas categorizing personal photo albums might work well with W = 90.
To mitigate this issue, recommend using the efficiency ratio alongside minimum accuracy threshold. This ap- proach ranks models based on energy efficiency only if they meet a baseline level of accuracy. Additionally, as with prioritizing accuracy, maximizing efficiency alone may not be ideal in certain use cases. In critical applications such as autonomous driving or medical diagnosties, accuracy is often paramount, regardless of power constraints.
To help researchers and practitioners explore these trade- offs, we developed an interactive web application that vi- sualizes and compares the energy efficiency and accuracy of all models in our dataset under various configurations. Users can select inference setups (c.g., GPU type or soft- ware library), evaluate accuracy on specific or combined test sets, and apply different scoring metrics all in real time. One of the primary aims of this tool is to promote
Therefore, propose second metric based on a weighted Manhattan distance to the ideal point (100% accuracy and 0
8

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
energy efficiency in machine learning by guiding the selec- tion of feature backbones for broader tasks such as image segmentation and detection. Models that excel on ImageNet critical appli-
track energy efficiency trends over time, foster transparency, and accelerate sustainable Al innovation.
This comprehensive study establishes a strong foundation in the underexplored field of energy-efficcin machine learn- ing and sustainable Al by this comprehensive study on the current trends, factors that influence energy consumption, rectified existing misconceptions, best practices for efficient model deployment, and provided various tools estimate energy consumption and tradeoff between accuracy. The data, results, and conclusions presented in our study are entirely novel, offering groundbreaking insights into the energy efficiency of deep learning models. We hope this work serves as a catalyst for the development of sustainable AI.
Adversarial or Rendition may suit cations, while those performing well on ImageNet Sketch could be ideal for abstract or low-resolution imagery. A screenshot of the webpage is shown in Fig. 11, illustrating how users can tailor parameters to their needs and instantly observe the impact on model rankings.
Through these complementary evaluation methods and our interactive tool, we hope to provide a balanced perspective on how to select architectures that meet both performance and sustainability goals.
5. Discussion
6. Conclusion
Our extensive benchmarking demonstrates that while state- of-the-art models achieve higher accuracy, they also incur substantial energy costs with diminishing returns. The log- arithmic rise in energy consumption for minimal accuracy gains raises sustainability concerns, both environmentally and economically.
In summary, our work provides a comprehensive analysis of the energy consumption of 1200 vision models, illuminat- ing the significant trade-offs between model accuracy and energy efficiency. By highlighting the diminishing returns in accuracy gains and introducing practical tools and met- rics, we hope to shift the focus towards more sustainable Al practices, Our findings lay the groundwork for further exploration into optimizing deep learning models for energy efficiency, encouraging a paradigm shift in how we evaluate and prioritize model performance.
Practitioners must balance accuracy, throughput, and energy consumption-tii the "iron triangle." As shown in Section 4.3, throughput and energy consumption are bounded by the GPU's TDP, emphasizing the need for efficiency-oriented model design. To aid in navigating these trade-offs, the interactive tool introduced in Section 4.4 allows users to visualize and balance key performance metries. promoting informed decision-making.
Acknowledgements
The authors would like to thank Karel Adámek for insightful and valuable discussions, which significantly influenced this work. WA would like to acknowledge support from the EPSRC Grant (EP/TO22205/I). The authors would like to acknowledge the use of the University of Oxford Advanced Research Computing (ARC) facility in carrying this work (Richards, 2015).
Our study significantly extends previous work by bench- marking 1,200 models on the latest A100 and H100 GPUs, offering a fair comparison and comprehensive insights. Al- though our experiments focus on these specific GPUs, the trends observed especially the high correlation of power consumption across different GPUs-suggest broader appli- cability. The comprehensive coverage of vision backbones would also help downstream tasks like object detection and semantic segmentation.
Impact Statement
The increasing energy consumption of deep learning mod- els, particularly in computer vision, presents critical eco- nomic and environmental challenges. This study provides the most comprehensive analysis to date of inference energy consumption across 1,200 ImageNet classification models, offering empirical insights and practical tools to promote more energy-efficient AI development.
The considerable energy costs tied to marginal accuracy gains highlight the need to adopt more sustainable Al prac- tices. By prioritizing efficiency and providing practical tools to measure energy consumption, we encourage the develop- ment and deployment of models that balance performance with environmental impact, aligning with the growing em- phasis on Sustainable AI (Van Wynsberghe, 2021).
Our findings reveal that improvements in model accuracy come with steeply diminishing returns in energy efficiency, raising concerns about the sustainability of current Al scal- ing trends. The introduction of energy efficiency scoring system and an interactive web tool empowers researchers and practitioners to make informed decisions, balancing
Looking ahead, we plan to update our dataset and inter- active web application as new models and GPUs emerge. We invite the community to use our open-source code to measure additional models and submit their findings. Such collaborative efforts will build comprchensive resource to
9

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
model performance with environmental impact.
J. Trends in ai inference energy consumption: Beyond the performance-wee parameeer parrraaateeeeeerrreeeeeeee laws of deep learning. Sustain- able Computing: Informatics and Systems, 38:100857, 2023.
From ethical standpoint, this work aligns with the grow- ing emphasis on Sustainable AI, encouraging shift towards more responsible AI development and deployment. By pro- viding transparency on the energy trade-offs associated with different architectures and inference methods, this study fos- ters awareness and accountability within the AI community.
Fang, Y., Wang, W., Xie, B.,, Sun, Q. Wu, L., Wang, X., Huang, T., Wang, X., and Cao, Y. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19358-19369 2023.
While our focus is vision backbones and possible down- stream tasks, the implications extend to broader Al applica- tions, including natural language processing and edge com- puting. We anticipate that this research will contribute to industry-wide efforts in reducing Al's carbon footprint and driving innovations in energy-efficient model design. Ulti- mately, we hope this work serves as a catalyst for greener AI solutions that balance computational advancements with long-term sustainability.
Fang, Y., Sun, Q. Wang, X., Huang, T., Wang, X., and Cao, Y. Eva-02: A visual representation for neon genesis. Image and Vision Computing, 149: 105171, 2024.
Google. Google environment report 2019. www.t:ttt/////to /wwww gstatic com/ httaa///aaaoocoo//////////////aa /wtp:///ssustsssiiiaiiii.../../.
google mhogle-20195 9 www.eeioeeeetallaeeepooooooo 2019.
References
Google. Google environment report 2024. https:ttt/////t /www.
Ansel, J., Yang, E., He, H., Gimelshein, N., Jain, A., Voz- nesensky, M., Bao, B., Bell, P., Berard, D., Burovski, E.,, Chauhan, G., Chourdia, A., Constable, W., Desmaison, A., Vitoo, Z., Ellison, E., Feng, W., Gong, J., Gschwind, M.. Hirsh, B., Huang. S., Kalambarkar, K., Kirsch, L., Lazos, M., Lezcano, M., Liang. Y., Liang, J., Lu, Y. Luk, C., Maher, B., Pan, Y., Puhrsch, C., Reso, M., Saroufim, M. Siraichi, M. Y., Suk, H., Suo, M., Tillet, P., Wang, E., Wang, X., Wen, W., Zhang, S., Zhao, X., Zhou, K., Zou, R., Mathews, A., Chanan, G., Wu, P., and Chin- tala, S. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation. In 29th ACM International Conference on Architectural Support Programming Languages and Operating Systems, Volume 2 (ASPLOS '4).. ACM, April 2024. doi: 10.1145/3620665 33440666. URL https: //pytorch. http:/chccrr.aasstssttttttttoaae
http://cwwwc./uuuooooo//uuaaaaaa googlee=2224-eev..ccommeata----- 2024.
Hashemi, S., Anthony, N., Tann, H., Bahar, R. I., and Reda, S. Understanding the impact of precision quantization on the accuracy and energy of neural networks. In Design, Automation Test in Europe Conference Exhibition (DATE), 2017, pp. 1474-1479, IEEE, 2017.
Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., and Pineau, Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248):1 -4,, 2020.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of out-of-distributt generalization. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8340-8349, 2021a
pdf.
Barroso, L. A. and Hölzle, U. The case for energy- proportional computing. Computer, 40(12))33-37, 2007.
Beyer, L., Hénaff, O. J., Kolesnikov, A., Zhai, X., and Oord, A. v. d. Are we done with imagenet? arXiv preprint arXiv:2006.07155, 2020.
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. -222111 2021b.
Canziani, A., Paszke, A., and Culurciello, E. An analysis of deep neural network models for practical applications. arXiv preprint ww05.v:605.0761 2016.
Jones, N. et al. How to stop data centres from gobbling up the world's electricity. Nature, 561(7722):663-166 2018
Contributors, 2023.
Pp-hgnetv2 model documenta- URL https://github. htom/Paaddeeaaddlee/eeoeoe develop/ docs/ / h
tion, October
Li, D., Chen, X., Becchi, M., and Zong, Z. Evaluating the energy efficiency of deep convolutional neural networks on cpus and gpus. In 2016 IEEE international confer- ences on big data and cloud computing (BDCloud), so- cial computing and networking (SocialCom), sustainable
PP HG-HNNetV2.c md.
Desislavoy, R., Plamee, F,, and Hernández-Orallo
10

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
computing and communications (SustainCom/(DDCouudd SpciiaCoom-SustaiiCCoo. pp. 477- IEEE, 2016.
Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize to imagenet? In Interna- tional conference machine learning, pp. 5389-5400. PMLR, 2019.
Lin, M., Chen, H., Sun, X., Qian, Q. Li, H., and Jin, R. Neu- ral architecture design for gpu-efficient networks. arXiv preprint arXiv:2006. 14090, 2020.
Richards, University of oxford advanced research com- puting, 2015. URL https: / /tt////o..i 5281/ zenodo. 22558.
Luccioni, S., Jernite, Y., and Strubell, E. Power hungry processing: Watts driving the cost of ai deployment? In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pp. 85-99, 2024.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bern- stein, M., al. Imagenet large scale visual recognition challenge. International journaal of computer vision, 115: 211 211-252, 2015.
Masanet, E., Shehabi, A., Lei, N., Smith, S., and Koomey, Recalibrating global data center energy-use estimates.
Science, 367(6481) (:98881))8449999 2020.
Samsi, S., Zhao, D., McDonald, J., Li, B., Michaleas, A., Jones, M., Bergeron, W., Kepner, J., Tiwari, D., and Gadepally, V. From words to watts: Benchmarking the energy costs of large language model inference. In 2023 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1-9. IEEE, 2023.
Meta.
Meta 2024 sustainability report.
https:
com/ hool://wccoooteer//pppoa///////2 Meta=2202-SSutttaaaa1ltt--ggggp 2024.
Schwartz, R., Dodge, J., Smith, N. A., and Etzioni, O. Green ai. Communicutions of the ACM, 63(12):54 63, 2020.
NVIDIA.
Nvidia a100 tensor
gpu archi-
tecture. https::/images.nn// nvidiaa com/ aem aem-
Sovrasov, ptflops: a flops counting tool for neural networks pytorch framework. attps://gither com/ wov.asvvvvvoooo/////// flops-
nvidia nvidia-ampere wtwpere-rrcccctttttteeeetttteeee
pdf, 2020.
Accessed: 2024-11-07.
2018-2024. URL https:://githh com/
NVIDIA.
Nvidia
h100
tensor core gpu
sovrasov flops-courte pytorch.
architecture.
nvidia. nvidia.com/entteeeee
Sudhakar, S., Sze, V., and Karaman, S. Data centers on wheels: emissions from computing onboard autonomous vehicles. IEEE Micro, 43(1):29-33 2022.
stc22-wwhitepaeer-hoperr 2023. Accessed: 2024-11-07.
Patterson, D., Gonzalez, J., Hölzle, U., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D. R., Texier, M., and Dean, J. The carbon footprint of machine learning training will plateau, then shrink. Computer, 58(7):18-28, 2022.
Tmamna, J., Ayed, E. B., Fourati, R., Gogate, M., Arslan, T., Hussain, A., and Ayed, M. Pruning deep neural networks for green energy-efficient models: survey, Cognitive Computation, (6(6):2931-2955 2024.
Van Wynsberghe, A. Sustainable ai: Ai for sustainability and the sustainability of ai. Al and Ethics, 1(3):213-218, 2021.
Peng, Z., Dong, L., Bao, H., Ye, Q., and Wei, F. Beit v2: Masked image modeling with vector quanti-q visual tokenizers. arXiv preprint urXiv:2208.0l 06366, 2022.
Wang, H., Ge, S., Lipton, Z., and Xing, E. P. Learning robust global representations by penalizing local predic- tive power. Advances in Neural Information Processing Systems, 32, 2019.
Percira, L. C., Guterres, B., Sbrissa, K., Mendes, A., Ver- meulen, F., Lain, L., Smith, M., Martinez, J., Drews, P., Duarte, N., al. The not-so-easy task of taking heavy-lift ml models the edge: A performance-war watt perspective. In Proceedings of the 38th ACM/SIGAPP Symposium Applied Computing, pp. 699-706, 2023.
Wightman, R. Pytorch image models, https: com/ hwtp://.ww....//ttt......... htw.:////eeeyyttrraomemmmemeeeoo image-modeeeeeeee 2019.
Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang. W., Banbury, C., Ye, C., Akin, B., et al. Mobilenetv4: Universal models for the mobile ecosystem. In European Conference on Computer Vision pp. 78-96. Springer, 2025.
Wu, C.-.,, Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Aga, F., Huang, J., Bai, C., et al. Sustainable ai: Environmental implications, chal- lenges and opportunities. Proceedings of Machine Learn ing and Systems, 4:795-8133 2022.

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
Xie, S., Girshick, R., Dollár, P., Tu, Z.,, and He, K. Aggre- gated residual transformations for deep neural networks. In Proceedings of the IEEE conference computer vi- sion and pattern recognition, pp. 1492-1500, 2017.
Yang, Z., Clark, A. B., Chappell, D., and Rojas, N. Instinc- tive real-time semg-based control of prosthetic hand with reduced data acquisition and embedded deep learning training. In 2022 International Conference on Robotics and Automation (ICRA), pp. 5666-5672. IEEE, 2022.
Yang, Z., Adamek, K., and Armour, W. Accurate and con- venient energy measurements for gpus: A detailed study of nvidia gpu's built-in power sensor. In SC24: Inter- national Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-17. IEEE, 2024.
Yao, C., Liu, W. Tang, W., Guo, J., Hu, S., Lu, Y., and Jiang, W. Evaluating and analyzing the energy efficiency of cnn inference on pigl-peeformance gpu. Concurrency and Computation: Practice and Experience, 33(6),60664 2021.
Yep, T. torchinfo.
com/ Sy/erYep/torchinffe 3 2020. URL https:// ttthub..comyyyllrr////ttochttttt
12

---

The Hidden Joules:
Evaluating the Energy Consumption of Vision Backbones
A. Appendix
A.I. Source Code
All the code used in this work are available at:
https:/ /ttps::/github...
A.2. Hardware and Software Configurations
A100 PCIe 40G
HI00 PCIe 80G
TDP
250W
310W
CPU
EPYC 7452
Xeon Gold 6342
RAM
ITB GIGABYTE
512GB
OEM
DELL
OS
CentOS 8.1.1911 525.116.00
CentOS 8.1.1911
GPU Driver
525. 16.04
CUDA vers.
11.8
11.8
PyTorch vers.
2.4
2.4
Table 2. Key hardware and software configurations.
A.3. Automated Measurement Procedure
Begin with Batchsize 1
(Clean-up Re)Load the model
TRT Optimization
Batchsize += 2
Warm-up Run
Run Inference with Measurements
No
Out of Memory?
Yes
Batchsize /= Step = Batchsize 4
Batchsize += Step
Clean-up & Reload the model
TRT Optimization
Warm-up Run
Run Inference with Measurements
No
Out of Memory?
Batchsize - Step
Step /=
Next model
Step 2
Figure 12. Automated model testing procedure.
13

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
A.4. Overall Result for the other Inference Setups
90
80
Accuracy (%)
70
60
A100 PyTorch FP32
MLP
50
CNN
Transformer
Hybrid
40
102
10a
10a
90
80
Accuracy (%)
70
60
A100 TensorRT FP16
MLP
50
CNN
Transformer
Hybrid
40
100
10a
102
10
104
90
80
Accuracy (%)
70
60
H100 PyTorch FP32
MLP
50
CNN
Transformer
Hybrid
40
101
1
103
104
Energy Consumption (mj/Image)
Figure 13. The overall scatter plot the other inference setups.
14

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
A.5. Fitted MLR Model for Energy Consumption Estimation
Since the all four of the Energy Consumption, Params, Gmacs and Activations exhibits uniformly in log-scale, the data follows power-law multiplicative distribution. Hence the Multiple Linear Regression model was done after taking the natural logarithm of both the depend and independent variables. The Log-Lineaa Regression Model has the form:
(nneryy Consumption) const A Alog(params) log(gmacs)
C' log (activatti)))
(3)
Energy Consumption = const x params
gmacs'
x activations e
(4)
The parameters for the 4 inference setups are as follows:
const
A
B
C
A100 PyTorch FP32
-14.1597
0.1426
0.1328
0.7687
A100
TensorRT FP16
-15.5186
0.1560
0.1622
0.7280
H100 PyTorch FP32
-14.0926
0.1348
0.0952
0.8103
H100 TensorRT FP16
-15.0298
0.1062
0.1 0.1141
0.7961
Table 3. Energy Estimation Model Parameters
A.6. Additional Justifications on Experiment Design Choices
Quantization and Pruning We acknowledge the ability of quantization and pruning to improve efficiency. The TensorRT inference setup used FP16. We excluded more detailed investigation because: Quantization and pruning architecture changes, but rather optimization techniques applied to any existing architectures. Our study focuses the inherent energy efficiency characteristics of different architectures themselves; 2 Often needs post-quantization fine-tuning or QAT, and similar for pruning. This process is infeasible for large-scale study like this; 3 There are some existing works already on this topic (Tmamna et al., 2024; Hashemi et al., 2017).
avidia-smi Query Frequency nvidia-smi updates the power draw reading at 10Hz to 50Hz. Querying nvidia-smi at 100 Hz was intentional, balancing the need to capture recent updates without unnecessary polling overhead. According to the Nyquist theorem, sampling above twice the signal frequency preserves information. A higher sampling frequency would do any harm.
A.7. Training & Year-on-year Improvement
90
2014
Accuracy (96)
70
2017
2018
2019
2020
2021
2022
2023
2024
40
10a
10a
102
10'
10*
Energy Consumption (mj/lmage)
Figure 14, Yearly progress on model accuracy and energy consumption. (A100 TensorRT FP16 as an example for illustration).
Training has significantly evolved over the past decade, involving larger datasets (LAION), self-supervised methods, and various training recipes. We categorized the models by the year their corresponding papers were published. Fig. 14 illustrates how each year's new models push the existing convex hull of models towards greater accuracy and efficiency. As the
15

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
field evolved and models introduced, the efficient frontier expanded in both the high accuracy, high energy consumption direction and the low accuracy, low energy consumption direction. Notably, observe consistent vertical increase in accuracy in the high consumption region and somewhat inconsistent horizontal shift towards lower consumption in the lower accuracy region. However, improvements in the middle region towards the top-left corner representing high accuracy and low consumption
to be more stagnant compared to others.
vit_base_patch 16_clip _244 trained with laion2b are 0.2% more accurate than the openai version, but with significantly more training data. While selecting a pretrained model for inference, one would pick the higher accuracy version; the question of whether consuming significantly more energy during training for marginal accuracy gains (e.g., 0.2%) is justified is beyond the scope of this work. This topic alone deserves future investigation by itself.
A.8. Robustness Analysis
Absolute and Relative Drop by Energy Percentile Bracket
-10
Drop in Accuracy
-2
-30
-10
Drop percentage points
Relative Drop compared to original (%)
10th
20th
30th
40th
50th
60th
70th
80th
90th
100th
Energy Percentile Bracket
Figure 15. Drop in accuracy by energy consumption percentile.
Absolute and Relative Drop by Architecture Type
Drop in Accuracy
-15
20
-25
-30
335
Absolute Drop in percentage paints
Drog compared origint
Overall
Hybrid
Transtomer
CHN
MLP
LSTM
CUP
Figure 16. Drop in accuracy by model architecture type.
We performed some additional analysis on the robustness of the models. We measured and calculated the absolute and relative accuracy decrease from the original ImageNet dataset with the average of the other 5 robustness datasets. We grouped models into 10 deciles by energy consumption, as shown in Fig. 15, and observed that nigher-energy-consumii models experience smaller accuracy drops, aligning with our findings presented in paper Fig. 3. Looking at different model architectures, as shown in Fig. 16, drop in accuracy for CNNs are 29.7%, hybrid models are 27.6%, and Transformers are 24.0%. We also observed that CLIP models only droped by 15%, demonstrating notably stronger out-o--diisributtin robustness.
16

---

The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones
A.9. Extension
Down-stream Tasks While our study directly evaluates image classification backbones, many popular detection and segmentation models (e.g., Faster R-CNN, Mask R-CNN, RetinaNet, DeepLab) commonly reuse these vision backbones as feature extractors. Given this standard practice, the energy efficiency characteristics observed for various architec- surs--suchae as CNN, Transformer, and Hybrid models serve as meaningful indicators for downstream task efficiency. For instance, backbones identified as energy-efficient at classification tasks could likely translate to efficient feature extraction in detection and segmentation pipelines, influencing overall inference efficiency. However, we do recognize the additional computational complexity and varying bottlenecks introduced by downstream modules (c.g., region proposal networks, mask heads, upsampling layers). Hence, explicitly measuring these downstream tasks remains important for precise validation. We plan extend methodology directly measure and confirm how the observed backbone energy characteristics generalize to these more complex tasks in future work.
Our current evaluation framework the importance and widespread use of large language models (LLMs) in various real-world applications, making their energy efficiency evaluation critically relevant to the sustainable Al community. Although our current study does not include LLMs explicitly, the core principles of our measurement methodology suchh as GPU utilization optimization, real-time energy monitoring, and securacy-performann To adapt our framework effectively to LLMs, we would primarily need to consider: Different inference characteristics: Token-based generation and longer context windows in LLM inference, compared fixed-size image inference in vision models; 2 Adaptation of evaluation metrics: Metrics such as perplexity, generation quality (c.g.. BLEU, ROUGE), task-specific evaluations (e.g., accuracy on reasoning benchmarks) instead of classification accuracy; 3 Adjustments in batching strategies: Optimal GPU utilization patterns for LLM inference, including considerations for sequence length and context size variability. In future research, we plan to explicitly extend our methodology to evaluate and analyze energy-accuracy trade-offs for large-scale language models, providing analogous insights that could significantly benefit the LLM research and deployment community.
LLMs
specifically tailored toward vision backbones. However, fully recognize
trade-off metrics are inherently transferable to large-scale LLM inference scenarios.
17


Reviews:
[{"review_id": "Wwp2UZ1z8G", "reviewer": "Reviewer_3dgS", "summary": "This paper conducts a comprehensive analysis of energy efficiency of image classification models, around 1200 ImageNet classifiers. The authors found that there is a steep diminishing return in accuracy gains compared to increase in energy use. They further identified key factors contributing to energy consumption of deep learning models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- Batched inference setting\n\nThank you for raising the important issue of batched inference. Our measurement methodology explicitly focuses on maximizing GPU utilization to ensure fair comparisons across models by testing a variety of different batch sizes and selecting the most energy-efficient inference settings. In fact, we deliberately critique Henderson et al. (2020), who ran all their experiments with batch size = 1 and concluded that FLOPs and Params have no correlation with energy. We will clarify and reinforce this point in the manuscript, ensuring readers understand our decision to measure energy per image at optimal GPU utilization.\n\n- CLIP models and robustness\n\nWe did not test any <multimodal> CLIP models, but we tested several models that were CLIP style trained and then fine-tuned for the ImageNet 1k dataset.   \n\nWe additionally evaluated three multimodal CLIP models (ViT-B-32, ViT-B-16, ViT-L-14) in a zero-shot scenario. Compared to their fine-tuned counterparts (text encoder removed, linear head added), their energy consumption per image remains largely similar (within 2%), while accuracy is much lower. \n\nWe also performed some further analysis on robustness: comparing original ImageNet validation accuracy with the average accuracy across five robustness datasets—reveals that CNNs drop 29.7%, hybrid models 27.6%, and Transformers 24.0%. The CLIP models only drop 15%, demonstrating notably stronger out-of-distribution robustness. (https://github.com/clinic-engulf-trowel-huihac/gurgling-theology-margarita/blob/main/rebuttal/robust2.png) Additionally, we grouped models into 10 deciles by energy consumption and observed that higher-energy-consuming models experience smaller accuracy drops, aligning with our findings presented in paper Fig 3. (https://github.com/clinic-engulf-trowel-huihac/gurgling-theology-margarita/blob/main/rebuttal/robust!.png)\n\n- More information on training, how does training affect energy and accuracy tradeoff, training has changed over the past 10 years\n\nWe agree that training has significantly evolved over the past decade, involving larger datasets (LAION), self-supervised methods, and various training recipes. We've analyzed publication dates of the evaluated models and found that while accuracy consistently improves year over year in high-accuracy regions, improvements in the high-efficiency region appear sporadic. Notably, the most efficient models tend to be older, dating back 2–3 years. (https://github.com/clinic-engulf-trowel-huihac/gurgling-theology-margarita/blob/main/rebuttal/yearly.pdf) We will provide more explicit information on model training in the appendix.\n\nvit_base_patch16_clip_224 trained with laion2b are 0.2% more accurate than the openai version, but with significantly more training data. While selecting a pretrained model for inference, one would pick the higher accuracy version; the question of whether consuming significantly more energy during training for marginal accuracy gains (e.g., 0.2%) is justified is beyond the scope of this work. This topic alone deserves a future investigation by itself.\n\n- Going forward, is there anything we can do to improve energy efficiency?\n\nBased on our findings, we suggest several practical guidelines: \n\nAvoid unnecessarily increasing input image sizes, as this significantly increases energy consumption with minimal accuracy improvements. \nFully utilize GPU capabilities through optimized batching. \nLeverage our interactive web tool to select and compare models effectively. \nUtilize our efficiency scoring system to evaluate models, potentially integrating it as an optimization loss function for neural architecture searches and designing new energy-efficient models. \n\nOverall, we laid a solid groundwork in the much-neglected field of energy-efficient machine learning and sustainable AI by this comprehensive study on the current trends, factors that influence energy consumption, rectified existing misconceptions, best practices for efficient model deployment, and provided various tools to estimate energy consumption and tradeoff between accuracy. The data, results, and conclusions presented in our study are entirely novel, offering groundbreaking insights into the energy efficiency of deep learning models. We hope this work serves as a catalyst for the development of sustainable AI.   \n\n- Could the authors provide a complete list of the models evaluated in the text?  \n\nThe complete list of evaluated models is currently available in our GitHub repo (https://github.com/clinic-engulf-trowel-huihac/gurgling-theology-margarita/blob/main/source/model_db.csv). We will include this list directly in the appendix for the reader's convenience.", "claims_and_evidence": "This paper suggest that there is diminishing return of accuracy when it comes to energy consumption. However, much of the conclusions are drawn from results on ImageNet accuracy. However, I am wondering if this is the correct metric to look at. When it comes to scaling deep learning models, model sizes and data both plays a crucial rule. But this paper seems to tell us that there is diminishing return. How does the authors reconcile this contradiction?", "methods_and_evaluation_criteria": "- The reported results in Figure 1 are focused on energy consumption per image. I wonder if a batched inference setting will affect the results. Would it be better to measure the energy consumption for running inference on a fixed set of images?  \n- How does the size of pre-training datasets play a role in determining the accuracy and energy efficiency tradeoff? I think the authors should provide more information on the pretraining data for the evaluated models.", "theoretical_claims": "This paper does not contain any theoretical proof.", "experimental_designs_or_analyses": "- Most of the analysis in this paper is focused on the accuracy and energy consumption tradeoff. I wonder if the authors have looked at the training methodologies. Over the last 10 years, there is a significant paradigm shift from supervised learning to self-supervised learning. I feel the authors could provide some analysis on characterizing the energy/accuracy tradeoff under different training schemes.   \n- CLIP models also achieve competitive ImageNet accuracy. Also, they mostly are better in out-of-distribution datasets. I wonder if the authors have evaluated CLIP models in the selected 1200 models.    \n- A major part of the analysis in section 4.1 is focused on characterizing architectures into three categories: Convnet, Transformers and Hybrid models. Could the authors provide some insights or suggestions on designing better model architectures in terms of energy efficiency and accuracy? From the provided results, it seems that Transformers are still the most promising architecture?", "relation_to_broader_scientific_literature": "This paper provides evidence on the energy vs accuracy tradeoff of ImageNet classifiers, which should be of interest to broader machine learning industry.", "essential_references_not_discussed": "This paper focuses on the inference energy efficiency of ImageNet classification models, which goes beyond the standard accuracy evaluation. It would be good to discuss this related work on efficiency evaluation of deep learning models [1].  \n\n[1] The Efficiency Misnomer. 2021.", "other_strengths_and_weaknesses": "Regarding the significance, I think the idea of evaluating energy consumption of deep learning models is a realistic problem to look at. This is an important problem given that AI is growing every year and would cost more electricity to run these models. \n\nI would like the authors to provide some discussions on the implications of the results for AI development. Like going forward, is there anything we can do to improve energy efficiency of these models?", "other_comments_or_suggestions": "Could the authors provide a complete list of the models evaluated in text? I want to check if the authors have included certain models but couldn't do it because no information is provided. It is much harder to check by going through each points in the provided website.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iLAohewPfA", "reviewer": "Reviewer_wUku", "summary": "This paper evaluates the energy consumption of ImageNet classification models, focusing on their efficiency across different architectures, datasets, and optimization techniques. The study aims to provide a more accurate assessment of energy consumption in deep learning models and examines accuracy gains relative to increased energy consumption. The findings emphasize the importance of optimizing GPU utilization and inference configurations for energy efficiency. Finally, this work provides an interactive web application to compare models based on accuracy and energy consumption.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- How is this work different?\n\nIn addition to evaluating a substantially larger number of models (50–100x more), our work differs from prior studies in several ways:\n\n1-We identified key methodological flaws from previous studies: Henderson et al. (2020) using batch size = 1, and Desislavov et al. (2023) relying on TDP-based energy estimates. Our careful methodology ensures accurate and realistic conclusions.\n\n2-Previous studies have lacked clarity regarding the specifics of deployment. Our research showed that running models in PyTorch can be up to an order of magnitude more energy consuming compared to TensorRT. We conducted our measurements under an industry-standard inference deployment scenario, thereby enhancing the relevance of our results for real-world applications and practitioners.\n\n3-The models and GPUs prior work evaluated are outdated, particularly in light of rapid advancements in the field. By expanding experiments using the RTX 4090 and 5090, and models published until 2024, our study stands as the most comprehensive study to date. Furthermore, our framework, publicly available on GitHub, is designed for easily evaluating new models or GPUs. We encourage the community to contribute by utilizing our framework to assess their models and GPUs and contribute to our GitHub repo and web app.\n\nWe will clearly articulate these in our revised manuscript.\n\n- Energy-proportional computing and Sustainable AI\n\nWhile we briefly discussed sustainable AI at the beginning of the related works, we agree with your suggestions. We will expand the discussion section of our paper along these lines:\n\nOur study empirically confirms that current deep learning models are significantly less energy-proportional than idealized hardware expectations suggest, aligning with insights from Barroso and Hölzle (2007). The steep diminishing returns we identify reinforce the need for more energy-proportional model architectures and inference methods.\n\nOur findings directly support the goals outlined by Schwartz et al. (2020), advocating for Green AI principles that prioritize energy efficiency alongside accuracy. By demonstrating the marginal gains in accuracy versus exponential energy demands, our work underscores the urgency emphasized by the Sustainable AI movement (Van Wynsberghe, 2021) to embed sustainability criteria into AI model evaluation and selection.\n\nUnlike prior studies that relied on theoretical calculations of FLOPs or idealized TDP-based estimations (Henderson et al., 2020; Desislavov et al., 2023), our real-world measurements provide realistic insights into the practical energy consumption patterns of models, bridging a critical gap in sustainable AI practices.\n\n- How the evaluation results generalize for real-life workloads\n\nWe agree that evaluating beyond standard ImageNet accuracy is essential for understanding model performance in more diverse, realistic scenarios.\n\nTo directly address this, our study included evaluations on five additional datasets specifically designed to assess robustness and out-of-distribution generalization. These datasets intentionally differ from standard ImageNet conditions, capturing real-world challenges such as distribution shifts and adversarial conditions. Please refer to the answer to reviewer 3dgS for more robustness analysis.\n\nWhile we recognize practical deployment evaluations (such as robotic applications) are valuable, they exceed the practical scope of our current study. We believe our extensive robustness analysis already significantly extends practical insights beyond standard benchmarks.\n\n- More variety of hardware: Please see answer to Reviewer b1xy.\n- Takeaways for the paper: Please see answer to Reviewer 3dgS.\n\n- Quantization and pruning\n\nWe acknowledge the ability of quantization and pruning to improve efficiency. The TensorRT inference setup used FP16. We excluded a more detailed investigation because:\n\n1-Quantization and pruning are not architecture changes, but rather optimization techniques applied to any existing architectures. Our study focuses on the inherent energy efficiency characteristics of different architectures themselves.\n\n2-Often needs post-quantization fine-tuning or QAT, and similar for pruning. This process is infeasible for a large-scale study like this.\n\n3-There are some existing works already. To name a few:\n\nUnderstanding the impact of precision quantization on the accuracy and energy of neural networks\n\nPruning Deep Neural Networks for Green Energy-Efficient Models: A Survey\n\nWe will clarify this scope in the manuscript and cite these works.\n\n- nvidia-smi measurement\n\nYour point about measurement frequency is valid. Querying nvidia-smi at 100 Hz was intentional, balancing the need to capture recent updates without unnecessary polling overhead. According to the Nyquist theorem, sampling above twice the signal frequency preserves all information. A higher sampling frequency would not do any harm. We will clarify the revised manuscript.", "claims_and_evidence": "Yes, claims and insights made in this paper are drawn from analysis and empirical results presented in the paper.", "methods_and_evaluation_criteria": "Yes, the propsed method and evaluation criterai, including evaluated models, datasets and the way of measuring energy consumption makes sense", "theoretical_claims": "The paper does not include any proof or theoretical claims", "experimental_designs_or_analyses": "Yes, the experiments appear to be sound", "relation_to_broader_scientific_literature": "The paper builds on prior work in energy-efficient deep learning by extending the analysis beyond FLOPs and latency to actual energy consumption. It aligns with studies on hardware-aware model optimization and inference efficiency but differentiates itself by incorporating real-world GPU utilization metrics. However, it could better contextualize its findings with prior work on energy-proportional computing and sustainable AI practices to strengthen its contributions.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strength:\n- The paper addresses an important and relevant problem for the ML community. Energy efficiency is an increasing important issue as the model grows in size and focusing on energy compared to traditional metrics like FLOPS of latency not only proposes optimizations for model efficiency but also contributed to sustainable AI\n- The study evaluates a large set of models on relevant datasets, and optimization techniques.\n- The analysis provides valuable insights on the trade-offs between model architecture, size and batch sizes on accuracy and energy consumption. \n- The work proposes a web application which can be a useful resource for the community. \n\nWeaknesses\n- The comparison to prior works is limited. It would be beneficial to more clearly discuss the difference between this work and prior works in the area. How is this work different from prior work except for the larger number of models evaluated? \n- The study primarily uses ImageNet, only one study uses 5 different datasets. When evaluating a set of different datasets, we observe a shift in the observed results. I wonder how the evaluation results generalize for real-life workloads. \n- The evaluation is limited to just two types of GPUS: A100 and H100. Although the evaluation is mainly focused on model architectures, to evaluate trends and impact of energy efficiency, it would be beneficial to include more variety of hardware (potentially also AMD GPUs) to see if the observed trends generalize to a larger set of hardware. \n- It is interesting to see the large number of experiments and data collected in the direction of energy consumption of models. However, it is unclear of what the key insights or takeaways are for the paper. and how it can help advance sustainable designs of ML models.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Does the observed trends in the analysis generalize beyond the two GPU types and the ImageNet dataset?\n2. How does quantization impact energy consumption across different models?\n3. The paper discusses the impact of model size (number of parameters) such as in Figure 5. But it seems to consider across multiple different model architecture. Does the analysis also compare models of the same or similar architecture? Have the authors considered the impact of model size in terms of pruning, and how does that affect energy consumption? Perhaps pruning could be used to optimize energy efficiency.\n4. To my understanding, the power measurement period for NVIDIA-SMI's power sensor typically operates at a frequency of 10 Hz, meaning it updates every 100 milliseconds. Section 3 of the paper states \"We recorded power usage and other GPU metrics at 100 Hz, then logged the data for subsequent analysis.\" Is there a possibility of getting repeated values since the work queries at a higher frequency than measured, and does that affect the interpretation of the results in any way?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YwRhaB4YUL", "reviewer": "Reviewer_b1xy", "summary": "This paper introduces an energy efficiency scoring system and develop the corresponding interactive web application for users to compare models based on accuracy and energy consumption.\n\nThe experimental results show that the proposed scoring system is kind of accurate when estimating the energy consumption from the throughput and TDP aspects of GPU.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- Reference: Carbon Emissions and Large Neural Network Training\n\nWe appreciate your suggestion to cite this important work. We actually referenced the updated version, \"The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink,\" which significantly inspired our research.\n\n- More variety of hardware and edge devices\n\nBattery-powered devices are indeed sensitive to energy efficiency. While A100 and H100 are the most representative GPUs in the AI industry, we agree and recognize the importance of evaluating diverse hardware. To this end, we recently tested approximately 100 models near the efficient frontier on four additional GPUs: RTX 3090, 4090, 5090, and mobile 1650Ti. Results (https://github.com/clinic-engulf-trowel-huihac/gurgling-theology-margarita/blob/main/rebuttal/new_gpus.png) strongly confirm our original trends, further validating the broad applicability of our findings.\n\nWe also observed that 5090 did not provide any significant efficiency gains over 4090, further highlighting the importance of our research (can’t just wait for new GPUs and expect them to be more efficient)\n\n- Down-stream tasks\n\nWe fully acknowledge the importance of evaluating downstream tasks like object detection and semantic segmentation for a comprehensive assessment of energy efficiency.\n\nWhile our study directly evaluates image classification backbones, many popular detection and segmentation models (e.g., Faster R-CNN, Mask R-CNN, RetinaNet, DeepLab) commonly reuse these vision backbones as feature extractors. Given this standard practice, the energy efficiency characteristics we observed for various architectures—such as CNN, Transformer, and Hybrid models—serve as meaningful indicators for downstream task efficiency. For instance, backbones identified as energy-efficient at classification tasks could likely translate to efficient feature extraction in detection and segmentation pipelines, influencing overall inference efficiency.\n\nHowever, we do recognize the additional computational complexity and varying bottlenecks introduced by downstream modules (e.g., region proposal networks, mask heads, upsampling layers). Hence, explicitly measuring these downstream tasks remains important for precise validation. We plan to extend our methodology to directly measure and confirm how the observed backbone energy characteristics generalize to these more complex tasks in future work.\n\nIn the revised manuscript, we will more explicitly highlight this link between vision backbones and downstream models, clarifying both the potential insights our current analysis provides and the scope for further empirical validation. \n\n- Can the framework be used for the LLMs?\n\nThank you for raising this important consideration. Our current evaluation framework was specifically tailored toward vision backbones.\n\nHowever, we fully recognize the importance and widespread use of large language models (LLMs) in various real-world applications, making their energy efficiency evaluation critically relevant to the sustainable AI community. Although our current study does not include LLMs explicitly, the core principles of our measurement methodology—such as GPU utilization optimization, real-time energy monitoring, and accuracy-performance trade-off metrics—are inherently transferable to large-scale LLM inference scenarios.\n\nTo adapt our framework effectively to LLMs, we would primarily need to consider:\n1. Different inference characteristics: Token-based generation and longer context windows in LLM inference, compared to fixed-size image inference in vision models. 2. Adaptation of evaluation metrics: Metrics such as perplexity, generation quality (e.g., BLEU, ROUGE), or task-specific evaluations (e.g., accuracy on reasoning benchmarks) instead of classification accuracy. 3. Adjustments in batching strategies: Optimal GPU utilization patterns for LLM inference, including considerations for sequence length and context size variability.\n\nIn future research, we plan to explicitly extend our methodology to evaluate and analyze energy-accuracy trade-offs for large-scale language models, providing analogous insights that could significantly benefit the LLM research and deployment community.\n\nWe will clearly articulate this limitation and the intended future extension to LLMs in our manuscript, emphasizing the broader applicability of our proposed framework toward sustainable AI practices beyond vision tasks.\n\n- Efficiency Score and Practical Applications\n\nWe appreciate your recognition of our proposed efficiency scoring system’s practical value. We will explicitly reinforce in the manuscript how this system facilitates immediate, actionable insights for hardware-oriented researchers and deployments. We will explicitly highlight how our efficiency scores can guide practical decisions, such as serving as an optimization objective in Neural Architecture Search, and influence future architecture design, directly promoting sustainable AI practices.", "claims_and_evidence": "claims with clear evidence:\n\n1. energy consumption can not be evaluated by the FLOPs alone: the empirical data shows that memory access, activation size, and hardware optimizations can cause the large difference between the energy consumption on the platform.\n\n2. trade-offs between accuracy and efficiency: the work shows multiple results for this.\n\n3. effectiveness of the proposed energy efficiency scoring system: the energy computed according to throughput and GPU thermal design power shows strong correlation with the actual energy consumption.", "methods_and_evaluation_criteria": "The proposed energy efficiency scoring system is quite useful for the hardware deployment related research area, as this work provides a simple and effective framework for researchers to quickly evaluate the power consumption of their designed models.", "theoretical_claims": "This paper does not contain formal theoretical proofs.", "experimental_designs_or_analyses": "This work mainly focuses on the experimental results with the ImageNet and GPUs (including H100 and A100 mainly).\n\nAlso, this work takes the FLOPs, activations, and throughput into consideration for the energy assessment, which shows more accurate compared to other works which only adopts the FLOPs as the major one.", "relation_to_broader_scientific_literature": "The method proposed by this work could be adopted for the normal hardware oriented design works for quickly evaluate the power efficiency of the designed model, which is convenient.", "essential_references_not_discussed": "This work may need to discuss the works that related to the large models like [1].\n\n[1] Carbon Emissions and Large Neural Network Training", "other_strengths_and_weaknesses": "Strength:\n\n1. the energy efficiency scoring system shows the effectiveness of evaluating the energy efficiency for vision models on ImageNet.\n\nWeakness:\n\n1. This work did not include the energy efficiency for the down-streaming tasks like object detection and semantic segmentation tasks.\n\n2. This work only includes the platform for GPUs (H100 & A100), while the results for edge devices, like FPGA, NVIDIA Jetson, and Mobile devices, are more important. because especially those battery-powered devices are more sensitive to the energy efficiency.", "other_comments_or_suggestions": "1. provide more results for the down-streaming tasks like object detection and semantic segmentation tasks.\n\n2. provide more results for edge devices.", "questions_for_authors": "This work did not take the large language models (LLMs) into consideration, can the framework used for the large-scale LLMs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper evaluates 1200 ImageNet vision models on energy consumption, introduces a scoring system and tools, with insights on factors affecting the energy cost. The review ratings are all positive (3 weak accepts). Reviewers recognize the importance of the problem studied, the comprehensive evaluation, and the practical scoring system. The AC recommends acceptance.
Decision: Accept (poster)