NExUME: Adaptive Training and Inference for DNNs under Intermittent Power Environments

Authors: Cyan Subhra Mishra, Deeksha Chaudhary, Jack Sampson, Mahmut Kandemir, Chita R. Das
Primary area: infrastructure, software libraries, hardware, systems, etc.
Keywords: Intermittent Computing, Energy Harvesting, Intermittency Aware Training, Hardware-Software codesign

Abstract:
The deployment of Deep Neural Networks (DNNs) in energy-constrained environments, such as Energy Harvesting Wireless Sensor Networks (EH-WSNs), introduces significant challenges due to the intermittent nature of power availability. This study introduces NExUME, a novel training methodology designed specifically for DNNs operating under such constraints. We propose a dynamic adjustment of training parameters—dropout rates and quantization levels—that adapt in real-time to the available energy, which varies in energy harvesting scenarios.

This approach utilizes a model that integrates the characteristics of the network architecture and the specific energy harvesting profile. It dynamically adjusts training strategies, such as the intensity and timing of dropout and quantization, based on predictions of energy availability. This method not only conserves energy but also enhances the network’s adaptability, ensuring robust learning and inference capabilities even under stringent power constraints. Our results show a 6% to 22% improvement in accuracy over current methods, with an increase of less than 5% in computational overhead. This paper details the development of the adaptive training framework, describes the integration of energy profiles with dropout and quantization adjustments, and presents a comprehensive evaluation using real-world data. Additionally, we introduce a novel dataset aimed at furthering the application of energy harvesting in computational settings.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730287864106

thread_type:
official_review

<thread_object>
creator: reviewer_tanu

<content>
summary:
This paper presents NExUME, a training methodology that is designed to cater to intermittent power energy harvesting systems.   The authors proposed two key contributions which are (1) a new method for training where one can dynamically adjust dropout rate and quantization levels to cater to varying energy availability in the EH system, and (2) a task scheduler that optimizes task completion in EH systems. The authors also contribute a machine status monitoring dataset. NExUME shows a 6 to 22 % accuracy improvement over existing baselines on simple ML tasks. However, at the same time, NExUME incurs a 5% overhead in computation, an increase in the number of instructions ranging between  11.4%-  34.2%, and an increase in memory bandwidth from 6 to 17%.

strengths:
+ This paper is the first to present novel training and inference methods that take dropouts and quantization into account in the context of energy harvesting systems. 
+ A new machine monitoring dataset 
+ The results shown are good when compared to the baselines presented in the paper.
+ Writing and the presentation of the work are clear. 
+ Choice of datasets is appropriate given that the work is designed for resource-constrained embedded systems. 
+ Decent ablation studies 
+ Actual implementation of such a system is not trivial.

weaknesses:
- The idea of dynamically adjusting dropout rates and quantization levels is not novel. It is novel in the context of EH systems. 
- Energy-aware scheduling is not novel. 
- The quantification of overheads is done. However, its implications are not discussed. The range in terms of % is indicated. However, how does it vary with the datasets? 
- Some of the existing work in intermittent systems are not compared such as ePerceptive: energy reactive embedded intelligence for batteryless sensors and Zygarde: Time-Sensitive On-Device Deep Inference and Adaptation on Intermittently-Powered Systems
- Details on the machine status monitoring dataset are missing in Sec 4.3 How are R1, R2, and R3 different? What are their RPM speeds? What is S1 and S2? 
- Only accuracy results are shown.  It is also important to know the latency/inference. and memory requirements of the system. 
- DynFit comprises adjusting quantization levels and dropouts. In the ablation studies, it is unclear which of these is bringing more benefits to the system.

questions:
Details on the machine status monitoring dataset are missing in Sec 4.3 How are R1, R2, and R3 different? What are the RPM speeds? What is S1 and S2?

Energy-aware scheduling is not novel. Can you clarify your novelty with respect to existing scheduling algorithms? 

The authors claim that their machine status monitoring is the first of its kind. Can they clarify what datasets already exist and how the dataset introduced in the paper is different?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730579261452

thread_type:
official_review

<thread_object>
creator: reviewer_gup2

<content>
summary:
The paper presents NExUME, a novel framework for training and deploying deep neural networks (DNNs) on energy-harvesting micro-computers with intermittent power. The authors introduce dynamic dropout rates and quantization levels that adapt based on real-time energy availability, improving the accuracy and robustness of DNNs in constrained power settings. The paper demonstrates NExUME's efficacy in optimizing both training and inference phases through extensive experiments, showcasing significant accuracy gains over traditional approaches in intermittently powered environments. Additionally, the introduction of a unique dataset to facilitate further research on energy-harvesting applications is a noteworthy contribution.

strengths:
1. The paper addresses an important challenge in deploying DNNs on resource-constrained, intermittently powered devices, an area that is underexplored in current literature. By incorporating real-time energy-aware adaptations, this work proposes a unique and valuable solution.
2. The work is thorough, presenting a well-structured methodology, clearly defined optimization functions, and a series of experiments across various datasets and hardware platforms. The choice of energy-aware dropout and quantization strategies tailored to intermittent environments is both innovative and well-validated.
3. The paper is well-written, with each component of the proposed framework (DynFit, DynInfer) and the optimization strategies clearly explained. Figures and tables effectively support the results and comparisons.
4. The proposed approach has broad implications for real-world applications in energy-limited environments, such as remote monitoring and IoT systems, where consistent power is unavailable. The improvements in accuracy (6-22%) and the novel dataset enhance the significance and impact of the research.

weaknesses:
1. The experiments, while comprehensive, rely on specific hardware configurations that may not be accessible for replication. The reliance on components like MSP430FR5994 and certain energy-harvesting setups may limit reproducibility.
2. While the paper compares NExUME to iNAS and other energy-aware methods, it lacks a detailed comparison with additional state-of-the-art adaptive or intermittent DNN training techniques. Including broader comparisons could enhance the validation of its claims.
3. The paper mentions challenges with larger networks and datasets. However, there is limited discussion on potential approaches to address these limitations, which would be valuable for practitioners aiming to scale this approach.
4. The profiling is based on conservative estimates, which, while practical, may not be universally applicable. Further analysis of the impact of profiling variations on model performance could strengthen the evaluation.

questions:
1. Can the authors clarify the robustness of NExUME across various hardware platforms beyond those tested? Would modifications be required for different types of microcontrollers or energy-harvesting setups?
2. How does NExUME handle environments with extremely low or sporadic energy levels, where consistent dropout and quantization adjustments may not be feasible?
3. Can the authors provide more detail on the potential effects of overfitting introduced by DynFit’s dropout variations? Would techniques like dropout scheduling help mitigate this?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730593420332

thread_type:
official_review

<thread_object>
creator: reviewer_qdus

<content>
summary:
The paper introduces a framework designed to enable consistent and accurate deep neural network inference on energy-harvesting wireless sensor networks that operate under intermittent power conditions. This framework addresses the challenges of unreliable energy supply and computational limitations in such environments. The proposed framework uses energy variability aware network architecture search, dynamic training optimizations, and an intermittency-aware task scheduler to adapt DNN computations based on real-time energy availability, in order to meet service level objectives (SLOs) in resource-constrained settings.

strengths:
The paper studies an interesting and important problem of enabling reliable DNN inference in energy-harvesting wireless sensor networks. The writing is overall clear and well-organized. The motivations, methods, and experimental findings are easy to follow.

weaknesses:
The proposed method relies on detailed profiling of the hardware to model energy consumption, computational capabilities, and memory footprint. This process can be time-consuming and complex, requiring extensive micro-profiling. 

In DynInfer, an energy-aware priority scheduling heuristic is used. With no theoretical analysis of its performance compared to optimal scheduling solution, its scheduling optimality is hard to estimate.

The explanations of some techniques in the methods section, particularly within the DynFit and DynInfer components, remain at a high level, lacking depth in technical specifics. For example, while the dynamic dropout and quantization strategies in DynFit are introduced, there is limited detail on how dropout rates and quantization levels are adjusted based on energy profiles or how these adjustments differ from standard implementations. Additionally, the methods used in each component lack a sense of innovation, as they seem to be a simple use of existing techniques without substantial enhancements. 

The impact of under-trained and overfitting weights requires further examination. More frequent updates of certain weights do not necessarily lead to "overfitting," and, conversely, infrequent updates do not inherently imply "underfitting." From a layer perspective, the effect of varying update frequencies on individual weights may be limited, suggesting that this issue may be less impactful than indicated. 

The experiments mainly focus on accuracy improvement. Other performance metrics, such as energy consumption, latency, computational overhead, and the number of power failures or SLO violations, are not extensively analysed. 

The experiments are conducted on relatively small datasets and models.

questions:
1. How much of the resource of the method is used or what is its time complexity?

2. As a sub-optimal scheduling solution, how would its scheduling performance to be ensured?

3. What is the rationale behind the overall method design?

4. Whether the accuracy is more important than the other performance metrics in your design?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730660327960

thread_type:
official_review

<thread_object>
creator: reviewer_p7bk

<content>
summary:
The paper introduces NExUME, a framework addressing issues with DNN training for energy-constrained environments which can't guarantee a sufficient amount of power at all times, such as Energy Harvesting Wireless Sensor Networks. In order to optimise DNN-Training for these unstable conditions, NExUME relies on an estimation of the available resources. For this, a first-of-its-kind dataset containing energy harvesting traces and available computation libraries is introduced.
The training process reduces intermittency-related failures by treating the number of loop iterations as learnable parameters and task-fusions to meet energy budgets. Additionally, dynamic dropouts during execution ensure the completion of layers and dynamic quantization balances out the accuracy degradation. An adaptive regularization strategy prevents weights from being undertrained. Lastly, the authors introduce a task-scheduler that adjusts in real-time to the energy conditions estimated.

strengths:
- Embedding energy variability into the training process is a novel idea.
- Extends existing Neural Architecture Search for intermittent computing systems
- Evaluation on SOTA datasets and DNNs (in the context of intermittent computing)

weaknesses:
- Evaluation on SOTA datasets and DNNs (in the context of intermittent computing). While also a strength, it also raises a questions. The ML community has moved on to Attention and Transformers and large scale datasets. This paper does not discuss how such modern architectures can be deployed in the intermittent setting. 

- contribution over SOTA remains unclear. The paper cites numerous NAS frameworks for intermittent/MCU computing (and there are more like [1]) and a large body of work on intermittent execution of DNNs such as [2, 3] and numerous papers cited in the introduction. To me, the contribution over these remains unclear, as many aspects are also in these papers. 

- lack of SOTA baselines: The paper should compare to SOTA approaches. 

- statement such as "Since we are the first work to propose a new training approach targeted for intermittent devices and inference optimizations" should be toned down. Instead, please carefully explain your contributions over SOTA and compare them to SOTA baseline. 

- ablation study: accuracy and overhead if full power is available 

- Overall: this paper is better suited at a system conference, such as SenSys or MobiSys

- the title is misleading, the paper is about more than DNN training. 

- BLE board does not have FeRAM and thereby not a classic board for intermittent computing. Why do the authors choose it? How does the intermittent part work here, especially QuantaTask?


-  On several occurrences, the paper is written (too) vaguely. E.g. 
      - There is no hint as to how tasks are "fused" when executing multiple quanta would exceed the energy budget. To my understanding, the function in only explained in the appendix as part of the source code but not the text.
      -  Dropout rates are adjusted on "specific" criteria. Even though the appendix provides details, this vague style of writing reads weirdly and is better served with examples

- As mentioned on several occasions, a big part of the presented work is the availability of the database of DynAgent which also contains hardware-information, yet only 2 different microcontrollers are used for evaluating the framework. Testing a broader variety of systems seems sensible here

- Drawbacks like up to 34% increased instruction count and up to 17% increased memory bandwidth usage are stated but hardly discussed or put into perspective. While this is not surprising for intermittent computing, the numbers should still be discussed and be compared to other approaches.

- Typos: Figure 3: Sensitivity and ablation study. DN is DynNAS, DF is FynFit, and DI is DynInfer: FynFit -> DynFit

- with a Pixel-5 phone as the host device: does this matter? Any device should do the job.

- the paper has a section LIMITATIONS AND DISCUSSION which also discussed limitations, such as the runtime overhead. The last two sentences, however, read a bit bumpy and should be streamlined and also discussed (and not just stated). 


[1] Edgar Liberis, Łukasz Dudziak, and Nicholas D. Lane. 2021. ΜNAS: Constrained Neural Architecture Search for Microcontrollers. In Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys '21). Association for Computing Machinery, New York, NY, USA, 70–79. https://doi.org/10.1145/3437984.3458836

[2] Chih-Hsuan Yen, Hashan Roshantha Mendis, Tei-Wei Kuo, and Pi-Cheng Hsiu. 2023. Keep in Balance: Runtime-reconfigurable Intermittent Deep Inference. ACM Trans. Embed. Comput. Syst. 22, 5s, Article 124 (October 2023), 25 pages. https://doi.org/10.1145/3607918

[3] C. -H. Yen, H. R. Mendis, T. -W. Kuo and P. -C. Hsiu, "Stateful Neural Networks for Intermittent Systems," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 11, pp. 4229-4240, Nov. 2022, doi: 10.1109/TCAD.2022.3197513

questions:
* the BLE board does not have FeRAM and thereby not a classic board for intermittent computing. Why do the authors choose it? How does the intermittent part work here, especially QuantaTask?

* what are the contributions over SOTA?

* what is the performance compared SOTA baselines?

* can you consider a different title?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733241101328

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank you Everyone and Our Final Remark

comment:
We sincerely thank all the reviewers for their thoughtful, constructive and encouraging feedback, which has greatly helped us improve our work. We thanks the PC members and area chairs for their dedicated efforts. We have carefully considered all comments and made revisions to the manuscript, which the reviewers have graciously accepted. Below, we summarize the key points:

**1. Our Novelty**
- **Integration of Energy Variability into Training and Inference**: We introduce NExUME, a novel framework that uniquely integrates energy variability awareness directly into both the training (DynFit) and inference (DynInfer) processes. Unlike existing methods, our approach enables DNNs to adapt dynamically to real-time energy conditions in energy-harvesting (EH) environments.
- **Adaptive Regularization Strategy**: We introduce an adaptive regularization technique to prevent underfitting and overfitting caused by uneven weight updates due to dynamic dropout, ensuring model robustness in EH settings.
- **Novel Task Fusion Mechanism**: Our scheduler includes a novel task fusion mechanism that combines smaller tasks into larger atomic units, optimizing execution under intermittent power and minimizing checkpointing overhead—a challenge unique to EH systems.

**2. Additional Evaluation**
- **Expanded Hardware Platforms**: We have extended our experiments to include additional microcontrollers such as ESP32 S3 Eye, STM32H7, and Raspberry Pi Pico. This demonstrates NExUME's applicability and robustness across various hardware configurations.
- **Comparison with SOTA Methods**: We have included comparisons with recent SOTA methods, including Keep in Balance, Stateful Neural Networks, ePerceptive, and DynBal. Our results show that NExUME consistently outperforms these methods in terms of accuracy and energy efficiency across multiple datasets and platforms.
- **Detailed Dataset Information**: We have provided more details about the machine status monitoring dataset, clarifying the experimental setup and the significance of each class in the dataset. This dataset is the first of its kind to capture machine status monitoring data across multiple operating conditions using EH sensors.
- **Comprehensive Metrics**: In addition to accuracy, we have included evaluations on energy efficiency (MOps/Joule), overheads during training and inferece, and memory requirements. We have also conducted an ablation study.
- **Overhead Analysis**: We have provided a detailed discussion of the computational overhead introduced by our methods, noting that while there is an increase in instruction count and memory bandwidth usage, the trade-off results in significant improvements in accuracy and energy efficiency, which is acceptable given the constraints of intermittent computing.

**3. Why ICLR is the Right Venue**
- **Interdisciplinary Contribution**: Our work lies at the intersection of machine learning and systems engineering, addressing challenges that require interdisciplinary solutions. We believe that the ICLR community values such contributions that push the boundaries of ML deployment in real-world environments.
- **Promoting Sustainable and Green Computing**: As the ML community continues to develop large-scale models, there is a growing emphasis on sustainability and reducing the environmental impact of computing. Our research contributes to this goal by enabling intelligent computations on devices powered by renewable energy sources, which aligns with the interests of the ICLR community.
- **Encouraging Co-Design of Algorithms and Hardware**: By co-designing algorithms, models, and hardware, we can achieve more efficient and effective solutions for highly dynamic and constrained systems. This philosophy aligns with the broader ML community's interest in holistic approaches to problem-solving.
- **Relevance to ML Deployment and Practical Applications**: Our work introduces novel machine learning methodologies tailored for emerging hardware platforms. We anticipate that presenting our research at ICLR will inspire others in the community to explore and develop algorithms that are not only theoretically sound but also practically deployable on next-generation, sustainable hardware.
- **Stimulating Further Research**: We believe that our work will encourage the ML community to build better algorithms, models, and hardware that are optimized for deployment in energy-harvesting environments, contributing to the advancement of sustainable, intelligent systems.
---
We appreciate the reviewers' recognition of our efforts and their acknowledgment of the contributions our work makes to the field. We believe that our paper offers valuable insights and advancements that are highly relevant to the ICLR audience. We look forward to the opportunity to share our work with the community and take the research on intermittent/sustainable AI forward.
</content>
</thread_object>

</thread>


Metareview:
metareview:
**Summary:** The paper proposes NExUME, a framework for training DNNs in energy-harvesting environments with intermittent power. It integrates real-time energy variability into DNN training via dynamic dropout rates, quantization adjustments, and a task scheduler to optimize computations under energy constraints. Results show 6–22% accuracy improvement over baselines with modest computational overhead.


**Strength:** 
1. The integration of energy variability into the DNN training process, enhancing adaptability in energy-harvesting systems, is novel.


2. The proposed framework achieves significant accuracy improvements under specific energy constraints, as validated across multiple datasets and hardware platforms.


3. In addition to the framework, this work also introduces a first-of-its-kind machine monitoring dataset for research in intermittent computing.


**Weakness:**
1. The techniques like dynamic dropout and energy-aware scheduling are extensions of existing concepts.

2. The proposed method is mainly evaluated on scale-scale models and datasets, lacking sufficient evaluation of its scalability to larger models and modern architectures, such as transformers.

3. SOTA training methods and systems like ePerceptive and Zygarde are not benchmarked. Additionally, the efficiency metrics are not sufficiently analyzed in the original manuscript.


**Reasons for the decision:**

While the paper's technical contributions are relatively incremental, it addresses a critical and underexplored challenge in training DNNs in energy-harvesting environments. In addition, the author's response provided more comprehensive experiments and efficiency analyses, effectively addressing most of the reviewers’ concerns. Therefore, I am inclined to accept this paper.