Learning-Augmented Dynamic Power Management with Multiple States via New Ski Rental Bounds

Abstract: We study the online problem of minimizing power consumption in systems with multiple power-saving states. During idle periods of unknown lengths, an algorithm has to choose between power-saving states of different energy consumption and wake-up costs. We develop a learning-augmented online algorithm that makes decisions based on (potentially inaccurate) predicted lengths of the idle periods. The algorithm's performance is near-optimal when predictions are accurate and degrades gracefully with increasing prediction error, with a worst-case guarantee almost identical to the optimal classical online algorithm for the problem. A key ingredient in our approach is a new algorithm for the online ski-rental problem in the learning augmented setting with tight dependence on the prediction error. We support our theoretical findings with experiments.


Reviews and author responses:

This paper is motivated by the challenge of dynamic power management in environments such as data centers. Computational devices like CPUs can transition to a sleep mode, for the purpose of preserving power. However, "waking up" entails consuming power, and so there is a tradeoff between entering a power-saving mode and avoiding future power spendings. The authors observe that this problem is reminiscent of the classical ski-rental problems and present a reduction from the former to the latter. This reduction enables borrowing deterministic and randomized algorithms for the ski-rental problem and applying them to this context. In addition, this enables using learning-augmented algorithms for ski-rental for dynamic power management (by predicting the duration of time in which the device is idle). Some experiments are presented.  Thank you for submitting to NeurIPS.

I enjoyed reading the paper. The results are easy to parse and the proof techniques are clearly presented.

The main weakness of the paper, in my view, is that the relation to dynamic power management in environments such as data centers is quite vague. This relation is highlighted in the first few paragraphs of the Introduction but is never revisited afterwards. The specific characteristics of different power-saving mode, the patterns of idle time periods and the extent to which these can be predicted in data centers, etc., are never discussed. Since dynamic power management is supposedly the "killer app" for the presented algorithms, I think that these issues should be discussed.

This also relates to the presented experiments. The evaluation framework is very limited and is quite detached from the motivating example. For instance, the drawing requests uniformly at random and considering predictions that are random perturbations of the actual labels seems overly simple. I realize that this was done in [35], too, but there the key goal was illustrating the usefulness of predictions for online algorithm, as opposed to tackling a specific problem, as is done here.

In light of the above, this paper should, in my view, be evaluated as a theoretical paper about the ski-rental problem. As such, I'm not sure that the results are sufficiently significant.

Following the authors' response:

Thank you for the detailed answer. I still have an issue with the positioning of this paper. I'm adjusting my score to 5. I do not think that the paper potentially has negative societal impact.

 We are glad to hear that you enjoyed reading our paper. You are right to conclude that the core of our contribution is theoretical (most importantly, tight bounds as a function of prediction error for one of the most fundamental online problems, whereas previous guarantees focused only on the extremes of perfect predictions and worst-case predictions).

We agree that our evaluation framework was limited. To address that issue we decided to run an additional experiment with a real-world scenario. Unfortunately, although some of the previous work on DPM that we cite includes experiments, these papers are not recent and the datasets used in them are no longer available. For that reason, we used I/O traces from Nexus 5 smartphone available at http://iotta.snia.org/traces/block-io. We took the five largest traces, from which we extracted durations of idle periods between requests. We haven't found power states specification for that smartphone, hence we used the same power states as for the synthetic experiments, i.e. an IBM mobile hard-drive's power states reported in [25]. Because of that, we had to scale up idle periods so that they have similar order of magnitude as in the synthetic experiments. For this real-world dataset, we also implemented a simple “real” predictor that generates predictions from past data using a multiplicative weight “Share update” learning algorithm of Herbster and Warmuth [21]. The competitive ratios obtained with this predictor, combined over the above smartphone datasets, are as follows:
>
        1.133 Our algorithm
        1.143 FTP  
        1.145 PSK
        1.146 Robust FTP 
        1.147 ADJKR
        1.401 Classical Online e/(e-1)-competitive

More detailed results for specific datasets can be seen here: https://imgur.com/a/RQ71Pmr
In particular, on each dataset, either our algorithm performs better than all others, or blindly following predictions (FTP, non-robust) is best and our algorithm is almost as good. We will include these new real-world experiments in the final version of our paper.

We will expand the discussion about practical DPM environments in the final version. In a data center environment, DPM is usually used on each machine independently and the sleep and active states usually correspond to the power states of the given machine, see paper on DPM for multi-tier data centers by Lim et al. (ISPASS'11). However, since applications are so varied, we do not make a general statement about e.g. patterns of idle times that would apply to all of them. The power states in our experiments correspond to the sleep, stand-by, idle and active states of an IBM mobile hard-drive.


 Thank you for your review.
The major differentiating factor between our and previous ski-rental algorithms is that our guarantee is a smooth (tight) function of the prediction error (which is also used in the DPM reduction), whereas previous works focused on the two extremes of perfect predictions vs worst-case predictions. Although Purohit et al. [35] actually also state a guarantee that could be phrased as $(\rho,\mu)$-competitiveness, in their case one has $\mu=\rho\ge 1$. Note that this guarantee is already trivially achieved by the algorithm that fully trusts the predictions, which is $(1,1)$-competitive. Interestingly, the best version of the algorithm of [35] in our experiments on the dataset of [35] is the $(1,1)$-competitive one. This might be additionally indicative of the notion of $(\rho,\mu)$-competitiveness being more useful for practical performance than merely optimizing robustness and consistency. Our new experiments (see our response to reviewer LdW3) also show a significant improvement on real-world data.    

There might have been a misunderstanding regarding robustness/consistency: When considering a single ski-rental instance in isolation, one typically requires $c=0$ in the definition of competitiveness (end of page 2; otherwise the problem would be trivial by buying at time 0 and choosing $c$ equal﻿ to the buying cost) and Purohit et al. obtained the optimal consistency-robustness trade-off for this setting. When several successive ski rental instances are considered, or a single DPM instance involving several idle periods, it is more common to allow a positive additive constant $c$ (since it is of negligible size compared to the overall cost). For this setting, one can achieve near-optimal consistency and robustness without the need to make any trade-off, and this is also (part of) the purpose of Theorem 5: It gives a *single* algorithm with consistency $1+\epsilon$ and robustness $e/(e-1)+\epsilon$ (and its performance degrades gracefully between these two values as the prediction error increases). Our other theorems show that one can achieve $(\rho,\mu(\rho))$-competitiveness for a fixed value of $\rho$, and Theorem 5 shows that rather than having to choose $\rho$ in advance, there is an algorithm that is $(\rho,\mu(\rho))$-competitive for all $\rho$ at the same time. So we do not skip over robustness, but rather we achieve near-optimal robustness (provided one allows $c>0$).


 You are right that our paper views the DPM problem mostly from an online algorithms perspective, abstracting away the question of how to generate these predictions. There do exist techniques in the literature to learn such predictions (we reference some in line 51 and lines 160-162), which were designed with the idea of simply following/trusting these predictions without accounting for prediction error/robustness. One way our paper may stimulate ML research on the side of generating these predictions is that since our performance guarantee is in terms of the $\ell_1$ error of predictions, it suggests this as a clear and simple loss function for learning these predictions.

For the final version of our paper, we will also include our additional experiments with real-world data where we actually learn predictions from past data. See our reply to reviewer LdW3 for details.

Thank you also for the link to the paper of Helmbold et al. on disk spin-down! We didn’t know about this paper, and we will cite it as additional related work. Interestingly, it uses the Share Update algorithm that we already mention in our paper as an alternative to the method of Blum and Burch. The empirical results of Helmbold et al. in fact inspired us to use Share Update for generating predictions in the additional experiments.


 Thanks a lot for reviewing our submission and for your positive comments.

This paper considers the problem of managing sleep states in dynamic power management in the framework of learning-augmented algorithms. This is a ski rental type problem but with multiple states, where idle periods of unknown lengths appear over time and the algorithm needs to decide which sleep state to transition to, where the deeper a sleep state it, the better it is for energy efficiency but the more expensive it is to bring it back to the awake state. The paper focuses on what it calls $(\rho, \mu)$-competitiveness, where alg $ \le \rho$ opt + $\mu \eta$ where $\eta$ is the (scaled) $\ell_1$ error in the predictions of the lengths of the idle periods. The goal is to optimize the tradeoff between $\rho$ and $\mu$. The optimal tradeoff is obtained by using a two-stage process: (a) show the the multi-state problem is equivalent to the 2-state problem, i.e., ski rental, provided the ski rental algorithm being used satisfies a technical monotonicity property, and (b) giving a monotone ski rental algorithm that optimizes the tradeoff between $\rho$ and $\mu$. This also gives a new result for the dynamic power management problem without predictions, by using the $e/(e-1)$ randomized ski rental algorithm and the above reduction. Finally, it is empirically validated through numerical experiments on synthetic datasets that the quality of the solution degrades smoothly with increasing prediction error.  This is a credible and interesting paper. Neither of the two steps -- the ski rental algorithm or the reduction from multi state DPM to 2-state DPM, i.e., ski rental -- is surprising, but each step requires some amount of careful technical work. For instance, in the second case, the idea is to set up the choice of transitioning from a state to its next state as a ski rental decision, and run these ski rental instances in parallel to obtain the overall DPM solution. Similarly, in the ski rental algorithm, the prediction has to be incorporated into the standard randomized algorithm for ski rental that chooses the buying threshold from a distribution with invariant competitive ratio. Overall, this paper adds interesting new ideas to the class of rent or buy problems that have been extensively studied in the learning-augmented setting. The paper does not discuss limitations explicitly. It would be beneficial to add this to the paper.

I do not see any potential negative societal impact.

The paper considers a variant of the dynamic power management problem (DPM) with a learning augmented scenario. They prove an optimal competitive ratio with a matching lower bound. The key technique is a novel reduction method from DPM to the classical ski-rental problem. The authors also give some extensions of the main result and show experimental results using artificial data, which validates the theoretical results.

  Comments:
The topic of the paper is about online algorithms and techniques shown are purely based on the literature. The proofs and techniques are solid and non-trivial but seem to follow rather standard techniques of the literature. The strength of the paper is showing the optimality of the prosed method. 

Communication between the online algorithm and machine learning communities might be fruitful in the long run, but currently, the discipline in the paper is purely from the online algorithm literature and I am not sure how this paper stimulates the related areas of machine learning.


There is a related paper on an application of online learning for tracking experts to the “disk-spin down” problem: https://doi.org/10.1023/A:1019129116852  This problem could be an instance of DPM. 

The writing is good in general, but it might be helpful for machine learning readers if the paper summarizes the proposed algorithm using, say, the algorithmic description.
 N/A

This paper provides an analysis of the learning-augmented dynamic power management (DPM) problem via a new randomized algorithm to the learning-augmented ski rental problem. These problems are examined through the lens of consistency and dependence on prediction error. The theoretical results are demonstrated by first detailing the new ski rental algorithm, and then reducing DPM to the ski rental problem. Finally, there are numerical results for comparison between several algorithms.   Originality: This work analyzes the consistency vs. prediction error dependence tradeoff, which is modestly different from the previous analysis of consistency vs. robustness tradeoffs. In particular, [35] also gives competitive ratio as a function of error/OPT. It is not entirely clear why the robustness metric is skipped over in this work. So, the authors are encouraged to motivate this new analysis concretely. The proposed ski rental algorithm is fundamentally original, as it is specified by a CDF. This is also the first work that applies the learning-augmented setting to DPM. 

Quality: The submission is technically sound.

Clarity: The submission is overall well organized and written. My main concern is with the presentation of Theorem 5. It is unclear what extended result this provides beyond Theorem 3 by "covering all \rho simultaneously", as it was introduced.

Significance: The experimental results demonstrate that the new algorithm achieves improved performance compared to previous ski rental algorithms in the DPM setting. However, the theoretical results are missing some comparisons against the robustness and consistency of Pareto-optimal randomized ski rental algorithms [35]. Also, the broader applicability of this new approach beyond ski-rental and its variants is not clear. 
 yes

Meta-review: This paper was borderline. The reviewers appreciated the contributions of this paper to the online algorithms literature, but some reviewers were skeptical about the learning aspect (fit for neurips vs theory conferences) and the application to power management. After discussions, I believe that papers in the area of algorithms with ML predictions are a good fit for neurips (in addition, there have been multiple such papers recently accepted to neurips and icml) and am in favor of acceptance.

Decision: accept