Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper provides a deep learning-based method to estimate value functions for multi-step dispatch on a power grid. This method entails training a different neural network to approximate the value function for each time step within a fixed horizon; supervision is provided via point-wise estimates of the states visited within rolled-out trajectories, as estimated via compound TD estimation. The method is demonstrated to have comparable performance to the "classic" Bellman iteration-based method that is currently used within the SimSEE dispatch system (currently employed by the Uruguayan electricity system operator), while running significantly (~150x) faster. 
main_review: Strengths: 
* The problem addressed in the paper is well-motivated from the perspective of both theory and practice, with direct application to a real-world system.
* The high-level approach and assumptions are clearly described.
* The performance of the method seems very competitive with respect to the existing baseline, though more details on the evaluation should be provided (see below). 

Weaknesses/questions:
* In general, I think the paper lacks some concreteness on the exact power systems setting, which make certain details of the paper harder to grasp. For instance, how is the value function actually used during planning? Do the cost functions represent only power generation constraints, or also e.g. constraint violations?
* I was not able to understand why a different model would need to be fit to every time step, or why this would make sense in the power systems setting. Is the idea that T=24 hours and t=1 hour, and since load values follow a roughly daily fluctuation, similar networks are used at times of similar load? Again, without more concreteness on the specific setting being studied, this was hard to grasp.
* Some critical details were missing, which made it hard to grasp the quality or soundess of the underlying method. For instance:
  * Section 4.1: More details should be given on the common random numbers technique (either in the main body or in an appendix), as the way the ensembles/chronicles are generated presumably has large implications for performance.
  * Section 4.3: I am not fully sure how the "anchor values" were employed.
  * Section 4.4: Why does this particular choice of $g$ make sense in the power systems context?
* Section 4.2: In the update equation, should the terms with/without tildes be switched? I am not sure the current update as written mathematically follows clearly from the verbal description/I am not sure where all the terms are coming from.
* Isn't doing online forward simulation at each step potentially expensive? 
* In general, it was hard to fully understand how the different steps presented in different sections of the paper fit together. An algorithm box would greatly enhance clarity.
* Additional details need to be provided in the experiments, in order to fully understand the efficacy of the proposed method. For instance:
  * Section 5: Some more details of the classic Bellman iteration-based method that is currently used could be provided. E.g., why is it currently so expensive?
  * Figure 3: More details on how the evaluation was conducted should be provided -- e.g. details of how $w_t$'s were generated in practice may have a large bearing on the results.
* Figure 4: It could be more thoroughly explained why parsimony between these two quantities is necessary/useful. Right now, the explanation given seems somewhat hand-wavy. 

Minor points (not affecting my score):
* Page 2, Section 1.2: "planification" -> "planning"
* Section 4.2: Why was $\alpha = 0.3$ chosen?
summary_of_the_review: The problem studied in this paper is important and well-motivated, and at a high level, the proposed approach seems reasonable and to work well. However, descriptions of several critical components of the theory and evaluation are either vague or not provided, making it difficult to truly evaluate the soundness and performance of the method.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper provides an approximation of the cost-to-go for a particular system. The particular system is the Uruguayan system. The results are compared to the approximate dynamic programming as provided in some classical books that are cited in the paper. 
main_review: I would say the only strength of this paper may be the application results yet it is difficult to be sure given that the method being compared to is not expected to perform well on a large and complex system. It may be currently in use as the authors are saying yet that should qualify their approach as a good option for whoever is interested in using it for the Uruguayan system but not as a scientific argument, in my opinion. 

As for weaknesses, the paper seem to be a concatenation of well known and simple techniques with some specifics that are not justified. For example, why is the third order polynomial chosen in Section 4.4 and why are betas as penalties chosen as in Section 5. There are no numerical characterizations of the performance of the algorithm, no bounds on the errors, no proofs of convergence, etc. In general, the paper provides no technical justifications for the claims. 

summary_of_the_review: The paper does not provide technical justifications for the claims. The approach is compared to a classical technique that is not expected to be very efficient when applied to a large and complex system, in general.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposed using time-dependent parameters to model the value function for evaluating the cost-to-go function in power systems. This work can be regarded as an extension from the traditional adaptive dynamic programming (ADP) with the study on how to represent the value function that fits the uncertainty of the dynamics of power systems induced by the installation decentralized energy resources (DERs). This work shows the performance of the proposed method on a realistic simulation. The background knowledge is well written and clear.
main_review: ## Strengths
1. The background knowledge is well written.
2. The presentation of the main contribution is clear enough.
3. A good attempt on combining data-driven ADP on the real-world problem.
4. Although the authors seem not a researcher in the ML community, the fundamental knowledge of ML described in the paper is almost correct and the logic is very clear.

## Weaknesses
1. The proposed method on representing the value function is actually not novel enough in the community of ML. It can be seen as a special case of using the hyper-networks to model the value function [1]. Nevertheless, the attempt to create innovations in the traditional fields should be encouraged.
2. Although the motivation of proposing time-dependent value function is correct, the representation and resolutions of **time steps** should be discussed. The inefficient representation of time steps could lead to the learning problem such as gradient explosion. Although in the paper the authors use hour as the time resolution, I doubt whether this can be generalized to all scenarios since it depends on the initial time points. If this is not generalizable, then the modelling of time dependency is less effcient than the time-independent but with extra context representing the state. Can the authors give more discussion on this point?
3. Can the authors give more details about the constructions of $g(\cdot)$ and $\beta\_{pt}$? In other words, why and how are the constructions yielded?
4. The authors lack many works on applying data-driven method to power system operations from IEEE transactions on smart grid and IEEE transactions on power systems. The comparisons with the related works in the same field is important. Can the authors give more related works in the discussion stage?


## References
[1] Ha, David, Andrew Dai, and Quoc V. Le. "Hypernetworks." arXiv preprint arXiv:1609.09106 (2016).

summary_of_the_review: In summary, this work is a good attempt on extending the traditional method with ML techniques. However, due to multiple concerns I have (see weaknesses), I recommend reject. On the other hand, if the authors can improve the work that can satisfy the standard of ICLR, the community should welcome such works dealing with the realistic problems.
review_title: null

Meta-review: The reviews are of good quality. The responses by the authors are commendable, but ICLR is selective and reviewers continue to feel that important choices in the research are not sufficiently clear and fully justified.
Decision: Reject