A Deep Generative Learning Approach for Two-stage Adaptive Robust Optimization
Authors: Aron Brenner
Rahman Khorramfar
Jennifer Z Sun
Saurabh Amin
Venue: ICLR 2025 Poster (2025)
Keywords: robust optimization
stochastic optimization
discrete optimization
deep learning
unsupervised learning
Primary area: optimization
TL;DR: We solve adaptive robust optimization using a variational autoencoder to generate realistic adversarial contingencies and obtain decisions that are sufficiently robust without being over-conservative.

Abstract: Two-stage adaptive robust optimization (ARO) is a powerful approach for planning under uncertainty, balancing first-stage decisions with recourse decisions made after uncertainty is realized. To account for uncertainty, modelers typically define a simple uncertainty set over which potential outcomes are considered. However, classical methods for defining these sets unintentionally capture a wide range of unrealistic outcomes, resulting in overly-conservative and costly planning in anticipation of unlikely contingencies. In this work, we introduce AGRO, a solution algorithm that performs adversarial generation for two-stage adaptive robust optimization using a variational autoencoder. AGRO generates high-dimensional contingencies that are simultaneously adversarial and realistic, improving the robustness of first-stage decisions at a lower planning cost than standard methods. To ensure generated contingencies lie in high-density regions of the uncertainty distribution, AGRO defines a tight uncertainty set as the image of "latent" uncertainty sets under the VAE decoding transformation. Projected gradient ascent is then used to maximize recourse costs over the latent uncertainty sets by leveraging differentiable optimization methods. We demonstrate the cost-efficiency of AGRO by applying it to both a synthetic production-distribution problem and a real-world power system expansion setting. We show that AGRO outperforms the standard column-and-constraint algorithm by up to 1.8% in production-distribution planning and up to 8% in power system expansion.

Published as a conference paper at ICLR 2025
A DEEP GENERATIVE LEARNING APPROACH FOR TWO-STAGE ADAPTIVE ROBUST OPTIMIZATION
Aron Brenner, Rahman Khorramfar, Jennifer Sun, & Saurabh Amin I Laboratory for Information Decision Systems Massachusetts Institute of Technology Cambridge, MA 02139, USA
(abrenner, khorram, jenni sun, (aainnsemmi edu
ABSTRACT
Two-stage adaptive robust optimization (ARO) is powerful approach for plan- ning under uncertainty, balancing first-stage decisions with recourse decisions made after uncertainty is realized. To account for uncertainty, modelers typi- cally define a simple uncertainty set over which potential outcomes are consid- ered. However, classical methods for defining these sets unintentionally capture a wide range of unrealistic outcomes, resulting in ineffective and costly planning in anticipation of unlikely contingencies. In this work, we introduce AGRO, solu- tion algorithm that performs adversarial generation for two-stage adaptive robust optimization using a variational autoencoder. AGRO generates high-dimensional contingencies that are simultancously adversarial and realistic, improving the ro- bustness of first-stage decisions at a lower planning cost than standard methods. To ensure generated contingencies lie in high-density regions of the uncertainty distribution, AGRO defines a tight uncertainty set as the image of "latent" uncer- tainty sets under the VAE decoding transformation. Projected gradient ascent is then used to maximize recourse costs over the latent uncertainty sets by leveraging differentiable optimization methods. We demonstrate the cost-efficiency of AGRO by applying it to both synthetic -ittuuctioonissiiuuttinn problem and a real-world power system expansion setting. We show that AGRO outperforms the standard aalumn-and-constrainn algorithm by up to 1.8% in diiiiitioon-dsttiuuttionn plan- ning and up to 8% power system expansion.
Introduction
Growing interest in data-driven stochastic optimization has been facilitated by the increasing avail- ability of more granular data for a wide range of settings where decision-makers hedge against uncertainty and shape operational risks through effective planning. Adaptive robust optimization (ARO) also known as adjustable multi-stage robust optimization is a class of stochastic op- timization models with applications spanning optimization of industrial processes (Gong & You 2017, transportation systems (Xie et al., 2020). and energy systems planning (Bertsimas et al,, 2012 In this work, we aim to solve data-driven two-stage ARO problems under high-dimensional uncertainty. These problems take the form:
{ TTTx max min d(E) T y
min #EX
gEU DEV(E.))
where x denotes "here-and-now" first-stage decisions and y denotes the "wait-and-see" recourse decisions made after the random variable RD is realized. The l C TR represents the uncer- tainty set, encompassing all possible realizations of & that must be accounted for when identifying the optimal first-stage decisions æ. Here, we assume that the first-stage decisions æ are mixed- integer variables taking values X' while the recourse decisions y take values the polyhedral set x) {y B(E 2 b(E) x(E) y > 0}, where d(.), B..,, b(.), and A() are affine functions of &. Additionally, we assume feasibility and boundedness of the recourse problem (i.e. complete recourse) for any first-stage decision E X and uncertainty realization U.

---

Published as a conference paper at ICLR 2025
Example (Power System Expansion Planning). As illustrative application of ARO, consider the problem of capacity expansion planning for a regional generation and transmission system. In this case, denotes long-tenn sin-term decisions (i.e., installing renewable power plants and transmission lines), which require an upfront investment e x. As time unfolds and demand for electricity & becomes known, electrical power y E J'EE. a:) can be generated and dispatched meet demand at recourse cost of d(E) y. To ensure robustness against uncertain demand, first-stage decisions balance investment costs with the worst-case recourse costs incurred over the uncertainty set U.
The uncertainty set plays a major role in shaping planning outcomes and should be carefully con- structed based on the available data. To optimize risk measures such as worst-case costs or value- at-risk, it is desirable to construct U to be as small as possible while still satisfying probabilistic guarantees of coverage (Hong et al., (02T). However, standard methods for constructing such uncer- tainty sets can yield highly conservative solutions to problem | -.e.., solutions that over-commil or otherwise allocate resources ineffectively in anticipation of extreme contingencies particularly in the case of high-dimensional and irregularly distributed uncertainties for which high-density regions are not well-approximated by conventional (e.g., polyhedral or ellipsoidal) uncertainty sets. While recent works have proposed using deep learning methods to construct tighter uncertainty sets for single-stage robust optimization (Goerigk& & Kurtz 2023] Chenreddy et al. (2022), such approaches have not yet been extended to the richer but more challenging setting of ARO.
We propose AGRO, a solution algorithm that embeds a variational autoencoder (VAE) within a columm-and-coonstriin generation (CCG) scheme to perform adversarial generation of realistic con- tingencies for two-stage adaptive robust optimization. Our contributions are listed below:
Extension of deep data-driven uncertainty sets to ARO. We propose a formulation for the adversarial subproblem that extends recent approaches for learning tighter uncertainty
sets in robust optimization (RO) (Hong et al. 2021; Goerigk & Kurtz 2023] Chenreddy et al.) 2022) to the case of ARO, richer class of optimization models for planning that allows for recourse decisions to be made after uncertainty is realized.
ML-assisted optimization using exact solutions. In contrast approaches that train pre- dictive models for recourse costs (Bertsimas & Kim, 2024 Dumouchelle et al., (0024). AGRO optimizes with respect exact recourse costs. Importantly, this eliminates the need to construct a large dataset of solved problem instances for model training.
Fast heuristic solution algorithm. To solve the adversarial subproblem quickly, we uti- lize projected gradient ascent (PGA) method for approximate max-min optimization that diffcrentiates recourse costs with respect to VAE-generated uncertainty realizations. By backpropagating through the decoder, with respect to a latent variable while limiting our search to high-density regions of the underlying uncertainty distribution.
able search for worst-case realizations
Application to ARO with synthetic and historical data. We apply our solution algorithm to two problems: (1) a didtuction-distriuutiin problem with synthetic demand data and (2) a long-term term power system expansion problem with historical supply/demand data. We show that AGRO is able to reduce total costs by up to 1.8% for the production-distributin problem and up to 8% for the power system expansion problem when compared to solutions obtained by the classical CCG algorithm.
2 Background
2.1 ARO Preliminaries
Uncertainty Sets. A fundamental modeling choice in both single-stage RO and ARO is the defi- nition of the uncertainty set U. which captures the range of uncertainty realizations for which the first stage decision æ must be robust. In data-driven setting, the classical approach for defining U is to choose a class of uncertainty set common examples include box, budget, or ellipsoidal uncertainty sets (see Appendix A.1 and to "fit" the set to the data using sample estimates of the mean, covariance, and/or mwmimmmummmmmuummmmmmmmmm values. A parameter I > 0 is usually introduced to determine the size of the uncertainty set, and by extension, the desired degree of robustness.
2

---

Published as a conference paper at ICLR 2025
I' is often chosen to ensure probabilistti guarantee of level a, or in the case of problem to ensure:
min d(E) y S = PE-PE min d(E) S (EV(E.))
2 a,
ŞeU (E)(E
where y can be understood as upper bound for the a-value-at-risk (VaR) of the recourse cost. To obtain such a probabilistic guarantee, one typically identifies an appropriate value for I' by leveraging concentration inequalities in the case of single-stage RO (Bertsimas et al. (02I). or more generally, using robust quantile estimates data "depth" (e.g., Mahalanobis depth) such that Pe u) 2 with high probability (Hong al. 2021).
In order to reduce over-conservative decision makisng,,, one would prefer to provide a tight ap- proximation of the probabilistic constraint. In other words, it is desirable to make 4 small as possible while still satisfying equation 2 Classical uncertainty sets, however, tend to yield looser approximations of such constraints the dimensionality this end, recent works have proposed learning frameworks for constructing tighter uncertainty sets in single-stage RO settings, which we discuss in Section 2.2
increases (Lam Qian 2019). To
Generation. Once 2/ is defined, one can apply a number of methods to either exactly (Zeng & Zhao 2013; Thiele et al.) 2009) or approximately (Kuhn al. [201]] I] solve
the ARO problem. We focus our discussion on the CCG algorithm (Zeng & 'Zhao 2013) due to its
prevalence in the ARO literature including
use in recent learning-assister methods (Bertsimas)
& Kim 2024 Dumouchelle et (024) and because it provides a nice foundation for introducing AGRO in Section 3 The CCG algorithm is an exact solution method for ARO that iteratively identifies "worst-case" uncertainty realizations & by maximizing recourse costs for given first-
stage decision x". These uncertainty realizations are added in each iteration to the finite scenario set S, which is used to instantiate the main problem:
min # #.3.7
cTx c
(3a)
s.t.
x E X' A x + B( 2 b(g"),
(3b)
i == 1,
S
(3c)
y' 2 0,
i [
(3d)
Y 2 d(E) y'
i
[S]
(3e)
where ? denotes the worst-case recourse cost obtained over S. In each iteration i of the CCG algorithm, additional variables (i.e., columns) y' and constraints y¹ E V x)) corresponding to the most recently identified worst-case realization are added to the main problem.
Solving the main problem yields a set of first-stage decisions, x=, which are fixed as parameters in the adversarial subproblem:
max min ŞEll (E)TE
d(E) y.
(4)
Solving the adversarial subproblem yields a new worst-case realization &' to be added to S. This max-min problem can be solved by maximizing the dual objective of the recourse problem subject to its optimality (i.e., KKT) conditions (Zeng & Zhao 2013). Despite its prominence in the litera- problem [ yields a bilinear optimization problem; specifically, one
ture, the KKT reformulation for which the number of bilinear terms scales linearly with the dimensionality of &. This need to repeatedly solve a large-scale bilinear program can cause CCG to be computationally demanding in the case of nigh-dimensional uncertainty. To this end, number of alternative solution approaches including some that leverage ML. methods have been proposed that approximate recourse decisions in order yield more tractable reformulations.
2.2 Related Work
Approximate and ML-assisted ARO. As alternative to CCG and other exact methods, linear decision rules (LDRs) are commonly used to approximate recourse decisions as affine functions of uncertainty (Kuhn al. 2011). To better approximate recourse costs using richer class of
3

---

Published as a conference paper at ICLR 2025
functions, Rahal al. (2022) propose "deep lifting" procedure with neural networks to learn piecewise LDRs. Bertsimas & Kim (2024) train decision trees to predict near-optimal first-stage decisions, worst-case uncertainty, and recourse decisions, which are deployed as part of solution algorithm for ARO. Similarly, Dumouchelle et al. (2024) train a neural network to approximate optimal recourse costs conditioned on first-stage decisions, which they embed as mixed-binary linear program (Fischetti & Jo 2018) within a CCG-like algorithm.
While these methods offer way avoid the challenging bilinear subproblem in CCG, they come with certain drawbacks. In particular, these methods optimize with respect to an approximation of recourse decision-man maniing that is either simplified and endogenized as first-stage variables (in the case of LDRs) or learned in supervised manner. As such, they cannot be expected to reduce plan- ning costs over CCG. Moreover, in the case of (Dumouchelle et al. 2024 Bertsimas & Kim 2024), a large dataset of recourse problem instances must be collected and solved, which can be compu- tationally demanding. Additionally, these works are focused on improving problem tractability and do not consider the challenge of loose uncertainty sets in high-dimensional settings.
Data-driven Uncertainty Sets. Some works have aimed to address this issue of loose uncertainty representation for single-stage RO by learning tighter uncertainty sets from data. One of the earliest such works, Tulabandhula & Rudin (2014), takes a statistical learning approach to building uncer- tainty sets from data. Hong al. (2021) provide a method for calibrating I' using measures of "data depth" and propose clustering and dimensionality reduction methods for constructing uncertainty sets. More similarly to our work, Goerigk & Kurtz (2023) introduce solution algorithm that first constructs a "deep data-driven" uncertainty set as the image of a Gaussian superlevel set under a neural network-learned transformation and then iteratively identifies worst-case realizations by op- timizing through the neural network. [Chenredyy el al.] (2022) extend their approach to construct uncertainty sets conditioned on the observation of a subset of covariates.
Both Goerigk & Kurtz (023) and Chenreddy et al. (2022) leverage the main result of Fischetti & Jo (0188) to embed a neural network within an adversarial subproblem formulated as mixed-binary linear program. However, extending this approach to ARO with high- diee-siiensio uncertainty poses significant challenges. Specifically, the lack of a closed-form expression for the recourse cost leads to adversarial subproblem that becomes mixed-binary bilinear program (see Appendix (A.2). While solution algorithms for such problems exist, their convergence will be slow for instances involving a large number of binary variables, is the when embedding neural networks. This underscores the need for fast, alternative approaches capable of generating approximate solutions for such bilinear adversarial problems, thereby enabling efficient worst-case analysis with "deep data-driven uncertainty" to extend beyond single-stage RO to the more general case of ARO.
3 Methodology
We now describe AGRO, a CCG-like solution algorithm that performs adversarial generation for robust optimization. As illustrated in Figure [ AGRO modifies the classical CCG algorithm in two key ways. (1) To reduce costs resulting from over-conservative uncertainty representations, construct an uncertainty set with known probability mass by training a VAE and projecting spherical uncertainty sets from the latent space into the space of &. (2) Rather than attempt to solve the resulting adversarial subproblem with an embedded neural network mixed-binary bilinear program, we propose PGA heuristic that quickly identifies cost- most-maximzzing realizations using the gradient of the recourse cost with respect &.
Like Dumouchelle et al. (002); Goerigk & Kurtz (2023): Chenreddy et al. (202)). we optimize "through" a trained neural network to obtain worst-case uncertainty realizations. However, rather than approximate recourse decisions using a predictive model as previous works have done for ARO (Dumouchelle et al. 2024 Bertsimas & Kim 2024], we exactly solve the recourse problem as a linear program in each iteration. Doing so also removes the computational effort required for building a dataset of recourse problem solutions - ie., solving a large number of linear programs to obtain an accurate predictive model for recourse cost given & and æ. In what follows, we describe how AGRO first learns tighter uncertainty sets and then solves the adversarial subproblem within a CCG-like scheme.
4

---

Published as a conference paper at ICLR 2025
Update fizst-stage docisions
Recourse Probiee
((6,x²)
min x0b1ca)1
y
f
u
Main Problem
dE/dz
Bo(E, x²)
Update S with wonst-case
E
(z)
8z
Figure I: The AGRO solution algorithm. (i) First-stage decisions x" are obtained by solving main problem, which approximates the original ARO uncertainty set by a finite scenario set S (see problem 3 (ii) The latent variable is sampled from within 2 and decoded to obtain an initial &. (iii) The recourse problem is solved given x" and & and its optimal objective value is differentiated with respect to & and (after backpropagating through the decoder fo) 2. (iv) gradient ascent step is taken update z, which then projected onto 2 and decoded to obtain an updated &. Steps (iii) and (iv) are iterated until converging a worst-case E. (v) The worst-case & is then added to S, at which point the main problem is re-optimized.
3.1 Deep Data-Driven Uncertainty Sets
AGRO constructs tight uncertainty sets nonlinear and differentiable transformations of spheri- cal uncertainty sets lying in the latent space R with L < D, where D is the dimensionality of (O IL) be
E. We approach learning these transformations as a VAE estimation task. Let an isotropic Gaussian random variable and let hs IP" + IR' and fo R - R respectively denote the encoder and decoder of VAE model trained generate samples from Pe (Kingma & Welling. 2013). Specifically, he is trained to map samples of uncertainty realizations to samples from N (0. Ir.) while fo trained to perform the reverse mapping. Here, T. denotes the VAE bottle- neck dimension, a hyperparameter that plays large role in determining the fidelity and diversity of generated samples (see Appendix A.3) which must be tuned through trial and error. We discuss the role of the bottleneck dimension further in Section 4
We note that our approach can be extended to other classes of deep generative models that learn such mapping from (0,, It) to Pş such as generative adversarial networks (Goodfellow et al. 2014), normalizing flows (Papamakarios et al. 202T). and diffusion models (Ho et 2020). which tend produce samples with higher fidelity in general (Bond-Taylor al.] (021). We employ VAE architecture as VAFs are known to exhibit relatively high stability in training and low compu- tational effort for sampling (Bond-Taylor et al., 2021), making them highly conducive integration within an iterative optimization framework. Additionally, experimental findings suggest significant cost reductions from AGRO over classical CCG even when using generative model that produces samples with only moderate fidelity (see Section E
To construct the uncertainty set U4 we first consider a "latent" uncertainty set given by the L-ball 2 = ( r). We select the radius I such that the decoded uncertainty set 2/ = {fo(z) z E 2} satisfies Pe (E E U)) 2 a with confidence level Letting E NxD be our training dataset, we leverage Theorem from Hong al. (2021) to obtain I according to the following steps.
1. Split =\t into two disjoint sets: one with N, 2 log / log a samples reserved for calibrating the size of the uncertainty set (i.e., steps and 3) and another with (N Ni) samples for training the VAE.
2. After training the VAE, project the calibration data into the latent space and sort the result- ing L2 norms, i.c. ry == ( I|2 for all ( the calibration set, to obtain the order statistics of (71, (f1.--.,7% ). We denote these order statistics by f(1) S .... S ((N))-
5

---

Published as a conference paper at ICLR 2025
3. Choose I
r(e), where is defined as:
Ni a a1 2
=
min ISSS
k=0
This procedure effectively prescribes to a robust (with respect to the sample size) o-quantile estimate of the norm of encoded samples drawn from Pe with confidence level 6. Figure T shows comparison classical uncertainty sets against tighter uncertainty set constructed by the proposed VAE-based method for samples from nonunimodal bivariate distribution.
2
2
$ 0
$ o
G 0
$
-2
-2
-2
-2
E1
&
(a) Box
(b) Budget
(c) Ellipsoidal
(d) VAE
Figure 2: A comparison of classical uncertainty sets (a)-(c) with the proposed VAE-learned uncertainty set (d) for an illustrative hivariate distribution. By covering smaller region IR. the VAF-leamed uncertainty set (d) is more likely to yield tighter approximation of the prohabilistic constraint in equation 2 and less conservative first-stage decisions.
3.2 Adversarial Subproblem
Bilinear Formulation. As the main problem in AGRO is the same as problem 3 for the classical CCG algorithm, we focus our discussion on the adversarial subproblem. Given VAE decoder with ReLU activations fo RR - RR,, latent uncertainty set 2 = B(0, r), and the first-stage decisions x" we formulate adversarial subproblem as
max min d(E) 6= VEX(E
{ Jo(=), = EE
(5)
To solve problem 5 one can reformulate it as single-level maximization problem by taking the
dual of the recourse problem and maximizing the objective with respect to subject to (1) opti-
mality conditions (i.e., primal/dual feasibility and strong duality) of the recourse problem and (2) constraints encoding the input-output mapping for the ReLU decoder network, & -= fe(z) (Fischetti)
& Jo 2018). This formulation is presented in full problem Z Appendix A.2 As discussed in
Section 2.2 however, this problem can be extremely computationally demanding to solve, There- fore, propose alternative approach for solving problem based on PGA to efficiently obtain approximate solutions.
Projected Gradient Ascent Heuristic. Let g: R X ER denote the optimal objective value of the recourse problem as a function of & and æ. is then possible to differentiate with respect to optimal solution
by implicitly differentiating the optimality conditions of the recourse problem (Amos & Kolter, 2017). Doing so provides an ascent direction for maximizing with respect to z by backpropagating through fa. As such, we maximize g using PGA Z.
Specifically, to solve the adversarial subproblem using PGA, we first randomly sample an initial from distribution supported on Z. We then transform
obtain the uncertainty realization E = fe(z) and solve the recourse problem with respect to & to obtain q(E, x²) and 0g(E, x²) //( Using automatic differentiation, we obtain the gradient of & with respect and perform the update:
dq(E. x dg
< ll
z +
ag
dz
where 17 > 0 denotes the step-size hyperparameter and II denotes the projection operator given by lT (2) min(T, ||=]|2] x z /|||=||2 By projecting onto the Z with radius T, we ensure

---

Published as a conference paper at ICLR 2025
that the solution to the adversarial subproblem is neither unrealistic overly adversarial. This procedure is repeated until convergence of ((f(()), x²) to a local maximum or another criterion (e.g., maximum number of iterations) is met.
PGA is not guaranteed to converge to a worst-case uncertainty realization (i.c., global maximum of problem (5 q[f((z), x=) is nonconcave in z. To escape local minima, randomly initialize PGA with different samples of z in each iteration and select the worst-case uncertainty realization obtained after maximizing with PGA. We then update the set S and re-optimize the main problem to complete one iteration of AGRO. Additional iterations of AGRO are then performed until conver- gence of the main problem and adversarial subproblem objectives, i.e., h (ff(2)), x²)| c, for tolerance e. Algorithm 2 provides the full pseudocode for AGRO.
4 Experiments
To demonstrate AGRO's efficacy in reducing planning costs, we apply our approach in of experiments: a synthetic production-distriiution problem and a real-world power system capacity expansion problem. For all experiments, we let a 0.95 and fix the confidence level be 0.05 (see Section 1.
4.1 Production-Distriiutiion Problem
Problem Description. We first apply AGRO to an adaptive robust production-diitribution problem with synthetic demand data in order to evaluate its performance over a range of uncertainty distri- butions. We consider a two-stage problem which the first-stage decisions a R correspond to the total number of items produced at each facility i II while second-stage decisions correspond to the quantities y E R Jl distributed to set of demand buses J after demands R|7 have been realized. Production incurs unit cost of in the first stage while shipping and unmet demand incur unit costs of d e [ and d' IR respectively. Here, transportation costs vary by origin and destination while the cost of unmet demand is uniform across buses.
Experimental Setup. In each experiment, we generate the demand dataset E R Nx|J by drawing samples from Gaussian mixture distribution with 3 correlated components. We vary the problem size over the range ([Z), [J) E {(4, (8, 6). (12, 9), (16, 12)), and perform 50 experimental tri- als for each problem size. In each trial, we sample dataset of 2500 samples, each of which is then divided into a 1000-sample VAE training set (split 80/20 for model training/validation)) 500- sample I l"calibration set, and 1000-sample test set. Additional implementation details are provided in Appendix B.I
Towards understanding how the VAE bottleneck dimension informs optimization outcomes, vary E {1, 2.. We report standard generative fidelity and diversity metrics for trained VAEs in Appendix B.1.1 To evaluate costs, we fix the solved investment decision æ and compute c x + f-1 ar), where a-; is the a-quantile of recourse costs obtained over the test given x. We compare cost and runtime results obtained by AGRO to those obtained by a classical CCG algorithm, for which we consider both budget and ellipsoidal uncertainty sets with sizes calibrated according to the approach described by Hong et al. (2021).
Costs and Runtime. The relative improvement in out-of-sample costs (i.e., c Tr + 5-1 (o; x) obtained by AGRO in comparison to CCG is shown in Figure 3 Cost comparisons for CCG with ellipsoidal uncertainty sets are omitted for the cases with [J] 6 as the resulting CCG subprob- lem could not be solved to optimality within the time limit of 900 seconds. Figure 3 shows that AGRO's 's relative advantage over classical CCG in minimizing costs grows as the dimensionaliiy of uncertainty increases. This is expected as classical uncertainty sets are known to provide looser approximations of probabilistic constraints in higher dimensional settings (Lam & Qian 2019). Runtime results provided in Table 2 much better than those for CCG with budget uncertainty set. In particular, CCG is solved 80 times faster than AGRO with L = for |J] = but is about 10 times slower than AGRO with L = 4 for [J] 12.
(see Appendix B.I.I] also show that runtimes for AGRO scale
7

---

Published as a conference paper at ICLR 2025
Bottleneck Dimension. The effect of the VAE bottleneck dimension out-of-sample costs can be observed from the box-plot in Figure 3 In particular, solutions obtained from AGRO using VAEs with L 4 outperform CCG more consistently than solutions obtained with L < 4. This is likely due to the fact that VAEs with I. = achieve a greater coverage of Pe (see Table 4 in Appendix [B..J]. and consequently, will consistently yield uncertainty sets that satisfy the proba- bilistic constraint. On the other hand, models with lower bottleneck dimensions are more prone to "overlooking" adversarial realizations due inadequate coverage of Pę, which causes occasional underestimation of worst-case recourse costs and under-prode peddu-production in the first stage. However, this effect is reversed in the low-dimensional case of |J] = the VAEs with higher bottleneck di- mensions, which score lower on generative fidelity metrics (see Table 33, over-produce in the first stage in anticipation highly adversarial yet unrealistic demand realizations.
[J]
Imgrovement over CCG (%)
14
6
12
8
8
8
Budg.
0.7% (1.0)
0.9% (1.0)
1.2%)((5)
1.5% (1.1)
Ellips.
0.4% (U.9)
(0.9)
Budg.
0.7% (0.7)
1.2% (0.6)
(1.0)
1.7% (1.0)
2
Ellips.
0.4% (U.5)
(0.6)
L= 1
1
== 2
Budg.
0.5% (0.7)
1.1% (0.8)
1.6% (0.8)
1.8% (1.0)
Ellips.
0.2% (0.5)
(0.6)
12
D
Figure 3: (Left): Average cost improvement with standard deviations for AGRO over CCG with budget and ellipsoidal uncertainly sets. (Right): Box-plots showing relative improvement for AGRO over CCG with budget uncertainty. The relative cost advantage of VAE-learned uncertainty sets over classical uncertainty sets increases with the dimensionality of uncertainty.
4.2 Capacity Expansion Planning
We now consider the problem of robust capacity expansion planning for a regional power system with historical renewable capacity factor and energy demand data. Capacity expansion models are centerpiece for long-term planning of energy infrastructure under supply, demand, and cost uncer- tainties (Zhang et al.) 2024 Roald et al. 2023). A number of works have adopted ARO formulations for energy systems planning with a particular focus on ensuring system reliability under mass adop- tion of renewable energy with intermittent and uncertain supply (Ruiz & Conejo 2015 Amjady et al. 2017 Minguez et al., 2017 Abbii et al. 2022). In models with high spatiotemporal resolu- complex, high- high-dimensional, and nonunimodal, exhibiting
tions, supply and demand uncertainties nonlinear correlations such as temporal autocorrelations and spatial dependencies. These patterns cannot be accurately captured by classical uncertainty sets. As result, tightening uncertainty repre- sentations for robust capacity expansion planning has the potential to significantly lower investment costs over conventional ARO methods while still maintaining low operational costs day-to-day.
Problem Description. We use AGRO to solve a two-stage adaptive robust generation and transmis- sion expansion planning problem for the New England transmission system. The model we consider most closely follows that of (Minguez et al.) (017) and has two stages: an initial investment stage and a subsequent economic dispatch (i.e., recourse) stage. The objective function is a weighted sum of investment costs plus annualized worst-case economic dispatch costs. We let N, T, and P denote the set of buses, hourly operational periods, and renewable energy sources respectively; here, |M1 6, |T| 24, and [P] 3. Investment decisions for this problem include installed capacities for (1) dispatchable, solar, onshore wind, and offshore wind power plants for all buses, (2) battery storage at each bus, and (3) transmission capacity for all edges in A A. The economic dispatch stage occurs after the realization of demand and renewable capacity factors, & E [RIII)I This stage determines optimal hourly generation, power flow, and storage decisions to minimize the combined cost of load shedding and variable costs for dispatchable generation; these decisions subject to ramping, storage, and flow balance constraints. The full formulations for both stages are presented in Appendix B.2
8

---

Published as a conference paper at ICLR 2025
Experimental Setup. After processing features (see Appendix B.2
have dataset with N = 7300 samples and D = 349 distinct features. Eight experimental trials are performed for AGRO with each trial having 6200 VAE training samples (split 80/20 for model training/validationn), 100 r-calibration samples, and 1000 test samples (all sampled uniformly at random). Similarly to
Section 4.1] we evaluate costs by fixing solved investment decisions and computing the a-quantile of economic dispatch costs over the test set. We consider a ball-box latent uncertainty set for AGRO, which intersects the latent uncertainty set described in Section 3.1 with a box uncertainty set in the latent space (i.e., min, ha (Er,), S 2j S max; ho ().,); for
1..... L). We compare AGRO against a classical CCG algorithm using budget-box box uncertainty set, which is commonly used in the power systems literature (Bertsimas et al.) 2012: Ruiz & Conejo 2015: Amjady et al. 2017
Abbin et al. 2022). Here, we calibrate U/ simila Similarly to using 100 samples (see Section 3.1). We do not report results obtained using an ellipsoidal uncertainty set as the corresponding CCG sub- problem could not be solved to optimality within the time limit of 900 seconds. As was the case in Section 4.1
we report results for three choices of the VAE bottleneck dimension: L E {2, 4, 8).
Value-at-Rise
Upper Bound
Error
Total
Train
Subproblem
CCG
11.3
(0.1)
15.3
1088%
(1935)
29183
1263
(937)
L == 2
14.9
(3.5)
11.2
(1.1)
14%
(52)
2162
(2128)
207
(5)
239
(796)
L 4
10.4
(0.3)
12.8
(0.2)
114%
(64)
2299
(509)
203
(2)
267
(225)
10.8
(0.3)
13.6
(0.5)
112%
(40)
3070
(618)
205
(1)
347
(253)
Table 1: AGRO and CCG cost and runtime performance estimates averaged over all trials (standard deviations shown in parentheses). Given the solved first-stage decisions x², Value-at- Risk denotes the sample-estimated objective value, c x=* + f æ)), while Upper Bound denotes the CCG/AGRO upper bound, c + 7 (both in units of $ billions). Error denotes the mean absolute percent error of worst-case recourse costs obtained by the subproblem compared an out-of-sample VaR estimate of recourse costs averaged over all iterations and folds. Total, Train, and Subproblem respectively denote average total solve time (not including training), average training time, and average subproblem runtime in seconds.
Costs and Runtime. Table I reports cost, runtime, and VaR approximation error results for our AGRO and CCG implementations. For L = 4 and L 8, AGRO obtains investment decisions that yield lower total costs compared to CCG as evaluated over held-out samples. In particular, costs are minimized by AGRO with L 4, which yields an 8% average reduction in total costs over CCG. For all L. the average total runtime for AGRO, which includes VAE training time, is also observed to be substantially lower than that of CCG. This is largely owing to the computational difficulty of solving the bilinear subproblem for CCG.
Approximation of VaR. To empirically compare the tightness of VAE-learned and classical uncer- tainty sets with respect to approximating VaR. we compare the relative error of worst-case recourse costs obtained by the adversarial subproblems (i.c., q(E", x))) with out-of-sample VaR estimates (i.e., (a; x²)) for all iterations. Table 1 shows that worst-case recourse costs obtained by AGRO with L 2 most closely approximate the true VaR. We visualize this phenomenon by plotting worst-case recourse costs against VaR estimates obtained in all iterations in Figure (see Appendix I.2.1.. Figure 5 shows that AGRO almost always overestimates VaR for T. 4 and I. = 8, which empirically suggests that the VAF-learned uncertainty sets provide a reliable ap- proximation for the probabilistic constraint in equation 2 The VAEs trained with L = 2 despite yielding the lowest error in approximating VaR - achieve a much lower coverage of pe than other models (see Table in Appendix B.2.Tp. As a result, the effective coverage of U is less than a, and U is no longer safe approximation for the chance constraint. Consequently, AGRO is unable to produce sufficiently adversarial realizations, and by extension, achieve low planning costs.
Realism of Generated Contingencies. We visualize AGRO's advantage over CCG in avoiding un- realistic contingencies by plotting solved worst-case realizations for CCG and AGRO in Figure 4 We observe that CCG generates highly unrealistic realizations in which capacity factors drop sharply to zero before immediately rising to their nominal values. Importantly, weather-related correlations between variables are not maintained; most notably, temporal autocorrelation (i.e.. smoothness) and spatial correlation of both load and capacity factors across the system. Accounting for these fea- tures during investment planning is crucial for effectively leveraging the complementarities between generation, storage, and transmission. On the other hand, AGRO is able to identify uncertainty realizations that simultaneously cost-maximizing and realistic when utilizing a sufficiently high-

---

Published as a conference paper at ICLR 2025
fidelity VAE (i.e., with L 4). This can be observed from the realistic temporal autocorrelation and spatial correlations demonstrated by the AGRO- EeROO-enertted realizations.
Load (MWh)
Solar CF
Offihore Wind CF
Onishore Wind OF
0.8
0.8
0.8
10000
0.4
0.4
0.4
5.000
0.2
0.2
0.0
10
15
20
10
13
20
20
mour
(a) CCG Uncertainty Realization
(MWhh)
20000
1.0
1.0
1.0
0.5
0.1
0.4
5000
0.0
0.0
0.0
10
15
20
10
5
flour 20
Itour
Hou
1100
(b) AGRO (L = 4) Uncertainty Realization
Figure 4: Comparison of worst-case uncertainty realizations as obtained by (a) CCG and (b) AGRO with L 4. Different colors used represent each of the six buses in the power system. A random sample 100 observations shown in gray to represent the historical distribution of load and capacity factors.
Bottleneck Dimension. As in the poduutionnddistribttion study, we observe that the optimal bot- tleneck dimension for the capacity expansion study (L = 4) does not necessarily yield the best generative model performance metrics (see Table 5 Appendix [..2. I.. This stems from the con- servative nature of uncertainty set-based approximations of chance constraints, which aim to cover probability mass rather than the region containing the least adversarial realizations, which would provide the tightest approximation. VAEs with reduced fidelity or diversity performance mitigate source over-commitment the first stage by avoiding overly adversarial scenarios. However, overly low fidelity diversity (e.g.. L 2) leads AGRO underestimate realistic contingencies, resulting in systemic commimmnm and higher recourse costs.
The effectiveness of AGRO's decision- making ultimately depends the quality of the underlying VAE as a generative model. To prevent systemic underestimation of recourse costs, it is essential that first select L large enough to ensure that generated samples are both diverse and realistic. This can be assessed through visual inspection and validated using quantitative metrics such as density and coverage. Reducing L narrow the uncertainty below a should only be considered when enough data is available to provide robust out-of-sample estimates of recourse costs, ensuring a reliable approximation of the chance constraint.
5 Conclusion
The increasing availability of data for large-scale system operations has fueled interest in data-driven optimization for planning. This work introduces AGRO, novel approach to two-stage ARO that integrates VAE within an optimization framework to generate realistic and adversarial represen- tations of operational uncertainties. Building the classical CCG algorithm, AGRO leverages differentiable optimization techniques identify cost-maximizing uncertainty realizations within VAE-learned uncertainty set. Empirical results demonstrate that AGRO outperforms standard ARO methods by more tightly approximating value-at-risk constraints on recourse costs, achieving up to 1.8% reduction in total costs in synthetic duction in a regional power system expansion problem using historical energy supply and demand data.
problem and up 8%
10

---

Published as a conference paper at ICLR 2025
References
Adam F Abdin, Aakil Caunhye, Enrico Zio, and Michel-Alexandre Cardin. Optimizing generation expansion planning with operational uncertainty: multistage adaptive robust approach. Applied Energy, 306:1 18032, 2022.
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and Zico Kolter. Differentiable convex optimization layers, 2019.
Nima Amjady, Ahmad Attarha, Shahab Dehghan, and Antonio J Conejo. Adaptive robust expansion planning for a distribution network with DERs. IEEE Transactions on Power Systems, 33(2): 1698-1715, 2017.
Brandon Amos and Zico Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International Conference Machine Learning, pp. 136-145, PMLR, 2017.
Dimitris Bertsimas and Cheol Woo Kim. A machine learning approach two-stage adaptive robust optimization. European Journal of Operational Research, 319(1):1-30, 2024. ISSN 0377-2217. doi:
URL https: //www. www.:wwwwwwiicc............
com/ http://cww/arrrceee///////222222
Dimitris Bertsimas, Eugene Litvinov, Xu Andy Sun, Jinye Zhao, and Tongxin Zheng. Adaptive robust optimization for the security constrained unit commitment problem. IEEE transactions on power systems, 28(1):52-666 2012.
Dimitris Bertsimas, Dick den Hertog, and Jean Pauphilet. Probabilistic guarantees in robust opti- mization. SIAM Journal on Optimization, 31 31(4):2893-2920, 2021. doi: 10. 10.1137/1MM1399967,
Sam Bond-Taylor, Adam Leach, Yang Long, and Chris G Willcocks, Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models. IEEE transactions on pattern analysis and machine intelligence, 44(1 44(11))7327-7347,22 2021.
Abhilash Reddy Chenreddy, Nymisha Bandi, and Erick Delage. Data-driven conditional robust op- timization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances Neural Information Processing Systems, volume 35, pp. 9525-9537 Curran Asso- ciates, Inc., 2022.
Justin Dumouchelle, Esther Julien, Jannis Kurtz, and Elias B. Khalil. Neur2ro: Neural two-stage robust optimization, 2024.
Matteo Fischetti and Jason Jo. Deep neural networks and mixed integer linear optimization. Constraints, 23(3):296-309, 2018.
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cyclical annealing schedule: A simple approach to mitigating vanishing. arXiv preprint arXiv:1903.10145, 2019.
Mare Goerigk and Jannis Kurtz. Data-driven robust optimization using deep neural networks. Computers & Operations Research, 151:106087, 2023.
Jian Gong and Fengqi You. Optimal processing network design under uncertainty for producing fuels and value-added bioproducts from microalgae: two-stage adaptive robust mixed integer fractional programming model and computationally efficient solution algorithm. AIChE Journal, 63(2):582-600, 2017.
Ian J. Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014.
Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc., 2020.

---

Published as a conference paper at ICLR 2025
L Jeff Hong, Zhiyuan Huang, and Henry Lam. Learning beasding-based robust optimization: Procedures and statistical guarantees. Management Science, 67(6):344733677, 2021.
Anna Jacobson, Filippo Pecci, Nestor Sepulveda, Qingyu Xu, and Jesse Jenkins. A Computationally Efficient Benders Decomposition for Energy Systems Planning Problems with Detailed Opera- tions and Time- Couplinn Constraints. INFORMS Journal on Optimization, 6(1):32-45, 45, January 2024. ISSN 2575-1492. doi: 10. 12277//00.2023.00000
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114 2013.
ar preprint
Daniel Kuhn, Wolfram Wiesemann, and Angelos Georghiou. Primal and dual linear decision rules in stochastic and robust optimization. Mathematical Programming. 130(1):177-299, 2011.
Henry Lam and Huajie Qian. Combating conservativenes: in data-driven optimization under uncer- tainty: A solution path approach, 2019.
Roberto Minguez, Raquel Perrfa-Bertrand, José M Arroyo, and Natalia Alguacil. On the solu- tion of large-scale robust transmission network expansion planning under uncertain demand and generation capacity. IEEE Transactions on Power Systems, 32(2): 1251, 2017.
Muhammad Ferjad Nacem, Scong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International Conference Machine Learning, pp. 717 7176-7185. PMLR, 2020.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lak- shminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1 2021.
Said Rahal, Zukui Li, and Dimitri J. Papageorgiou. Deep lifted decision rules for two-stage adaptive optimization problems. Computers & Chemical Engineering, 159:107661, 2022. ISSN 0098- 1354. doi: https://doi.org/10 h1t.:0011/....omoceeengg...o.. 107661..
Albert Reuther, Jeremy Kepner, Chansup Byun, Siddharth Samsi, William Arcand, David Bestor, Bill Bergeron, Vijay Gadepally, Michael Houle, Matthew Hubbell, Michael Jones, Anna Klein, Lauren Milechin, Julia Mullen, Andrew Prout, Antonio Rosa, Charles Yee, and Peter Michaleas. Interactive supercomputing on 40,000 cores for machine learning and data analysis. In 2018 IEEE High Performance extreme Computing Conference (HPEC), pp. 1-6. IEEE, 2018.
Line Roald, David Pozo, Anthony Papavasiliou, Daniel K Molzahn, Jalal Kazempour, and An- tonio Conejo. Power systems optimization under uncertainty: A review of methods and applica- tions. Electric Power Systems Research, 214: 108:25,,5, 2023.
C. Ruiz and A.J. Conejo. Robust transmission expansion planning. European Journal of Operational Research, 242(2):390-401, 2015, ISSN 0377-2217. doi: http://o...rr/1...011.i...o..oo 030.
Aurélie Thiele, Tara Terry, and Marina Epelman. Robust linear optimization with recourse. Rapport technique, pp. 4-37, 2009.
Theja Tulabandhula and Cynthia Rudin. Robust optimization using machine learning for uncertainty sets, 2014.
Shiwei Xie, Zhijian Hu, and Jueying Wang. Two-stage robust optimization for expansion planning of active distribution systems coupled with urban transportation networks. Applied Energy, 261: 114412, 2020.
Bo Zeng and Long Zhao. Solving two-stage robust optimization problems using a column-and- constraint generation method. Operations Research Letters, 41(5):457-461, 2013. ISSN 0167- 6377. doi: https://doiorg// 10. 10.1100111.. 1016//.orl.2013.055
Hongyu Zhang, Nicolè Mazzi, Ken McKinnon, Rodrigo Garcia Nava, and Asgeir Tomasgard. A stabilised Benders decomposition with adaptive oracles for large-scale stochastic programming with short-term and long-term uncertainty. Computers & Operations Research, pp. 106665, 2024.
12

---

Published as a conference paper at ICLR 2025
A
Methodological Details
A.1 Uncertainty Sets
The box, budget, and ellipsoidal uncertainty sets are constructed in a data-driven setting
unt = {E E
12
Emin
&
Émux
(6a)
D
vaddet
- { ERD
Σ
15: - Aisl T budget
(6b)
1=1
cllipsee = {
(E T T ( S
elllise
(6c)
Here, ja, E. Eiin and Émax denote the empirical mean, covariance, minimum value, and maximum value of & observed from the dataset while the parameters denote the uncertainty "budgets."
A.2 Bilevel Formulation for AGRO
Letting π denote the dual variables of the recourse problem, the max-min problem 5 cean be refor- mulated as the following mixed-binary bilinear program:
max s.t.
d(E)
(7a)
B(E)y 2 b(E) A())=
(7b)
x' T BB)) < d(E),
(7c)
y. 0,
(7d)
" Tb((E) - A(E)²") ) = d(E) y.
(7e)
|=||2 ST,
(7f)
(( (0)) =0() $01,
(7g)
((0 + (() =x( (0+1) ((++1)
EE [K -1]
77h)
(KK) bb E.
(71
( =1 = 2) 0,
j E [DA] LE [K -
(7j)
=0 = 2) 0,
j E [Dd. l E IK -1]
(7k)
00 (0 20,
Le [K] E [K -1]
(7l)
a E Pt.
(7m)
Here, constraints 7b and 7C respectively enforce primal and dual feasibility, constraint 7d enforces nonnegativity of the primal and dual variables, and constraint 77 enforces strong duality. Together, constraints
sufficient conditions for optimality of the recourse problem. Constraint 7
enforces E Z to bound the range of uncertainty realizations that considered. Additionally, apply the main result of Fischetti & Jo (018) to embed the mapping & = fo (z) by introducing the binary decision variables
]ee[K-i] I] and real-valued decision variables {z (4) TEE[K) through constraints 7g 7m Here, Dr denotes the number of output units in layer while (((), E RR
and (()) E R denote the weight matrix and offset parameters of layer f. Note that, for any X. problem is feasible consequence the primal feasibility (i.e., complete recourse) and boundedness assumptions.
A.3 Fidelity and Diversity Metries
We quantify performance of the VAEs as generative models for Pe using standard metrics for fidelity (precision and density) and diversity (recall and coverage). Semantically, fidelity can be understood
13

---

Published as a conference paper at ICLR 2025
measuring the "realism" of generated samples while diversity be understood measuring the breadth of generated samples. These metrics are formally defined by Nacem et al. (2020) as follows:
M
precision
M
EU Emanifold(())........
j=1
M
density
kM
E, (E(E
/=1
1
recall
N
f. Emaniifold((()) (......)
N
coverage
f s.t. BUNEBIECO,NNDACC
where & and & respectively denote real (N total) and generated samples (M total), denotes the indicator function, (NDx denotes the distance from ((9) to its k-th nearest neighbor (excluding itself) in the dataset (g(1), ,...., ((N)), and manifolds are defined
, (...,.(()))
U O NNDD&
Precision and density quantify the portion of generated samples that are "covered" by the real sam- ples while recall and coverage quantify the portion of real samples that are "covered" by the gen- erated samples. All results for both sets of experiments are obtained over the held-out test datasets using i = 5 and with M N 1000.
A.4 AGRO Algorithm
Algorithm 2 AGRO Algorithm
Require: Decoder Jo. Confidence Level a, Step-size 17, Initializations 7, Convergence tolerance
Ensure: Optimal solution x¹
I: Initialize Scenario set S 0. Iteration counter <-
2: while converged do
2=, =, <- arg min mingexx, (f(z) + +715(()=) VE e S}
Solve main problem
4: for j= to do
Solve adversarial subproblem
5: Sample z'
6: while not converged do
7:
Decode go(22)
Solve recourse problem for x5,x*)
9:
x' - 11z (2' + 2 12
2 de
10:
end while end for
1 +- arg max (ggos (,))))))
Select worst-case realization
13:
t go(=")) if S xx) E' then
Check convergence
15:
Terminate and return x² else
16:
17:
Update S < S Increment i+1
Add worst-case realization to scenario set
18:
19:
end if
20:
end while
21:
return x:
14

---

Published as a conference paper at ICLR 2025
B
Experimental Details
B.1 Production-Distributiin Problem
Formulation, We formulate the two-stage production-disttibuuii problem as
min
Σ
G.
+ max ((E,
(8a)
eell
VEI
s.t. 220,
(8b)
where the recourse cost ((, æ) is given as the solution to the following linear program:
q x)
min
ΣΣ
d'yuky Σ>
(9a)
y
JEJ
jEJ
s.1.
Σ uly + +1 2 Eg:
j jEJ
9bb)
VEI
Σ Yi S p,I.. JEJ
iez
(9c)
y¹ y²² 2 0.
(9d)
This problem is adapted as a continuous relaxation (with respect to the first-stage decisions) of the facility location problem introduced by |Berttsimas & Kim (2024). Here, the first-stage decisions are the real-valued decision variables x; determining the amount of items to produce at facility i at a unit cost of 6 for E After the demands & for all delivery destinations j E I are realized, items shipped from facilities delivery destinations at unit cost of de for facility and destination j. These shipment quantities are captured for i E I and j E J by the continuous recourse decision variables Vij. An additional penalty is incurred for any unmet demand y² at destination j with unit cost dZ Constraints 9b define yf as unmet demand while constraints 9c ensure the total number of items shipped from a facility does not exceed its production, p,x', where p: is the production factor for facility i.
Generating Instances. In each trial, we randomly sample both mean and covariance parameters of the Gaussian mixture Pş well parameters of the distriitnn problem, c, d and, p (d' is fixed to 5 in all iterations). To define the Gaussian mixture distribution pe, we first sample the mixing parameters Ç E [o, 1]s from a symmetric Dirichlet distribution, i.e., Ç ~ Dir(1). For each component A = 1,..., 3, we sample the population mean μ² ~ (O [})) and the population covariance * from the Wishart distribution with J degrees of freedom and identity scale matrix | d E | and, p [R] we follow the
1 To generate the problem parameters procedure described by Bertsimas & Kim (2024). Specifically, we sample d uniformly at random from the interior of the ball B(d, 1.5) where d, ~ Unif(2, 22) for i E I. We then sample pı ~ Unif(8, 18) and c; Uni 4) for i
Computational Details. Both the encoder and decoder of our VAE correspond to three-layer net- works with ReLU activations and batch normalization between layers; all intermediate fully con- nected layers have 32 hidden units. All VAE models are trained using the Adam optimizer with a learning rate of 0.001. Additionally, we utilize cyclical annealing schedule that varies the weight of the KL-divergence term for the VAE training objective (Fu et al. 2019). All training is performed on the MIT Supercloud system (Reuther et al.) 2018) using an Intel Xeon Gold 6248 machine with 40 CPUs and two 32GB NVIDIA Volta V100 GPUs, which takes approximately 30 seconds for all instances. All optimization also performed on the MIT Supercloud system (Reuther et al. 2018) using an Intel Xeon Platinum 8260 machine with 96 cores and using Gurobi II (Gurobi Optimiza- tion. LLC )023) except when solving the AGRO subproblem, in which case results are obtained using the Cvxpylayers package (Agrawal et al. 2019)
To solve the adversarial subproblem with AGRO, we perform 10 random initializations of z. For each initialization, we perform normalized PGA with learning rate of 17 = 0.1 until either (a) costs obtained in successive PGA steps have converged within tolerance of 0.01% or (b) 1000 PGA steps have been performed. If the worst-case realization & obtained over all 10 initializations does
15

---

Published as a conference paper at ICLR 2025
not exceed the lower bound obtained by the most recent iteration of the master problem (i.e., 7), perform additional initializations until either (a) a new worst-case realization obtained or (b) 200 initializations have been performed (in which case, AGRO terminates).
B.1.1
Experimental Results
Z
[
Budget
Ellipsoid
AGRO (L 1)
AGRO (L 2)
AGRO (L
3
1.13
1.13
11.93
46.61
84.77
6
7.1
371.18
13.8
45.53
109.69
9
250.09
16.29
73.85
115.08
12
1199.22
26.91
76.59
113.4
Table 2: Runtime results for produuciion-diitribution problem (not including VAE training).
[ 3 3
[]]
|
| 12
L
Precision
Density
Precision
Density
Precision
Density
Precision
Density
0.98
1.08
0.98
1.43
0.97
1.94
0.97
2.65
2 0.98
1.05
0.95
1.22
0.93
1.56
0.93
2.1
4
0.97
1.0
0.94
1.09
0.92
1.36
0.92
1.78
Table 3: Fidelity metrics for VAEs trained using the production-diisrriutiee problem data averaged over all 50 instances.
J| Coverage
3
[J]
||]
J| 12
L
Recall
Recall
Coverage
Recall
Coverage
Recall
Coverage
0.08
0.32
0.01
0.28
0.01
0.3
0.0
0.34
2
0.78
0.83
0.32
0.66
0.15
0.65
0.08
0.67
4
0.96
0.95
0.89
0.93
0.61
0.89
0.37
0.88
Table 4: Diversity metrics for VAEs trained using the cooduction-distrbuttin problem data averaged over all 50 instances.
B.2 Capacity Expansion Model
The capacity expansion model is given by
min
Σ
a
c'r +
Edtt Σ
Ealy + A Amx q(E, x)
(10a)
gell
IEN
iEN
PEP
iEN
(K.))E
Σ x P
Pe
(10b)
iEN
d
(10c)
20
a x x 2 0.
(10d)
Here the first-stage bus-level decision variables include installed dispatchable generation capacity (x:f Z4), installed renewable plant capacity (x, 2 0) for renewable technologies P. and installed battery storage capacity (x) 2 0). Additionally,
consider transmission capacity (zf, 2
0) first-stage decision variable for all edges (i, E. Importantly, installed renewable plant
16

---

Published as a conference paper at ICLR 2025
capacities obey system wide resource availability constraint. Finally, maxçeu ({,, denotes the worst-case recourse cost (i.e., economic dispatch cost given by equation[II), which is incurred with unit cost À part of the capacity expansion objective function. We "annualize" the worst-case recourse cost by setting À = 365. For given investment decision x and uncertainty realization &, the recourse cost is given by
min ΣΣ 11
d' vhe + d
(Ila)
LET
s.t. yse +
Σ
Vist
Σ J:(8,))EE
Vist +
2
3 +
4 Vsc
E/t
Σ S
iENt
(11b)
PEP
Vigt <a/
( ETt (1lc)
Viz1 S oig
(1,)) (1ld)
v S Vxz
iENT (Ile)
U/e Vr,1-1 S Kuzz
NENT (III)
v - via S -KVI?
NttTT (1lg)
11 S o
N,NT (1lh)
v V
+ _/ -
iENTEET (1Ii)
1 1/ 00
iEN,EET (Ui
y V. y 20 0.
(11k)
Here, the total dispatchable generation at bus i period is denoted by Vir vijr denotes the inflow of power from bus to bus j in hour t. yar denotes the discharge rate of the battery located at bus i in hour while y denotes its total charge. Load shedding is denoted by Sgg which is assumed to be the positive part of VÂg V denotes the nameplate capacity of the dispatchable plants while k and a respectively denote the maximum ramp-up and ramp-down rates for dispatchable generation. The objective function [1a minimizes the combined cost of fuel and a carbon tax for dispatchable generation (with unit cost jointly denoted by e as well as load shedding (with unit cost c*). First- stage decisions xy x². x and z'y parameterize the constraints of 11 Constraints 1 impose flow conservation each bus. Specifically, the constraint balances dispatchable generation, power transmitted to/from other buses, nodal load shedding, and battery discharging with net load (i.e., demand minus total generation from renewables). Constraints TTc and Tid impose line limits on power flows. Constraints 11e impose limits on total dispatchable generation while constraints [TT and 1g limit ramping up and down of dispatchable generation. Constraints ITh limitss the total state- of-charge of batteries. Constraints links battery discharging charging (taken to be the negative of discharging) rates to battery state-of of-charge. Constraints ensure that only the positive part of load shedding incurs cost. To simulate inter-day dynamics of storage and ramping, we implement circular indexing, which links battery state-of-charge and dispatchable generation rates at the end of the day with those at the beginning (Jacobson et al.) 2024).
[ijensure
Data Processing. To process the data for the capacity expansion planning problem, we remove components corresponding solar capacity factors for nighttime hours and offshore wind capacity factors for landlocked buses, both of which correspond to capacity factors of zero in all observa- tions, which gives E I2427 In implementing CCG, we also remove redundant (i.e., perfectly multicollinear) components so that the dataset of & observations has full column rank when defining the uncertainty set. We then re- in-nrroduuee the redundant components as affine transformations of the retained components in the adversarial subproblem using equality constraints. Consequently, the ef- fective dimensionality of L/ is 349 while the dimensionality of & is 427. For AGRO, we only remove components that are constant (i.c., zero) and train the VAE on the original 427-dimensional dataset.
Computational Details. Both the encoder and decoder of our VAE correspond to three-layer net- works with LeakyReLU activations and batch normalization between layers; each fully connected layer has 64 hidden units. All VAE models are trained using the Adam optimizer with a learning rate of 0.01. As in the case for the production-distribution problem, we utilize cyclical annealing schedule during training. All training is performed on the MIT Supercloud system (Reuther et al. 2018) using Intel Xeon Gold 6248 machine with 40 CPUs and one 32GB NVIDIA Volta V100
17

---

Published as a conference paper at ICLR 2025
GPUs. Training times range from approximately 190 250 seconds for all instances. All optimiza- tion is also performed on the MIT Supercloud system (Reuther et al. 2018) using an Intel Xeon Platinum 8260 machine with 96 cores and using Gurobi II (Gurobi Optimization, LLC 2023) ex- cept when solving the AGRO subproblem, in which case results are obtained using the Cvxpylayers package (Agrawal et al. 2019).
To solve the adversarial subproblem with AGRO, we perform 10 random initializations of z. For each initialization, costs obtained in successive PGA steps have converged within a tolerance of 0.01% or (b) 1000 PGA steps have been performed. the worst-case realization & obtained over all 10 initializations does not exceed the lower bound obtained by the most recent iteration of the master problem (i.e.,
perform normalized PGA with learning rate of 0.03 until either (a)
7), we perform additional initializations until either (a) new worst-case realization obtained or (b) 30 initializations have been performed (in which case, AGRO terminates).
B.2.1
Experimental Results
10
5.0 le7
1e9
4.5
Subprabicm Estimate
Estimate
Subprablem Estimate
1+
10a
3.0
CCG
107
2.5
Sample Estimate
Sample Estimate
1e7
Sample Estimate
1e9
Figure 5: Comparison of sample-estimated VaR, 8 fai x=",, and estimated worst-case costs ob- tained from solving the adversarial subproblem, f(E' x="). Each point is obtained from one iteration of AGRO/CCG, Points that are closer to the diagonal line indicate more accurate estimate of VaR. The middle and right-hand plots provide zoomed-in view of regions with high concentration of points. Subproblem estimates are almost always greater than sample estimates except in the case of AGRO with L (blue).
Precision
Density
Recall
Coverage
2
0.84
1.05
0.11
0.57
4
0.89
1.45
0.21
0.77
8
0.96
1.72
0.39
0.89
Table 5: Fidelity and diversity metries for VAEs trained using the capacity expansion problem dataset averaged over all eight trials.
18


Reviews:
[{"review_id": "ytsLmZTiij", "reviewer": "Reviewer_vrQ2", "summary": "This paper introduces AGRO, a novel method for two-stage adaptive robust optimization (ARO) using a variational autoencoder (VAE) to generate adversarial and realistic uncertainty sets. The authors demonstrate that AGRO reduces planning costs in ARO tasks, outperforming classical approaches.", "questions": "1.\tWhy did you choose a VAE over other generative models (e.g., GANs, normalizing flows) for constructing uncertainty sets in AGRO? Would these models offer any advantages or limitations compared to VAEs in this application?\n2.\tHow does the bottleneck dimension (L) influence the overall performance and reliability of AGRO? Could you elaborate on any trade-offs between computational cost and uncertainty set coverage as L varies?\n3.\tThe paper discusses using VAE-based uncertainty sets to achieve tighter approximations. Could you clarify how you ensure these sets are both realistic and adversarial? Are there any specific quantitative or qualitative metrics that assess the accuracy of these generated uncertainty sets?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed AGRO framework is innovative, embedding a VAE within a column-and-constraint generation (CCG) scheme to achieve high-dimensional adversarial generation with cost efficiency.\n2. It seems that the empirical results highlight an over 10% cost reductions over classical methods.", "weaknesses": "1.\tThe introduction lacks a comprehensive motivation for using a VAE for uncertainty sets over other generative models. The authors should justify why a VAE was chosen and discuss the potential advantages over alternatives, like GANs or normalizing flows, which may also be suitable.\n2.\tThe discussion on the choice of the VAE bottleneck dimension (parameter L) could be expanded. The authors should provide more insight into how different L values affect the uncertainty set’s coverage and the balance between computational cost and model fidelity.\n3.\tWhile the experiments are detailed, there is no mention of computational time for VAE training or comparison with other ARO solutions. Including such results would enhance transparency about AGRO’s feasibility in larger-scale applications.\n4.\tThe paper does not explore alternative formulations for the adversarial subproblem. A comparison with different optimization methods or a discussion on the limitations of projected gradient ascent could further clarify AGRO's robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FXEjdflaxb", "reviewer": "Reviewer_foZ7", "summary": "This paper presents a novel deep generative approach using Variational Autoencoders (VAE) to tackle two-stage adaptive robust optimization (ARO) under high-dimensional uncertainty. Traditional ARO approaches for constructing the uncertainty set $\\mathcal{U}$ tend to be overly conservative, often leading to excessive resource allocation in scenarios with high-dimensional and irregularly distributed uncertainties. The proposed AGRO method mitigates this issue by incorporating VAE embedding and column-and-constraint generation (CCG). The method also uses projected gradient ascent to solve the formulated subproblem. Experiments on two problems demonstrate the advantages of this method over conventional CCG approaches.", "questions": "- In Figure 1, the authors use two 3D visualizations to illustrate the two-stage operations on $\\mathcal{U}$ and $\\mathcal{Z}$. Could the authors provide a brief description of the specific main problem addressed here to give the audience a better understanding?\n\n- In the experiment on the production-distribution problem, the authors observed a reverse effect of the bottleneck dimension in the low-dimensional case of $|\\mathcal{J}|=3$. Could the authors elaborate on possible reasons for this?\n\n- Based on the experiments, could the authors provide practical guidelines for selecting the appropriate bottleneck dimension for VAEs according to the dimensionality or complexity of the uncertainty set?\n\n- For tabular results such as those in Figure 3 (left) and Table 1, could the authors also report the standard deviation across trials? This would help readers understand the robustness of AGRO in different practical scenarios.\n\n- Could the authors provide more details on the VAE architecture and training settings used in each experiment? Such as layer dimensions, normalization, optimizer, and learning rate,  for better reproducibility.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The primary contribution of this work is the innovative application of VAE to construct a tighter uncertainty set, thereby reducing over-conservatism in high-dimensional decision-making, which is then addressed through CCG. The paper is clearly presented, with informative visuals such as Figure 2, and well-organized notation and formulations. The experimental results highlight the promise of the proposed method.", "weaknesses": "The proposed approach involves training a VAE, whose performance might be sensitive to hyperparameters, computational resources, and the amount of available training data. Additional experiments and discussion could enhance the paper’s applicability. Please see the following questions for further details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i2UdoFR7XT", "reviewer": "Reviewer_A31W", "summary": "This paper addresses the two-stage adaptive robust optimization (ARO) problem, where a key challenge is constructing an effective uncertainty set. The authors propose using a deep generative model to learn the uncertainty set, aiming to avoid overly conservative optimization. The method is evaluated on a synthetic production-distribution problem and a regional power system expansion problem.", "questions": "1. The paper suggests that the framework is general and could also be applied to diffusion models. However, diffusion model training involves matching the score of the noised distribution, and samples cannot be easily obtained during training. Could you elaborate on how diffusion models would integrate with your proposed framework?\n\n2. How do you ensure the uncertainty set learned by the VAE is sufficiently tight? When optimizing over the latent space, are there any constraints? If not, is there a risk that the algorithm could select an overly conservative worst-case realization?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and clear.\n\n2. Leveraging deep generative models to learn the uncertainty set is a promising approach.", "weaknesses": "1. The Projected Gradient Ascent (PGA) method does not guarantee convergence to the worst-case uncertainty realization. Although the authors propose randomly initializing PGA with different samples of $z$ for empirical performance, providing some theoretical analysis on the approximation error would be beneficial.\n\n2. The performance improvement of the proposed method is minimal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uUCzaoeazl", "reviewer": "Reviewer_GwvX", "summary": "This paper proposes the AGRO algorithm, which performs adversarial generation for two-stage adaptive robust optimization using a variational autoencoder. By decomposing the optimization problem to solving the 'main' problem and the adversarial subproblem iteratively, AGRO can provide tighter uncertainty estimation and lead to better optimization outputs.", "questions": "My main question for this work is: Is the optimization problem the author wants to solve exactly a linear optimization problem (see Eq. 1, 7, 9, 10)? If so, there are already many tools for solving linear optimization problems, so why is the proposed method better than those?\n\nIf not, what kind of optimization problem does AGRO solve? Only convex?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, easy to follow, and understand.\n\n2. With VAE-learned uncertainty, the proposed AGRO method does tighten the uncertainty bounds and leads to better optimization outcomes. An intuitive example in Figure 2 and experimental results clearly demonstrate this.", "weaknesses": "1. In Section 3.2, the author proposes a projected gradient ascent heuristic method to optimize $q$. Although this PGA method is well-explained in the article and I understand why the author uses it, I still expect an ablation study on directly optimizing $q$ to show if PGA could still guarantee some level of optimization quality and if there are any speed improvements.\n\n2. Although the proposed AGRO method is an improvement based on the CCG method, in the experiments section, the author should compare with more baselines for the two-step optimization problem, which I believe is a well-studied problem with many methods proposed to solve it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)