Data-Driven Conditional Robust Optimization
Authors: Abhilash Reddy Chenreddy
Nymisha Bandi
Erick Delage
Venue: NeurIPS 2022 Accept (2022)
Keywords: Robust Optimization
Contextual Optimization
Conditional Stochastic Optimization
Unsupervised learning
Deep Neural Networks
Conditional Robust Optimization

Abstract: In this paper, we study a novel approach for data-driven decision-making under uncertainty in the presence of contextual information. Specifically, we solve this problem from a Conditional Robust Optimization (CRO) point of view. We propose an integrated framework that designs the conditional uncertainty set by jointly learning the partitions in the covariate data space and simultaneously constructing partition specific deep uncertainty sets for the random vector that perturbs the CRO problem. We also provide  theoretical guarantees for the coverage of the uncertainty sets and value at risk performances obtained using the proposed CRO approach. Finally, we use the simulated and real world data to show the implementation of our approach and compare it against two non-contextual benchmark approaches to demonstrate the value of exploiting contextual information in robust optimization.

Data-Driven Conditional Robust Optimization
Abhilash Chenreddy
Nymisha Bandi McGill University Montréal, Quebec, Canada nymisha.. bandi @mcgill ca
GERAD & Dept. Decision Sciences
HEC Montréal
Montréal, Quebec, Canada
abhilash. chenreddy@hec.com
Erick Delage
GERAD & Dept. of Decision Sciences
HEC Montréal
Montréal, Quebec, Canada
erick.delage@hee
Abstract
In this paper, we study novel approach for data-driven decision-making under uncertainty in the presence of contextual information. Specifically, we address this problem using new Conditional Robust Optimization (CRO) paradigm that seeks the solution robust optimization problem where the uncertainty set accounts for the most recent side information provided by a set of covariates. We propose an integrated framework that designs the conditional uncertainty set by jointly learning partition in the covariate data space and simultaneously constructing region specific deep uncertainty sets for the random vector that perturbs the CRO problem. We also provide theoretical guarantees for the coverage provided by conditional uncertainty sets and for the value-at-risk performances obtained using the proposed CRO model. Finally, use simulated and real world data to illustrate the implementation of our approach and compare it against two non-contextual robust optimization benchmark approaches demonstrate the value of exploiting contextual information robust optimization.
Introduction
In most real world decision problems, the decision maker (DM) faces uncertainty either in the objective function that he aims to optimize, or some of the constraints that he needs to satisfy. Stochastic Programming and Robust Optimization (RO) are the most popular methods for addressing this issue. With the growing availability of data, there has recently been a surge of interest in modeling optimization under uncertainty as contextual optimization problems that seek leverage rich feature
observations to make better decisions |ann and Rudin 2019 Bertsimas and Kallus 2020] In
simple cost minimization problem, where X C Rn and c(x, respectively capture the feasible set of actions and cost that depends on both the action a and a random perturbation vector E Rmm the "contextual" DM has access to a vector of covariates E T assumed to be correlated to &. This DM therefore traditionally wishes to identify optimal policy, i.c. functional R X' that suggests an action in X adapted to the observed realization of 1., with respect to his expected cost over the joint distribution of {):
min ([(c(=(())4))) (x-))
From theoretical point of view, one can exploit the interchangeability property (see Theorem 14.60. Rockafellar and Wets. 2009) to identify an optimal policy for Problem using the following
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

conditional stochastic optimization (CSO) problem:
(CSO)
x (+) E arrgmnn E E)
ZEX
While the literature that treats contextual optimization through the CSO problem is rich, much less attention has been given to contextual optimization the risk averse setting. Namely, one can easily think about replacing the risk neutral expected value operator in problem (2) with a risk measure such as value-at-risk or conditional value-at-risk in order to prevent the DM from being exposed to the possibility of large costs. Morcover, while robust optimization is being used pervasively in disciplines that employ decision models, including chemical, civil, electrical engineering, medicine, and physics (see respectively [Bernardo and Saraiva 1998 Bendsøe et al. 1994 Mani et al. 2006 Chu et al. 2005 Bertsimas et al. (2007] to name a few, the question of how to systematically integrate contextual information in this important class of decision models remains to this day unexplored.
In this work, we therefore tackle for the first time the contextual optimization problem from the point of view of robust optimization. Namely, we will consider a contextual DM that wishes to exploit the side information in the design and solution of a robust optimization problem. This naturally gives rise to the following conditional robust optimization (CRO) problem
xx (+) argmin max c(x, ZEX {EM(())
where U(bb is uncertainty set designed to contain with high probability the realization of conditionally on observing yh. Our proposed approach will be data-driven the sense that the design of the CRO problem will make use of historical observations of joint realizations of 1 and
Our contribution can be summarized as follows.
We propose for the first time framework for learning from data uncertainty set for RO that adapts to side information. The "training" of this conditional uncertainty set done by jointly learning partition in the covariate data space using deep clustering methods, and simultaneously constructing region specific deep uncertainty sets, using techniques from one-class classification, for the random vector that perturbs the CRO problem.
We establish theoretical connections between CRO and Contextual Value-at-Risk Optimiza- tion (CVO):
min VaR (c(x))))
(3)
where VaRi-c(2)) == inf S 2 -) refers to the value-at-risk of 1 - confidence
level of Z.
We demonstrate empirically that contextual robust optimization can improve the performance of robust optimization models in data-driven portfolio optimization problem that employs real world data from the US stock market. In particular, we find that in conditions where side information carries a strong signal about future returns, the risk of the portfolio can be reduced by up to 15%.
The paper is organized as follows. Section 2 surveys related work. Section summarizes the approach discussed in [Goerigk and Kurtz 2020]. Section 4 presents a Deep Cluster then Classify (DCC) scheme and our Integrated Deep Cluster then Classify (IDCC) scheme generate conditional uncertainty sets. It also establishes the connections to CVO. Our case study based on real world portfolio optimization presented in section 5 followed by conclusions in section
2 Related work
Conditional Stochastic Optimization HHannah et al. 2010 was possibly the earliest work on CSO, where: a kernel density estimation approach exploited formulate and solve a CSO problem. [Ban and Rudin] 2019] apply CSO to a newsvendor optimization problem where the performance of linear policies and kernel density estimation is explored and where generalization error can be controlled using regularization. [Kallus and Mao 2020 studied methods to train forest decision policies for CSO in a way that directly targets the optimization costs. [Ban et al. 2019) use residual tree methods solve general multi-stage stochastic programs where information about the underlying

---

uncertainty is available through covariate information. [Kannan et al. 20200] propose data-driven SAA frameworks for upproximating the solution to two-stage stochastic programs with access to finite number of samples of random variables and concurrently observed covariates. Recently, Lin al. [2022] has applied a conditional VaR constrained CSO formulation to the newsvendor problem. While most of the related work focuses on an
approach (see also [Srivastaval ct al., 2021b] and [Hu et al. 2022]), ) there have also been recent efforts in designing CSO models using an end-to-end paradigm (see [Elmachtoub and Grigas 2022] and [Donti et al. 2017)
Distributionally robust CSO One common challenge with the applications of CSO is due to the fact that often there are only few samples (if any at all) drawn from the conditional distribution of given for each realization of ab (Hu et al.) 2020]. This in turn causes a poor approximation of the true conditional distribution resulting in poor out-of-sample performance. Most proposed solutions to this issue have relied on distributionally robust optimization (DRO). For example, (Bertsimas) and Van Parys 2021], (Bertsimas et al. [022 Nguyen et al.) [02T], and [Srivastava et al.,) 202 all propose DRO approaches that employ distribution sets that are centered at either the estimated conditional distribution or joint empirical distribution of (v, [Kannan et al. 2020bb applies distributionally robust optimization to the residual-based CSO model proposed in [Kannan et al.)
2020a]. We finally note that none of these works have considered the problem of conditional DRO
where the distributional ambiguity set itself, namely its support or size, depending on contextual information.
Data-driven Robust Optimization and One-class Classification There has been a growing set of papers (see [Ohmori 202T, McCord 2019 Wang and Jacquillat 2020) ) proposing various frameworks that use both supervised and unsupervised one-class classification techniques in designing the uncertainty sets which are further integrated into the RO problems. Some approaches make use of variance and covariance of historical data [Natarajan et al.) [2008) while others [Goerigk and Kurtz, 2020 Wang al. 1021 have exploited the representative power of deep neural networks to construct compact uncertainty sets Up to this day, none of the data-driven robust optimization approaches have considered accounting for contextual information.
Deep Clustering Methods Traditional clustering methods like Gaussian Mixture Models (GMM) and k-means clustering rely on the original data representations and suffer from the curse of di- mensionality. Recent developments in DNNs led to the learning of high quality representations, especially auto-encoder(AA) and decoder systems are particularly appealing as they are able to learn the representations in a fully unsupervised fashion. Several works like [Chang et al.] 2017 Guo et al., 2017 Ji et al., 2017] combine variational AEs and GMMs to perform clustering and non-linearly map the input data into a latent space. Few works like [Fard et al. 2020) try to jointly learn the representations and jointly cluster with k-means and learning representations. We modify these algorithms to introduce a probability simplex that interacts with the centroids and also the center of the uncertainty sets.
3 The Deep Data-Driven Robust Optimization (DDDRO) Approach
Focusing on a classical robust optimization model, i.e. mingex max c(2, E), the authors of [Gocrigk and Kurtz [2020] propose to employ deep learning to characterize the uncertainty set U in a data-driven environment. In particular, they consider describing the uncertainty set U in the form:
u(W R) = (E E RR.. (E) Foll s R)
where Sw Rmm Rd is a deep neural network, parametrized using W. that projects the perturbation vector & to a new vector space where the uncertainty set can be more simply defined as a sphere of radius R centered at some Ju-
Given a dataset D:- = &2 ..
E. ]. they propose discovering the underlying structure of li by training the NN using method found in the one-class classification literature, namely minimizing the empirical centered total variation of the projected data points:
N (fw(E)) 2
min N
Σ
(5)
3

---

where fo:= (1/N) EiE[N] (Gi) is the center of the projected points under some initial random choice of fwo. Once the network is trained, they calibrate the radius R of U in order to reach targeted coverage e of the data set.
In terms of NN architecture, they favor special class of fully connected neural networks of depth L: fwi(c) = of
o'' (e' (c)...).
(6)
where each W captures linear projection while each of captures term-wise piecewise linear activation function (e.g. ReLU, Hardtanh, or hard sigmoid):
d((s)) = aiw, + bq. if a S wy S a k ==
with {af, 24..f. aef.,af)1 as the parameters that identifies each of the K affine pieces.
The motivation for such an architecture comes from the proposed solution scheme for the RO problem, which relies on a constraint generation approach (See Algorithm 3
4 in Appendix). This scheme relies on progressively adding scenarios reduced set U' S 2 until the worst-case cost of the solution under 4t is the same as under U. Numerically, a critical step consists in identifying the worst-case realization in 24 which shown to reduce to a mixed inteegrr linear program when c(x, {) is linear in under the selected NN architecture due to the following representation of U(W R):
Bu E {0, 1|dxKxI.
S Rdx Φ RRdx
Σ u k, 1, Vj,
W/
u(W, R)
E
CJ Lk=1
K
uj kel of + =E u
Vj.l
W'C'-1, WE2
Σ J 01 S of < E "
ag
ag. Vj.l
1 foll S
where we assume for simplicity that each layer of the deep neural network has d neurons and o' the output I-th layer of the neural network. We refer interested readers [Goerigk and Kurtz 2020 for more details.
Deep Data-driven Conditional Robust Optimization
Let (v),)) be pair of random vectors defining respectively the side-information and random perturbation vectors of a contextual optimization problem. We can call our dataset Deg := ((11, (N), Our objective is to train data-driven conditional uncertainty set (1b) that will lead to robust solutions that are adapted to the type of perturbance that experienced when is observed. In this section, we propose (DCC) and the Integrated Deep cluster then classify (IDCC), to do so, and propose a calibration procedure that offers some guarantees with respect to contextual value-at-risk problem.
algorithms, namely the Deep cluster then classify
4.1 The Deep "Cluster then Classify" (DCC) Approach
direct extension of G&K's DDDRO approach in section consists in redueing the ande-information s to set of K different clusters, which provides states of the environment in which one wishes to design customized data-driven uncertainty sets. Mathematically, ((+)) : Ma(wy where a Im. - [K]. is a trained K-class cluster assignment function for vh, and each UR for k 1, 1..... K, is an uncertainty sets for that is trained and sized using the procedure described in section 3 with the dataset D := (oo
This process implicitly involves multiple sequential steps of training deep neural networks. Following [Moradi Fard et al., 2020), when performing deep K-mean clustering to obtain a(t), training can take the form of A Agrrithm where the deep K- means algorithm trains simultaneously a representation OV&- R. - Rdd using an encoder and gVv : IRd + RT, using a decoder network, and K-mean classifier a (o) := argmin 1|0-0* l|2 by minimizing, using stochastic gradient descent coordinate descent scheme, a trade-off (using ax) between reconstruction error and the within cluster centered total variation in the encoded space:
N
(V. = (1 - -ak)
1
1 lev. (ov, vb.12 112 + +aK N
lev (sha) -a(0)))12
(8)
N

---

where a(v) := 2 (gv (v)). To solve this problem, we iterate between improving V = (VE, VD) while keeping 9 fixed, and improving 9 while preserving V fixed.
Once the -mmean and one-class classifiers are trained, we correct for a deficiency of DDDRO approach, which assumes wrongfully that the projected fwx are normalized for each Dr. Namely, we replace U(W, R) with a that employs an ellipsoid in the projected space according to the statistics of :
u(W* S*, :=))= (E ER" |2g 1/2 (Sw n/)il S R²),
(9)
where 5he is short for (u'g. En) with
2 = 1D
Σ CEDE
fwy(S) (E) and プ = p (fwx)()) (4 (fw=((E)) - =²²
EED
The calibration of each R² can finally be done using the same procedure as in [Goerigk and Kurtz 2020 but using the reduced dataset D..
4.2 The Integrated Deep Cluster-Classify (IDCC) Approach
While the simplicity of the approach presented in section 4.1 makes appealing, we identify two important weaknesses. First, by separating the training into multiple steps, it omits tackling the conditional uncertainty set learning problem as a whole. Namely, that low total variation in the vi space (or projection of it) does not necessarily imply that low total variation can easily be achieved projection of the & space. Second, it is unclear how adapt the approach a context where a clear separation of the clusters is impossible and where the notion of partial membership to cluster more appropriate.
To address the first problem. we propose an integrated framework for performing deep clustering and deep uncertainty set design jointly. Namely, we propose optimize all of V. A, and {W*) kk= jointly using loss function that trades-off between the objectives used for clustering and each of the K versions of one-class classifiers. We also tackle the issue of hard assignments by training parameterized random assignment policy : Rm - Ak. where AK is the probability simplex in R. and the parameters that define the policy space. In the context of employing soft version of deep K K-means [Fard et al. 2020]; this random assignment policy takes the form of T(Φ) == # (gv
where
(b) (v) 0
(1b)
(10)
ΣK k 1xp(-3Bllv[(e - 0% |2]
With these adjustments, our proposed loss function takes the form of:
C. (W*]K =as (1 ax llblllv (ove (v)) - whc [|2]
+ a ES Var (v)) gā(w) (a(+))1)
(a(v)
K
(1 as)
Σ
min Total (E), 0² (ā((b) k))
(11)
K
where ā(v) ~ # (9ve (b)) is the randomized assignment based on s), Total 0lā(b)) = ES ET (0b; - 8,, (,))(((())) is the conditional centered total variation of given a(v). In fact, all
statistics are measured using the empirical distribution expressed in Die and the conditional dis- tribution produced by the randomized assignment policy # (gv (g))))),, i.c. PS((V,,,0 e E)
K (1/N) EN K1 1((1), &,, k)) E (vx)). The explicit form of equation (11) can be found in Appendix B.
Overall, C3 trades off (using o:ş) between the reconstruction error of the encoder-decoder networks on E, the expected recognizability of the K clusters, i.e. the fact that the observed features gV3 (15) form distinct clusters of points, and the average compactness of the produced conditional uncertainty
5

---

sets. In particular, as as - 1, we can expect the minimizer of La to converge to the minimizer of the cluster and classify approach. At the other end of the spectrum, when aş + 0, the model will produce more self contained conditional uncertainty sets but at the price of less distinguishable clusters (in terms of 1)) that might poorly exploit the side-information. Algorithm presents our proposed training scheme for the IDCC approach.
Given that we employ random assignment policy, we propose replacing the deterministic CRO problem with its randomized version:
(v) E argmin max {), #EX geu(s)
where ((b) == U(Wa(+), Ra(0), ( is random uncertainty set, and where we express the fact that conditionally on v. (()) is random policy that depends on the realization of a. Given the randomness of (1b), one needs to more careful in defining a calibration scheme each R². Our
proposed scheme is motivated by the following Lemma, which proof can be found in appendix A
Lemma 4.1. Let the random uncertainty set (1b) satisfy:
(E E ((v))a(v)) k) 2 1, Ak,
(12)
then it satisfies:
IE E ((})) 21 -
(13)
In particular, this lemma suggests calibrating each R using the bisection to solve:
EN (E, E E(W,, R, R, )T (v+))
2 =1-
g.
inf
R
(14)
E (ove (w))
given that the resulting R are the smallest that satisfy 12
Algorithm Integrated deep cluster claasify with deep K -means
Data-set Deç. Number of clusters K, hyper- parameerrssss ag,, a.s, B Randomly initialize Co. Vo. and Wo
Input:
Let mT == #0 (gVeo (v)) and W = Wo for all k's
Set :: 0
repeat
Set b= :=
Update 0f == 1T (V) (V)))()) = k] using Wt-1
Update (V,, (W+) K 1k=1) using gradient descent on with 0
Get π.. == (gVE. (b))
until > T' or convergence
Let (()) == x+(-) and W == W for all kts
for
1, 1..... K do
Calibrate R² using 14
Let th : U(W
end for
Return π(.) and (U ]==
4.3 Connections to Contextual Value-at-Risk Optimization
In the previous subsections, we proposed two different schemes to produce possibly randomized uncertainty set (*bb that can be employed in randomized CRO problem We also proposed
Here, 8h refers V
(F
6 ) with
(((on (Ex)(E)) F.. (Ww
5 Wo =
PB.V (
En 7 (ove
NNote that in the case of section randomized version of CRO to pure CRO problem
4.1 the conditional uncertainty set is deterministic thus reducing the

---

scheme for radii calibration so that they would satisfy the coverage property in (3) Hence, one can derive the following connection between conditional robust optimization and the CVO problem ( The proof is pushed to Appendix A
Lemma 4.2. When Ut satisfies with
the random policy at.) to the randomized CRO problem together
esssupt min max c(x,E) zEX (EU())
provide a conservative approximate solution to the CVO problem under the empirical measure PT Namely,
VaR -e (c(((()))))))))))))))))) S
In particular, in the case of the proposed DCC and IDCC approaches we have that
max min
max c(x,
kE|K] ZE &EM(W*,R*...
As the robust optimization paradigm traditionally aims at offering statistical guarantees on the out-of- sample performance of the prescribed solutions, we describe below how a bootstrap method can be used to estimate the radii '
Remark 4.1. Using bootstrapping methods, we can get a conservative approximation of each Rg as:
#B (tt) 1(& E u(W" R. (2 (2:22110) (ove (v))
RR inf RRED
Σ E I
where Fd measures the probabibility when resampling a new dataset of size N with replacement from Dus. When N is large enough and assuming that each data point is drawn i..d according to
some unknown probability measure P, we asymptotically get the guarantee that IP( ((})) 2 1-c with probability higher than approximately 1 - K5.
5 Experiments
In this section, we illustrate the coverage aspect of the IDCC approach using simulated data. We will further demonstrate the advantage of the CRO problem using a standard risk minimizing portfolio optimization problem. We compare the performance of IDCC with that of DCC, DDDRO (with cllipsoidal correction in (9). and the classical ellipsoidal uncertainty approach (i.c. DCC with K = 1 and (E) = E). The IDCC and DCC methods incorporate the covariate information whereas DDDRO and ellipsoid approaches ignore this information. The neural network architecture and other modeling information are available in Appendix B The code can be found on githut Our code uses the Pytorch implementation from [Goerigk and Kurtz 2020], which is available online"
5.1 Conditional uncertainty set illustration using simulated data
For ease of illustration, we consider simulation environment where T 5T) R- is random vector whose distribution is an equal-weighted mixture of two 4-d multivariate normal distributions. We consider N 500 and train IDCC (with K = 2), DDDRO, and the ellipsoid and calibrate the uncertainty sets for a probability coverage of 90%, 99% (i.e. e E (1%, 10%]). As a result, DDDRO and IDCC, which use deep neural networks, identify non-convex uncertainty sets, whose convex together with the calibrated ellipsoid.The figure also presents the
hulls are presented in Figure [ conditional distribution of according to P (-1ā(1)) = kc), using IDCC's randomized assignment, and the training dataset. One can remark that the conditional sets produced by IDCC exploit the side information by concentrating the uncertainty set on the region that has the most mass according to (- (+)ā())) = k) thus leading to a less conservative RO problem then DDDRO and the ellipsoid, which oblivious to th. In fact, it appears to have successfully learned to least partially recognize the mixture membership using & and exploit this information to adapt the uncertainty set.
httpe://anonymous wwtpeeeeeeeeeeeeeeeeeeeeeeeeeeee Deeraa-Diveen--eee Drivenn-oondiiionaa htt.:/ooawnttouuuuooottiiiiiiiia E160/
https://github. htw.ecubb.oooggoogg////000-----.
7

---

E1 (a) R(o)) = 1, 90% cowerage
61
E1
(b) A(v)
(c) a(v) = 2, 90% coverage
(d) R(ml) = 2.
99% coveraee
99% coverage
IDCC
DDDRO
Eliipoid
Figure 1: Convex hull of trained uncertainty sets for two levels of coverage and with a conditional uncertainty set for IDCC that exploits two clusters. The heatmap represents the conditional distribution of & according to P (-1ā(1b) = k). The cloud of points represents the training dataset.
5.2 Robust portfolio optimization
We further investigate the empirical out-of-sample performance of the proposed uncertainty sets on classical robust portfolio optimization problem. Namely, we consider a situation where an investor trying to minimize the worst-case return based on an uncertainty set that provides probabilistic coverage of the uncertain future return vector. In particular, given that captures vector of investment
m different assets whose return are captured using E. we let c(x,,)) == - to capture the return investment, and let X' := {I E R" Σ ii=1 I; 1, 0} to capture the need to invest one unit of wealth among the available assets. Following Lemma [4.2 this model can in turn be interpreted as conservatively approximating a min Va TT where the objective is risk averse value-at-risk metric.
Dataset Our experiments make use of historical data from the US stock market. We collect the adjusted daily closing prices for 70 stocks (as used in [Xu and Cohen 2018) coming from 8 different sectors from January 1, 2012, to December 31, 2020, using the Y!Finance's ' API. Each year has 252 data points and we compute the percentage gain/loss w.r.t the previous day to create our dataset for As for side information, we use the trading volume of individual stocks and other market indice over the same period as covariates. Our algorithm gives the flexibility to use any number of such metrics as contextual information. Given the time series nature of the data, at given instance, use years of data to train and the following year as validation to pick the hyperparameters of our model such learning rate, weight decay, and the optimal number of clusters. We then retrain the model using the 4 years of data to build the final model. Upon calibrating the uncertainty set, we use it to solve the robust portfolio optimization problem. We then apply this policy to the next 1 year's of data and compute the performance metric, namely Value at risk (VaR) for different confidence levels to compare the performances. VaR quantifies the level of risk of portfolio over specified time frame. Here, gives an estimate of the maximum % loss the decision maker can incur over a period of 1 year when he uses the policy from the RO model. Intuitively, lower the VaR, less riskier is the generated policy. Many financial institutions use VaR to determine the amount of collateral needed when trading financial products so lowering VaR for high confidence levels is crucial.
Experiment Design To test for the robustness of the IDCC algorithm, we experiment on various randomly sampled stock combinations across different time periods. We randomly sampled subset of 15 stocks in time window and repeated the experiment for 10 runs on moving time frames. We used learning rate 0.01, ag- 0.5, as 0.5, 3 0.1 for all the experiments. We use a cold start K-means approach to determine K for each run. We do this across all these experiments as will be computationally expensive tune the parameters through grid search for each and also intention to show the learning capability of our algorithm even with minimal tuning. The parameter tuning and implementation details be found in appendix B.3
VVoltility Index (VIX), 10-year Treasury Yield Index (TNX), Oil Index (CL-F), S&P 500 (GSPC), Global Income & Currency Fund (XGCFX), Dow Jones Index (DJI)

---

Aog.
05-
FI1
Confidence lovel
Confidence
(a) 2017
(b)
(C) 2019
IDC
DCC
DDDRO
Filipaoid
Figure 2: Avg. VaR across portfolio simulations. Error bars report 95% CL.
Results Fig. 2 shows the avg. VaR across the runs at different confidence levels. It is evident that IDCC generally performs better than the baseline models. This difference is especially noticeable at higher confidence level and vanishes as we move to lower confidence levels. [ablle[]roovde more details by comparing the overall and conditional cluster level VaR with the baseline models. Specifically, in each run, we identify each cluster either the "majority" or "minority" cluster depending on its frequency and report averages of VaR (among the 10 runs) for each of these labels. The average frequencies for each label are also reported in the table. In particular, one can observe that the improvement on average overall VaR can reach up to 155% (see in 2019 at a 0.99 confidence clearly visible when look at the individual cluster-level
level). This advantage is even conditional VaR. For instance, the year 2018 for the 0.99 confidence level, the majority cluster (~68% data) provides improvement of 19% and overall improvement of 9% compared the second best baseline model. A similar pattern is observed for the year 2019 as well. In the year 2017, the overall performance of IDCC is close and for models. However, we see that the majority cluster (~80% data) is performing better than the baseline models while the minority cluster has slightly higher risk. We attribute this loss in performance to the fact that the minority clusters are much less frequent (~200% data) and therefore have fewer data available to properly learn its conditional uncertainty This large difference in frequencies might also indicate that the side information does not have a strong signal for the behavior of the returns during this period of time.
confidence levels slightly above the baseline
2017
2018
2019
Conf. 1
0.8
0.9
0.95
0.99
0.8
0.9
0.95
0.99
0.8
0.9
0.95
0.99
IDCC
0.30
0.55
0.75
1.37
0.64
1.16
1.67
2.86
0.44
0.77
1.11
2.02
Overall
DDDRO
0.31
0.52
0.79
1.46
0.63
1.24
1.84
3.17
0.45
0.84
1.27
2.35
Ellipsoid
0.30
0.49
0.75
1.45
0.72
1.45
2.04
3.19
0.47
0.81
1.30
2.52
Cond. on
Cluster Freq.
80%
68%
59%
Majority
IDCC
0.52
1.30
0.57
1.08
1.50
2.62
0.44
0.75
1.17
1.88
Cluster
DDDRO
0.31
0.52
0.74
1.35
0.59
1.15
1.63
3.23
0.45
0.85
1.31
2.06
Ellipsoid
0.32
0.52
0.74
1.41
0.69
1.29
1.92
3.08
0.47
0.85
1.25
2.31
Cond. on
Cluster Freq.
20%
32%
41%
Minority Cluster
IDCC
0.30
0.61
0.77
1.43
0.96
1.57
2.05
3.13
0.48
0.82
1.15
2.22
DDDRO
0.30
0.56
0.84
1.39
1.00
1.66
2.04
3.30
0.49
0.84
1.40
2.39
Ellipsoid
0.28
0.47
0.69
1.13
1.17
1.80
2.43
3.43
0.49
0.82
1.38
2.57
Table 1: Comparison of average value-at-risk (over 10 runs) for different levels of probability coverage. Both the overall VaR and conditional VaR given the membership to the majority/minority clusters are presented.
Conclusion and Future Work
In this work, we introduced new approach, Conditional Robust Optimization, for solving contextual optimization problems in risk averse setting. We proposed novel integrated approach to design uncertainty sets that adapt to revealed covariate information. We identified connections to contextual value-at-risk optimization and showed empirically that method reduces the out-of- VaR

---

considerably compared to non-contextual RO schemes when the level of protection needed is high. As future work, we find that it should be interesting to integrate data-driven conditional uncertainty sets in the context of multi-stage robust optimization models. Given that clustering techniques are often prone to learning correlations from the data that not reflect true causal relations, so there might be need to integrate causal inference methods into our approach. One might also be concerned regarding fairness considerations contexts where side information might allow to treat of a certain class of individuals differently from others. This last issue might be addressed by adding fairness consideration in our integrated loss function.
Acknowledgement
The authors gratefully acknowledge support from the Institut de Valorisation des Données (IVADO) and from the Canadian Natural Sciences and Engineering Research Council [Grant RGPIN-2016 05208 and 492997-2016).
References
Gah-Y Yi Ban and Cynthia Rudin. The big data newsvendor: Practical insights from machine learning. Operations Research, 67(1):90-108 2019.
Gah-Yi Yi Ban, Jérémie Gallien, and Adam J Mersereau. Dynamic procurement of new products with covariate information: The residual tree method. Manufacturing & Service Operations Management, 21(4):798 815, 2019.
M. P. Bendsøe, A. Ben-Tal and J. Zowe. Optimization methods for truss geometry and topology design. Structural optimization. 7(3):141-159, 1994.
Fernando P. Bernardo and Pedro M. Saraiva. Robust optimization framework for process parameter and tolerance design. AIChE Journal, 4409):2007-2117, 1998.
Dimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Management Science, (663):1025-1044 2020.
Dimitris Bertsimas and Bart Van Parys. Bootstrap robust prescriptive analytics. Mathematical Programming, pages 1-40, 2021.
Dimitris Bertsimas, Omid Nohadani, and Kwong Meng Teo. Robust optimization in electromagnetic scattering problems. Journal of Applied Physics, 101(7):074507, 2007.
Dimitris Bertsimas, Christopher McCord, and Bradley Sturt. Dynamic optimization with side information. European Journal of Operational Research, 2022.
Jianlong Chang, Lingfeng Wang, Gaofeng Meng. Shiming Xiang, and Chunhong Pan. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, pages 5879-5887, 2017.
Millie Chu, Yuriy Zinchenko, Shane G Henderson, and Michael B Sharpe. Robust optimization for intensity modulated radiation therapy treatment planning under uncertainty. Physics in Medicine and Biology, 50(23):5433-547, nov 2005.
Priya L. Donti, Brandon Amos, and J. Zico Kolter. Task-based end-to-end model learning. CoRR, 0e//1773.05522 2017.
Adam N Elmachtoub and Paul Grigas. Smart "predict, then optimize". Management Science, 68(1): 9-26, 2022.
Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier. Deep k-means: Jointly clustering with k-means and learning representations. Pattern Recognition Letters, 138:185-192, 2020.
Mare Goerigk and Jannis Kurtz. Data-driven robust optimization using unsupervised deep learning. arXiv preprint arXiv:2011.09769 2020.
10

---

Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In Ifeai. pages 1753-1759, 2017.
Lauren Hannah, Warren Powell, and David Blei. Nonparametric density estimation for stochastic optimization with an observable state variable. In Lafferty, C. Williams, Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23, Curran Associates, Inc., 2010.
Yichun Hu, Nathan Kallus, and Xiaojie Mao. Fast rates for contextual linear optimization. Manage- ment Science, 2022.
Yifan Hu, Siqi Zhang, Xin Chen, and Niao He. Biased stochastic first-order methods for conditional stochastic optimization and upplications in meta learning. Advances in Neural Information Processing Systems, 33:2759-22 2770, 2020.
Pan Ji, Tong Zhang, Hongdong Li, Mathieu Salzmann, and Ian Reid. Deep subspace clustering networks. Advances in neural information processing systems, 30, 2017.
Nathan Kallus and Xiaojie Mao. Stochastic optimization forests. arXiv preprint arXiv:2008.07473, 2020.
Rohit Kannan, Güzin Bayraksan, and James R Luedtke. Data-driven sample average approximation with covariate information. Optimization Online. URL: http://www. optimization-onlinc org/DB HTMLL20200077932. html, 2020a.
Rohit Kannan, Güzin Bayraksan, and James R Luedtke. Residuals-based distributionally robust optimization with covariate information. arXiv preprint arXiv:2012.01088, 2020b.
Shaochong Lin, Youhua (Frank) Chen, Yanzhi Li, and Zuo-Jun Max Shen. Data-driven newsvendor problems regularized by profit risk constraint. Production and Operations Management, 31(4): 1640,1644, 2022.
Murari Mani, Ashish K. Singh, and Michael Orshansky. Joint design-time and post-silicon minimiza- tion of parametric yield loss using adjustable robust optimization. In 2006 IEEE/ACM International Conference on Computer Aided Design, pages 19-26, 2006.
Christopher George McCord. Data-driven dynamic optimization with auxiliary covariates. PhD
thesis, Massachusetts Institute of Technology, 2019.
Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier. Deep k-means: Jointly clustering with k-means and learning representations. Pattern Recognition Letters, 138:185-192, 2020.
MOSEK ApS. MOSEK Fusion API for C++ 9.3.20, 2022. URL https://docs mosek. com/
latest/ l wttpc/cconoooinneeeeeeeeeoe html
Karthik Natarajan, Dessislava Pachamanova, and Melvyn Sim. Incorporating asymmetric distri-
butional information in robust value-at-risk optimization. Management Science, 54(3):573-585,
2008.
Viet Anh Nguyen, Fan Zhang, Jose Blanchet, Erick Delage, and Yinyu Ye. Robustifying conditional portfolio decisions via optimal transport, 2021.
Shunichi Ohmori. A predictive prescription using minimum volume k-nearest neighbor enclosing ellipsoid and robust optimization. Mathematics, 9(2):119, 2021.
Thomas Page Jr. Multivariate statistics: A vector space approach. JMR, Journal of Marketing Research (9re-1986), (1000000)) 1984.
Tyrrell Rockafellar and Roger J-B Wets. Variational analysis, volume 317. Springer Science & Business Media, 2009.
Prateek R. Srivastava, Yijie Wang, Grani Hanasusanto, and Chin Pang Ho. data-driven prescriptive analyties with side information: A regularized nadaraya-watson approach, 202
11

---

Prateek R Srivastava, Yijie Wang, Grani A Hanasusanto, and Chin Pang Ho. On data-driven prescriptive analyties with side information: A regularized watson approach. arXiv preprint 1r.xiv:211004488. 2021b.
Cong Wang, Xin Peng, Chao Shang, Chen Fan, Liang Zhao, and Weimin Zhong. A deep learning- based robust optimization approach for refinery planning under uncertainty. Computers Chemical Engineering, 155:107495, 2021.
Kai Wang and Alex Jacquillat. From classification to optimization: A scenario-based robust opti- mization approach. Available at SSRN 3734002, 2020.
Yumo Xu and Shay B Cohen. Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1979, 2018.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] 6
(c) Did you discuss any potential negative societal impacts of your work? [Yes] c
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] A
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]] 5
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [TODÓ]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work existing assets, did you cite the creators? (Yes) https: ttttp:///githr
com/
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental mate- rial or as a URL? [Yes] https:// 4open. science/r/ Data--riieen-Condtiion
htt.---buutttttttmatttoo-o------ -at.aadittttot-oott------- Opt.miiatiionnnnnnni E160/README. md
(d) Did you discuss whether and how consent was obtained from people whose data you're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "Qr3UrUWi053", "reviewer": "Reviewer_k1N2", "summary": "The authors study risk aversion in a data-driven problem with contextual data using robust optimization. They extend previous work that uses deep neural networks to build unconditional uncertainty sets. They develop conditional uncertainty sets that leverage contextual data. Building conditional uncertainty sets for a new observation is based on two steps: (1) assigning the new observation to a data cluster, and (2) creating a local unconditional uncertainty set for the cluster. The authors present two methods to build these clusters based on minimizing total variation. The second method is a generalization of the first method and allows a trade-off minimizing variations in the feature space and disturbance space.\n\nThe performance of the method is demonstrated on a portfolio optimization problem. The results show that the integrated data-driven approach (IDDC) provides the best risk reduction for high-risk aversion, and when data can be clustered in two clusters that are relatively balanced.", "questions": "Exposition:\n\n(Q1.1) What is the intuition/motivation for minimizing total variations of features/disturbances when building uncertainty sets? in particular: why should this lead to more robust decisions than elliptical or polyhedral uncertainty sets?\n\n(Q1.2) The approach seems limited to linear uncertainty in the cost function. Could this be extended to a cost function that is non-linear in the uncertainty (this happens already for simple problems e.g. newsvendor with uncertain demand) or to uncertainty in the constraints?\n\n(Q1.3) Is my understanding correct that the uncertainty sets are \"constant\" within each cluster? Could this be improved? There seems to be a trade-off between having many clusters using few data points but being more adaptive, and using few clusters but maybe not using the contextual data to its fullest extent. I think a sensitivity analysis of the number of clusters and its impact on decisions and costs could be interesting.\n\n(Q1.4) I do not understand the statement regarding the \"deﬁciency of DDDRO approach, who assume wrongfully that the projected fWk(ξ) are normalized\" (p.4, lines 161-162). Could you explain it in more detail?\n\n(Q1.5) Could you develop the intuition behind the first drawback of the DCC approach? (p. 5, lines 171-121) This is probably related to (Q1.1).\n\n(Q1.6) The statement \"as α_S → 1, we can expect the minimizer of L_α^3 to converge to the minimizer of the cluster then classify approach\" seems unnecessarily vague (p.5, line 194). It seems that α_S = 1 reduces exactly IDCC to DCC. Is this correct? I would also be interested in seeing a sensitivity analysis of α_S and how it impacts decisions and costs.\n\n(Q1.7) The cluster assignment is a random policy. Is this a problem in practice? Random decisions may be hard to use and accept by practitioners. Why not use the most likely cluster?\n\nNumerical study:\n\n(Q2.1) Why is there no benchmark from recent literature outside of DDDRO on which the work is based? Comparing with a data-driven benchmark without risk consideration (e.g. Bertsimas & Kallus (2019) or Bertsimas et al. (2022) who also use ideas from robust optimization) and for instance with a distributionally robust benchmark without conditional information could provide a more precise idea of the value of the approach. Bertsimas & Kallus (2019) could be an easily implementable benchmark: it uses a Machine Learning model to weight historical data and provides a conditional empirical probability distribution. The (conditional) value-at-risk portfolio optimization model could then be optimized using a sample-average method. This approach has been discussed by Lin et al. (2022) for a risk-averse newsvendor problem.\n\n(Q2.2) Is the number of replications sufficient (N=10) to highlight a statistical difference between the performance of the methods (especially since IDCC is a random policy)? What do the error bars in Figure 2 show (confidence interval, standard deviation)?\n\n(Q2.3) What are the majority and minority clusters? This suggests that only K=2 clusters are used, but it is said on line 269 that the number of clusters is optimized.\n\n(Q2.4) Is \\epsilon used to build the uncertainty sets radii the same as when evaluating VaR_\\epsilon? If (I)DCC is a conservative approximation of Var(\\epsilon) then maybe the best value of \\epsilon to build the sets is smaller than the actual target. I would also be interested in seeing the effect of \\epsilon on the trade-off between average costs and value-at-risk.\n\nMinor comments:\n- I cannot find the footnote relating to note [1] on page 1.\n- I believe \\alpha_K and \\alpha_S are not properly introduced.\n- I believe that when Algorithms 2 and 3 are referenced in the main paper, it should be Algorithms 3 and 4 resp.\n\nReferences:\nDimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Management Science, 66(3):1025–1044, 2020.\nDimitris Bertsimas, Christopher McCord, and Bradley Sturt. Dynamic optimization with side information. European Journal of Operational Research, 2022.\nLin, Shaochong, Youhua Chen, Yanzhi Li, and Zuo‐Jun Max Shen. Data‐Driven Newsvendor Problems Regularized by a Profit Risk Constraint. Production and Operations Management 31(4):1630-1644, 2022.", "limitations": "I find the work to be very complete. I see only two significant limitations that are already detailed in the question part: (1) the lack of benchmark and comparison to recent literature, and (2) not discussing how to extend the approach to problems where the cost function is not linear in the uncertainty.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper provides an interesting approach to a well-motivated and significant problem. I find the methodology sound and well presented. In particular, the approach leverages very well results from diverse fields such as clustering, outlier detection, robust optimization, and mixed-integer linear optimization.\nMy main concern relates to the experimental setting, in particular the lack of benchmark from recent and relevant literature. I also have several questions/suggestions regarding the exposition of the method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8VFXUX1SZqi", "reviewer": "Reviewer_34vX", "summary": "In this paper, the author has proposed a Conditional Robust Optimization (CRO) paradigm that seeks the solution to a robust optimization problem using contextual information. The uncertainty set accounts for the most recent side information provided by the set of covariates.\nThe conditional uncertainty is designed with an integrated framework (IDCC) that jointly learns a partition in the covariate data space and simultaneously constructing region specific deep uncertainty sets for the random vector that perturbs the CRO problem. The paper tested the\nIDCC algorithm using the US stock market dataset, and shows empirically that their approach reduces the out-of-sample VaR considerably compared to non-contextual RO schemes when the level of protection needed is high.", "questions": "1. Use of more real or synthetic datasets to get a better generalization of the approach in various settings, and to clarify some of the claims, such as the fact that the loss in performance for minority clusters is due to its low frequency which makes it difficult to\nlearn the conditional uncertainty sets.\n2. It might be nice to have the algorithms for the DCC also mentioned to help us understand the differences between DCC and IDCC, as well as the individual approaches, more intuitively.\n3. Here the author has used K-means clustering, which works very well for smaller data, but can take very long to run for large datasets, since it iterates over the entire dataset. It can be useful to try out larger datasets using density based clustering methods such as\nDBSCAN. Further, it might be useful to try out some soft clustering methods like fuzzy k-means or fuzzy c-means for partial clustering.\n4. A mention of the run-time of each of the experiments might give a better idea of the comparisons of the various methods, unless their run-times are similar.\n5. Are the above algorithms useful only for time-series data? Can we extend these algorithms to perhaps a computer vision system? If yes, it would be nice to generalize the approach in various different settings either through a short description or more\nvariety of datasets.\n6. Can the authors mention more explicitly the limitations of the proposed algorithm?", "limitations": "\nTied to K-Means Clustering although it's clear now that the clustering algorithm can be tailored.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. CRO is a novel approach that utilizes the contextual information. It reduces the out of sample value at risk considerably compared to the conventional non-contextual optimization methods.\n2. The paper corrects a wrong assumption made previously by the authors of the DDDRO approach that the projected perturbation vector is normalized for each data, by replacing the simply defined uncertainty set that uses an ellipsoid in the projected space instead of\na sphere.\n3. The IDCC framework performs deep clustering and deep uncertainty set design jointly which ensures that the uncertainty set is learnt correctly, by ensuring that low variation in the side information vector results in low variation in the perturbation vector.\n4. Deals well with partial clustering situations by training a parameterized random assignment policy.\n5. DDDRO and IDCC, which use deep neural networks, are capable of identifying non-convex uncertainty sets.\n6. IDCC exploits the side information by concentrating the uncertainty set on a region that has most mass thus leading to a less conservative optimization problem than DDDRO and ellipsoid.\n7. The paper is well documented in terms of the details for the experiments with no significant ambiguities.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4CgxU89hqPs", "reviewer": "Reviewer_8P9p", "summary": "This paper addresses the robust optimization problem with the covariates provided and provides data-driven solutions. It proposes an integrated framework for performing deep clustering and deep uncertainty set design jointly to ensure the total variation in the covariate space can provide a similar guarantee for the one in the perturbation parameters. Moreover, they rigorously prove the connections between conditional robust optimization and the CVO problem. Simulation data and robust portfolio optimization using real-world data from the US market are provided for empirical evidence.", "questions": "None.", "limitations": "Cannot find any limitations about this paper.", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is the first to introduce the conditional robust optimization framework for solving contextual optimization problems in a risk-averse setting based on the classical robust optimization approach but with novel development, i.e., an integrated approach to design the uncertainty sets adapted to the covariate information. The story is complete and the analysis is comprehensive. The problem is well-motivated and the solutions are sound and concrete.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept