Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors study risk aversion in a data-driven problem with contextual data using robust optimization. They extend previous work that uses deep neural networks to build unconditional uncertainty sets. They develop conditional uncertainty sets that leverage contextual data. Building conditional uncertainty sets for a new observation is based on two steps: (1) assigning the new observation to a data cluster, and (2) creating a local unconditional uncertainty set for the cluster. The authors present two methods to build these clusters based on minimizing total variation. The second method is a generalization of the first method and allows a trade-off minimizing variations in the feature space and disturbance space.

The performance of the method is demonstrated on a portfolio optimization problem. The results show that the integrated data-driven approach (IDDC) provides the best risk reduction for high-risk aversion, and when data can be clustered in two clusters that are relatively balanced.
strengths_and_weaknesses: The paper provides an interesting approach to a well-motivated and significant problem. I find the methodology sound and well presented. In particular, the approach leverages very well results from diverse fields such as clustering, outlier detection, robust optimization, and mixed-integer linear optimization.
My main concern relates to the experimental setting, in particular the lack of benchmark from recent and relevant literature. I also have several questions/suggestions regarding the exposition of the method.
questions: Exposition:

(Q1.1) What is the intuition/motivation for minimizing total variations of features/disturbances when building uncertainty sets? in particular: why should this lead to more robust decisions than elliptical or polyhedral uncertainty sets?

(Q1.2) The approach seems limited to linear uncertainty in the cost function. Could this be extended to a cost function that is non-linear in the uncertainty (this happens already for simple problems e.g. newsvendor with uncertain demand) or to uncertainty in the constraints?

(Q1.3) Is my understanding correct that the uncertainty sets are "constant" within each cluster? Could this be improved? There seems to be a trade-off between having many clusters using few data points but being more adaptive, and using few clusters but maybe not using the contextual data to its fullest extent. I think a sensitivity analysis of the number of clusters and its impact on decisions and costs could be interesting.

(Q1.4) I do not understand the statement regarding the "deﬁciency of DDDRO approach, who assume wrongfully that the projected fWk(ξ) are normalized" (p.4, lines 161-162). Could you explain it in more detail?

(Q1.5) Could you develop the intuition behind the first drawback of the DCC approach? (p. 5, lines 171-121) This is probably related to (Q1.1).

(Q1.6) The statement "as α_S → 1, we can expect the minimizer of L_α^3 to converge to the minimizer of the cluster then classify approach" seems unnecessarily vague (p.5, line 194). It seems that α_S = 1 reduces exactly IDCC to DCC. Is this correct? I would also be interested in seeing a sensitivity analysis of α_S and how it impacts decisions and costs.

(Q1.7) The cluster assignment is a random policy. Is this a problem in practice? Random decisions may be hard to use and accept by practitioners. Why not use the most likely cluster?

Numerical study:

(Q2.1) Why is there no benchmark from recent literature outside of DDDRO on which the work is based? Comparing with a data-driven benchmark without risk consideration (e.g. Bertsimas & Kallus (2019) or Bertsimas et al. (2022) who also use ideas from robust optimization) and for instance with a distributionally robust benchmark without conditional information could provide a more precise idea of the value of the approach. Bertsimas & Kallus (2019) could be an easily implementable benchmark: it uses a Machine Learning model to weight historical data and provides a conditional empirical probability distribution. The (conditional) value-at-risk portfolio optimization model could then be optimized using a sample-average method. This approach has been discussed by Lin et al. (2022) for a risk-averse newsvendor problem.

(Q2.2) Is the number of replications sufficient (N=10) to highlight a statistical difference between the performance of the methods (especially since IDCC is a random policy)? What do the error bars in Figure 2 show (confidence interval, standard deviation)?

(Q2.3) What are the majority and minority clusters? This suggests that only K=2 clusters are used, but it is said on line 269 that the number of clusters is optimized.

(Q2.4) Is \epsilon used to build the uncertainty sets radii the same as when evaluating VaR_\epsilon? If (I)DCC is a conservative approximation of Var(\epsilon) then maybe the best value of \epsilon to build the sets is smaller than the actual target. I would also be interested in seeing the effect of \epsilon on the trade-off between average costs and value-at-risk.

Minor comments:
- I cannot find the footnote relating to note [1] on page 1.
- I believe \alpha_K and \alpha_S are not properly introduced.
- I believe that when Algorithms 2 and 3 are referenced in the main paper, it should be Algorithms 3 and 4 resp.

References:
Dimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Management Science, 66(3):1025–1044, 2020.
Dimitris Bertsimas, Christopher McCord, and Bradley Sturt. Dynamic optimization with side information. European Journal of Operational Research, 2022.
Lin, Shaochong, Youhua Chen, Yanzhi Li, and Zuo‐Jun Max Shen. Data‐Driven Newsvendor Problems Regularized by a Profit Risk Constraint. Production and Operations Management 31(4):1630-1644, 2022.
limitations: I find the work to be very complete. I see only two significant limitations that are already detailed in the question part: (1) the lack of benchmark and comparison to recent literature, and (2) not discussing how to extend the approach to problems where the cost function is not linear in the uncertainty.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper addresses the robust optimization problem with the covariates provided and provides data-driven solutions. It proposes an integrated framework for performing deep clustering and deep uncertainty set design jointly to ensure the total variation in the covariate space can provide a similar guarantee for the one in the perturbation parameters. Moreover, they rigorously prove the connections between conditional robust optimization and the CVO problem. Simulation data and robust portfolio optimization using real-world data from the US market are provided for empirical evidence.
strengths_and_weaknesses: This paper is the first to introduce the conditional robust optimization framework for solving contextual optimization problems in a risk-averse setting based on the classical robust optimization approach but with novel development, i.e., an integrated approach to design the uncertainty sets adapted to the covariate information. The story is complete and the analysis is comprehensive. The problem is well-motivated and the solutions are sound and concrete.
questions: None.
limitations: Cannot find any limitations about this paper.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the author has proposed a Conditional Robust Optimization (CRO) paradigm that seeks the solution to a robust optimization problem using contextual information. The uncertainty set accounts for the most recent side information provided by the set of covariates.
The conditional uncertainty is designed with an integrated framework (IDCC) that jointly learns a partition in the covariate data space and simultaneously constructing region specific deep uncertainty sets for the random vector that perturbs the CRO problem. The paper tested the
IDCC algorithm using the US stock market dataset, and shows empirically that their approach reduces the out-of-sample VaR considerably compared to non-contextual RO schemes when the level of protection needed is high.
strengths_and_weaknesses: 1. CRO is a novel approach that utilizes the contextual information. It reduces the out of sample value at risk considerably compared to the conventional non-contextual optimization methods.
2. The paper corrects a wrong assumption made previously by the authors of the DDDRO approach that the projected perturbation vector is normalized for each data, by replacing the simply defined uncertainty set that uses an ellipsoid in the projected space instead of
a sphere.
3. The IDCC framework performs deep clustering and deep uncertainty set design jointly which ensures that the uncertainty set is learnt correctly, by ensuring that low variation in the side information vector results in low variation in the perturbation vector.
4. Deals well with partial clustering situations by training a parameterized random assignment policy.
5. DDDRO and IDCC, which use deep neural networks, are capable of identifying non-convex uncertainty sets.
6. IDCC exploits the side information by concentrating the uncertainty set on a region that has most mass thus leading to a less conservative optimization problem than DDDRO and ellipsoid.
7. The paper is well documented in terms of the details for the experiments with no significant ambiguities.
questions: 1. Use of more real or synthetic datasets to get a better generalization of the approach in various settings, and to clarify some of the claims, such as the fact that the loss in performance for minority clusters is due to its low frequency which makes it difficult to
learn the conditional uncertainty sets.
2. It might be nice to have the algorithms for the DCC also mentioned to help us understand the differences between DCC and IDCC, as well as the individual approaches, more intuitively.
3. Here the author has used K-means clustering, which works very well for smaller data, but can take very long to run for large datasets, since it iterates over the entire dataset. It can be useful to try out larger datasets using density based clustering methods such as
DBSCAN. Further, it might be useful to try out some soft clustering methods like fuzzy k-means or fuzzy c-means for partial clustering.
4. A mention of the run-time of each of the experiments might give a better idea of the comparisons of the various methods, unless their run-times are similar.
5. Are the above algorithms useful only for time-series data? Can we extend these algorithms to perhaps a computer vision system? If yes, it would be nice to generalize the approach in various different settings either through a short description or more
variety of datasets.
6. Can the authors mention more explicitly the limitations of the proposed algorithm?
limitations: 
Tied to K-Means Clustering although it's clear now that the clustering algorithm can be tailored.
review_title: null

Meta-review: metareview: The paper proposes a conditional robust optimization framework for solving contextual optimization problems in a risk-averse setting. All three reviewers seem to agree on the usefulness and originality of the proposed approach. As Reviewer 8P9p finds the story to be complete, the analysis to be comprehensive and the problem to be well-motivated with sound and concrete solutions, Reviewer k1N2 points out that the approach leverages very well results from diverse fields such as clustering, outlier detection, robust optimization, and mixed-integer linear optimization. Reviewer 34vX stresses that the method reduces the out of sample value at risk considerably compared to the conventional non-contextual optimization methods. While k1N2 originally phrased some concerns relating to the experimental setting, in particular the lack of benchmark from recent and relevant literature, the autor response lead k1N2 to a score increase. Similarly, 8P9p stated following the author response that from their extra explanations and extra experimental results, the reviewer gained confidence in the soundness of the proposed approach, also resulting in a score increase. Reviewer 34vX put to the fore that the experimental aspects of the paper are well documented. Also, the IDCC algorithm of the paper is illustrated using the US stock market dataset, showing empirically that the approach reduces the out-of-sample VaR considerably compared to non-contextual RO schemes when the level of protection needed is high. From a methodological perspective, it is also point out by 34vX that as neural network-based approaches, the considered DDRO and IDCC approaches are capable of identifying non-convex uncertainty sets, a desirable property. Overall, this paper was found to be a very well written and original reseach work on a timely topic w,ith methodological and empirical results of potentially string interest to the NeurIPS readership and beyond.  For all these reasons, I am recommending the paper to be accepted. 
confidence: Certain
Decision: Accept