SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. This is a direct application of Proposition 6. We first show the relation between two LPs with different objective vectors. For any two LPs with different objective vectors  $c_{1}, c_{2}$  but with the same constraint  $(A, b)$ , denote  $x_{1}^{*}$  and  $x_{2}^{*}$  as their optimal solutions. Then, if the optimal solutions are contained in the unit ball, we have
$$
\begin{array}{l} c _ {1} ^ {\top} \left(x _ {2} ^ {*} - x _ {1} ^ {*}\right) = \left(c _ {1} ^ {\top} x _ {2} ^ {*} - c _ {2} ^ {\top} x _ {2} ^ {*}\right) + \left(c _ {2} ^ {\top} x _ {2} ^ {*} - c _ {2} ^ {\top} x _ {1} ^ {*}\right) + \left(c _ {2} ^ {\top} x _ {1} ^ {*} - c _ {1} ^ {\top} x _ {1} ^ {*}\right) \\ \leq \left\| c _ {1} - c _ {2} \right\| _ {2} \left\| x _ {1} ^ {*} \right\| _ {2} + \left\| c _ {1} - c _ {2} \right\| _ {2} \left\| x _ {1} ^ {*} \right\| _ {2} + \left(c _ {2} ^ {\top} x _ {2} ^ {*} - c _ {2} ^ {\top} x _ {1} ^ {*}\right) \tag {33} \\ \leq \left\| c _ {1} - c _ {2} \right\| _ {2} \left\| x _ {1} ^ {*} \right\| _ {2} + \left\| c _ {1} - c _ {2} \right\| _ {2} \left\| x _ {1} ^ {*} \right\| _ {2} \\ \leq 2 \| c _ {1} - c _ {2} \| _ {2}, \\ \end{array}
$$
where the first step is obtained by rearranging terms, the second step is obtained by Cauchy inequality, the third step is obtained by the optimality of  $x_{2}^{*}$  with respect to the objective vector  $c_{2}$ , and the last step is obtained by the condition that  $x_{1}$  and  $x_{1}$  are in the unit ball.
Then, since in the event of Proposition 6 that happens with probability no less than  $1 - \delta$ ,
$$
\| \mathbb {E} [ r | z _ {0} ] - r _ {0} \| _ {2} \leq \frac {b _ {R}}{b _ {r}} \cdot \left(\frac {1 6 L}{\delta} \cdot \bar {p} d ^ {s / 2} R ^ {s} + \frac {3 2}{\delta^ {1 / 2}}\right) \cdot T ^ {- \frac {s}{2 s + 2 d}} \log T,
$$
and the difference between the objective vectors is the same as the difference between the corresponding residuals, the inequality (33) implies that
$$
\mathbb {E} [ c | z _ {0} ] ^ {\top} \left(x ^ {*} (z _ {0}) - \hat {x} (z _ {0})\right) \leq \frac {b _ {R}}{b _ {r}} \left(\frac {3 2 L}{\delta} \cdot \bar {p} d ^ {s / 2} R ^ {s} + \frac {6 4}{\delta^ {1 / 2}}\right) \cdot T ^ {- \frac {s}{2 s + 2 d}} \log T, \tag {34}
$$
and we finish the proof.
In this section, we state some useful lemmas in the information theory that are helpful in our proof. Lemma 3 (Hoeffding's inequality). Let  $X_{1}, \ldots, X_{T}$  be independent random variables such that  $X_{t}$  takes its values in  $[u_{t}, v_{t}]$  almost surely for all  $t \leq T$ . Then the following inequality holds
$$
\mathbb {P} \left(\left| \frac {1}{T} \sum_ {t = 1} ^ {n} X _ {t} - \mathbb {E} X _ {t} \right| \geq s\right) \leq 2 \exp \left(- \frac {2 T ^ {2} s ^ {2}}{\sum_ {i = 1} ^ {n} (u _ {t} - v _ {t}) ^ {2}}\right)
$$
for any  $s > 0$
Proof. We refer to Chapter 2 of Boucheron et al. [2013].
Due to the space constraints in the main text, we provide additional discussion on related work here.
Contextual RO with a parametrized prediction model. An alternative approach to contextual robust optimization relies on the assumption that the relationship between the outcome variable  $c$  and the covariate  $z$  is governed by a parameter  $\theta$  in the true underlying model. In this approach, the estimation/prediction process and the optimization problem are integrated by constructing an uncertainty set for the parameter  $\theta$  to mitigate the estimation uncertainty. Zhu et al. [2022] constructs the uncertainty set for  $\theta$  that contains all the parameters with training loss no worse than a threshold, but no coverage guarantee is provided. This can also be viewed as a special case of the decision-driven regularization model introduced in Loke et al. [2022]. Another study by Cao and Gao [2021] extends the consideration of uncertainty to both the parameter and residual. However, the incorporation of estimation and optimization processes in one optimization model can raise tractability issues, restricting the choice of complex yet expressive prediction models.
For the contextual stochastic optimization problem with a risk-neutral objective, below is the other two streams of work in addition to the predict-then-optimize paradigm illustrated in the main text.
Local learning-based conditional stochastic optimization. One stream of work tends to construct stochastic optimization problems with estimated conditional distributions learned by some local
learning methods. Hannah et al. [2010] approximates it by using Nadaraya-Watson kernel regression [Nadaraya, 1964, Watson, 1964] to reweight the historical data. Bertsimas and Kallus [2020] further generalizes it to a framework using a broader range of non-parametric machine learning methods based on local learning, such as kNN [Altman, 1992], classification and regression trees [Breiman, 2017]. To overcome the overfitting effect of these methods when the sample size is small, distributionally robust optimization (DRO) methods are introduced. Bertsimas and Van Parys [2022] enhances the robustness of bootstrap data by constructing the entropy-based DRO model. Nguyen et al. [2020, 2021], Wang et al. [2021], Bertsimas et al. [2023] investigate the Wasserstein-based DRO model. Ho and Hanasusanto [2019] designs a regularized approach and obtains a tractable approximation by leveraging ideas from DRO with a modified  $\chi^2$  ambiguity set.
End-to-end prescriptive analytics without prediction. Instead of learning to predict uncertain outcomes, Ban and Rudin [2019] directly learns the function mapping from covariates to decisions by employing the linear decision rule for the newsvendor problem. There are also further studies investigating such integrated approaches without prediction using more complicated models [Bertsi-mas and Koduri, 2022], such as random forests [Kallus and Mao, 2022] and neural networks [Chen et al., 2022]. Nevertheless, the robustness of the decisions to the epistemic and aleatoric uncertainty is rarely considered in these papers.