The Bias-Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective
Authors: Haixiang Lan
Luofeng Liao
Adam N. Elmachtoub
Christian Kroer
Henry Lam
Haofeng Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Data-driven optimization
model misspecification
local asymptotic analysis
contiguity theory
bias-variance tradeoff
Primary area: optimization
TL;DR: This paper is a local misspecificiation analysis of three data-driven stochastic optimization methods.

Abstract: Data-driven stochastic optimization is ubiquitous in machine learning and operational decision-making problems. Sample average approximation (SAA) and model-based approaches such as estimate-then-optimize (ETO) or integrated estimation-optimization (IEO) are all popular, with model-based approaches being able to circumvent some of the issues with SAA in complex context-dependent problems. Yet the relative performance of these methods is poorly understood, with most results confined to the dichotomous cases of the model-based approach being either well-specified or misspecified. We develop the first results that allow for a more granular analysis of the relative performance of these methods under a local misspecification setting, which models the scenario where the model-based approach is nearly well-specified. By leveraging tools from contiguity theory in statistics, we show that there is a bias-variance tradeoff between SAA, IEO, and ETO under local misspecification, and that the relative importance of the bias and the variance depends on the degree of local misspecification. Moreover, we derive explicit expressions for the decision bias, which allows us to characterize (un)impactful misspecification directions, and provide further geometric understanding of the variance.

The Bias- Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective
Haixiang Lan' Luofeng Liao' Adam N. Elmachtoub',
Christian Kroer', Henry Lam', Haofeng Zhang
Department of Industrial Engineering and Operations Research, Columbia University Morgan Stanley (h13725, 113530 ae2516, ck2945, kh12114, h22.1144,222553)ollllmmmo edu
Abstract
Data-driven stochastic optimization ubiquitous in machine learning and operational decision messionnmaking problems. Sample average approximation (SAA) and model-based approaches such as estimate-then-oppimize (ETO) or integrated estimation-opimizatton (IEO) are all popular, with model-based approaches being able to circumvent some of the issues with SAA in complex context-depent denendtt problems. Yet the relative performance of these methods is poorly understood, with most results confined to the dichotomous cases of the model- approach being either well-specified misspecified. We develop the first results that allow for more granular analysis of the relative performance of these methods under local misspecification setting, which models the scenario where the model- based approach is nearly well-specified. By leveraging tools from contiguity theory in statistics, IEO, and ETO under local misspecification, and that the relative importance of the bias and the variance depends the degree of local misspecification. Moreover, we derive explicit expressions for the decision bias, which allows to characterize (un)impactful misspecification directions, and provide further geometric understanding of the variance.
show that there is a bias variance tradeoff between SAA.
Introduction
Data-driven stochastic optimization arises ubiquitously in machine learning and operational decision- making problems. Generally, this problem takes the form argmin. Eq [c(w, =]] where represents the decision that we aim to optimize, and is a random variable (or vector) drawn from an unknown distribution Q. The non-linear cost function c is given and can represent loss function in machine learning, financial portfolio losses, or resource allocation costs. In this paper, we focus on the setting where the expectation Eq is unknown, but observe data from Q.
natural approach to use empirical optimization, known as sample average approximation (SAA) (hapiro al. 2021]. which approximates the unknown expectation with the empirical counterpart from the data. This approach straightforward, but may not suitable for complex scenarios in constrained and contextual optimization, when one needs to obtain feature-dependent decision (i.e., a decision as a "function" " of the features) and maintain fcasibility (Hu et al.) 2022]. In such cases, model-based approaches provide workable alternative. A model-based approach fits a parametric distribution class to the data, say {Po E θ}, and this fitted distribution is then injected into the downstream optimization to obtain decision. Just as in standard machine-learning problems, this parametrization helps maintain generalizability from supervised data.
'Equal Contribution and Corresponding Authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Our focus in this work is the statistical performance of model-based approaches for data-driven optimization with nonlinear cost objectives, as compared to SAA. More specifically, we study the question of how to fit the data into the parametric distribution models. There have been two major methods proposed in the literature: Estimation- iitimation-then-OOe Optimizee (ETO) and Integrated Estimation--Opimizaaii
(IEO). ETO separates the fitting step from the downstream optimization, by simply fitting Pg via maximum likelihood estimation (MLE). IEO, on the other hand, integrates the downstream objective with the estimation process, by selecting the distribution parameter 0 that minimizes the empirical expected cost. Conceptually, ETO can readily leverage existing machine learning tools and fits the model disjointly from the downstream decision task, while IEO attempts to take into account the downstream decision task (in many cases with a nontrivial additional computational expense). Intuitively, then, IEO should have better statistical performance than ETO in terms of the ultimate objective value of the chosen decision.
Our main goal is to dissect the varianar tradeoff between ETO, IEO, and also SAA, especially under the setting of model misspecification. In particular, our study reveals not only the variance arising from data noise, but also the bias of the resulting decision elicited under model misspecification. This allows us to gain insight on how the direction and amount of misspecificatiio impacts decision pipeline, quality. More concretely, well-specified model means that in the estimation-opiimizati the chosen parametric class {Po : 0 E }} contains the ground-trute distribution Q- a case that is rarely seen in reality. In other words, model misspecification arises generically, the question only being by how much. Unfortunately, the theoretical understanding of the statistical performance approaches, especially in relation to this misspecification,
among the various estimation-optimizatiii has been rather limited. Ellaactoub et al.] (2023) compare these approaches in large-sample regimes via stochastic dominance, but their analysis divides into the cases of well-specification and misspecification, each with different asymptotic scaling. Unfortunately, there is no smooth transition in between that captures the impact of varying the misspecification amount. [Elmachtoub] iet al. 2025) attempt to address this issue by deriving finite-sample bounds that depend on the sample size and misspecification amount.
In this paper, we remedy the shortcomings in the above literature by leveraging the notion of local misspecification, originated from contiguity theory in statistics (Le Cam and Yang, 2000 Copas and Eguchi [001] [Andrews et al. 2020]. to derive large-sample results in relation to both the amount and direction of misspecification. Our results explicitly show the decision bias and variance, and its resulting regret, that arises from misspecification. This allows us to smoothly compare ETO, IEO and SAA. We show the following results. When model misspecification is severe relative to the data noise level, SAA performs better than IEO, and IEO performs better than ETO, in terms of both bias and regret. This matches the intuition described previously that IEO should outperform ETO by integrating the estimation-optimizate amount is mild, the performance ordering reversed, which generalizes previous similar findings in Elmachtoub et al. (2023) that focused on zero misspecification. Most importantly. in the most relevant case where the misspecification is roughly similar to the data noise, which we call the balanced case. the ordering of the methods exhibits a bias vtrsinariance tradeoff: SAA performs the best on bias, whereas ETO performs the best on variance, and IEO is in the middle for both metrics. This defies universal performance ordering, but also points to the need for deep understanding of the characteristics of the bias term in relation to the misspecification direction. Table summarizees our performance ordering findings.
pipeline. On the other hand, when the misspecification
mild (a > 1/2)
balanced (a- = 1/2)
severe (0 < a < 1/2)
bias
variance
regret bias
variance regret bias
variance
regret
ETO
best
best worst best
depends
worst
R 0
worst
IEO
2 O
middle
middle
middle
middle
depends
middle
R 0
middle
SAA
worst
worst
best (+s 0) worst
depends
best (*= 0)
RS 0
best
Table 1: Summary of our results on performance orderings. "As 0" means asymptotically negligible.
is a parameter that signals the misspecification amount relative to the data noise level and will be detailed later.
Our next contribution is to provide an explicit formula for the bias attributed to model misspecification, which allows us to gain insights into how the bias impacted by the misspecification direction. We went beyond the classical local minimax theory by showing the non-regularity of the ETO and

---

IEO estimators, which is rarely seen in standard statistical literature [van der Vaart 2000]. In the severely misspecified regime, where there no available contiguity theory tools, we develop a novel technique to characterize and compare the asymptotics of the three estimators. We further identify sufficient conditions on approximately impactless misspecification directions - model misspecificatiio directions that result in bias that is first-order negligible compared to the data noise variance. In general, this direction is orthogonal to the difference between the influence functions of solutions obtained from the considered estimation-optimizali are interpreted as the gradients with respect to the underlying data distribution, and they appear not only in the bias but also variance comparisons. This characterization in particular suggests how SAA is always (and naturally) the best in terms of bias (see Table D.. but also how the biases for IEO and ETO magnify when the obtained solution has a different influence function from that of SAA. Moreover, to enhance the transparency of our characterization, we show that a sufficient condition for being approximately impactless is to be in the linear span of the score function of the parametric model. While this latter condition imposed purely on the parametric model (i.e., not the downstream optimization), it already shows the intriguing phenomenon where model misspecification could be insignificant in impacting the performance of the ultimate decision.
pipeline and SAA. Here, influence functions
1.1 Related Works
Data-Driven Stochastic Optimization. Data-driven optimization, a cornerstone machine learning and operations research, addresses problems where decision are informed from optimization problems with parameters or distributions learned from data. Existing popular methods include SAA (Shapiro) al. 2021] and distributionally robust optimization (DRO) (Delage and Ye 2010). Recently, there has been a growing interest in an integrated framework that combines predictive modeling of unknown parameters with downstream optimization tasks [Kao et al. 2009 Donti et al. (017). When the cost function is linear, Elmachtoub and Grigas 2022 propose a SSmart-redic--the---ppimmm
(SPO) approach that integrates prediction with optimization to improve Docision-making Recent
literature explore further properties of the SPO approach [Mandi et al. 2020 Blondel et al., 2020
Ho-Nguyen and Kılinç- Karzan 2022 Liu and Grigas 2022 Liu et al. 2023 El Balghiti et al. 2023
Hu et al. [022] further compare the performances of different data-driven approaches in the linear
cost function setting. In the context of non-linear cost functions, Grigas et al. (2021] propose an
integrated approach tailored to discrete distributions, and Lam 2021 compares SAA with DRO and
Bayesian extensions.
Local Misspecification. Model misspecification is extensively studied in the statistics and econometrics and machine learning literature [Marsden et al.] 2021]. In this paper we focus on local misspecification, where the magnitude of misspecification vanishes as the sample size grows. Newey [1985] analyzes the asymptotic power properties of the generalized method of moments tests under a sequence of local misspecified alternatives. Kang and Schater (2007) design a doubly robust procedure to estimate the population mean under the local misspecified models with incomplete data. Copas and Eguchi [2001] 2005 discuss the impact of local misspecification on the sensitivity of likelihood-based statistical inference under the asymptotic framework. Local misspecification is
also discussed in robust estimation Kitamura et al. [2013] Armstrong et al. 2023), causal inference [Conley al. 2012 Fan et al. 2022], econometrics [Bugni et 2012| Andrews et al. [017 2020
Bugni and Ura 2019 Armstrong and Kolesár 2022 Bonhomme and Weidner 2022 Candelaria and
Zhang 2024] and reinforcement learning [Dong al. [2023] To the best of our knowledge, we
the first to study the impact of local misspecification in data-driven optimization.
2 Settings and Methodologies
2.1 Data-Driven Stochastic Optimization
Consider a data-driven optimization problem in the following form:
argmin (vo(w) == EE [c(w, =)]}
WC
where Ω is an open subset in Rdw E 2 C d is the uncertain parameter with unknown data- generating distribution Q, .(...) a known non-linear cost function, and vo(-) (-) the expectation of the cost function under ground-trun truth distribution Q under decision w. We are given independent
3

---

and identically distributed (i.i.d.) data 21..... Zn drawn from Q. and the goal is to approximate the optimal decision w" using the data.
In model-based approaches, we use parametric distribution family (Po, 0 E ) where E e C Idd is the model parameter and & is an open subset of Igde To explain further, we define the oracle solution we by
E argmin {u(ur, 0) == Er, [c(w, z)]} wEll
(2)
where v(w, 0) is the expected cost function under the distribution Pp. Depending the choice of the
model, the ground-truth distribution Q may or may not be in the parametric family {Po E }). We say {Po ) is well-specified if there exists 0g- A such that Pa. = Q. We say (Po 0 O) is misspecified if it is not well-specified.
Notations. We denote Epl]] and varp as the expectation and variance under the distribution P. We denote [- as Ep, [:] and varø(-) varp, (-) in the parametric case. For symmetric matrix A, we write 2 0 if it is positive semi-definite and A > if it is positive definite. For two symmetric matrices A₁ and A2. we write A₁ 2 A if A A 2 0 and A A₂ if A1 A > 0. For a matrix A E Rmx"" we define the column span of A as col(A) = {Ax x ]" For vector E _º and a positive semi-definite matrix A E R=X=, we define the matrix-induced
norm | == Vx Ax. We define P as convergence in distribution under the measure P".
Precisely, Xn. + X if (X, t) + IP(X S for all continuous points of the distribution of we say
X where IP denotes the distribution X. For a sequence of random variables { Xn = =0p= (1) if it is stochastically bounded under the probability measure 1a i... for all 6 > 0 there exists M and N E N such that for all n 2 N. P" ([X,,, > M) S 8. We say X,, = 0pm (1) if it converges to zero in probability under the probability measure Pm. i.e. Xn Opn. (1) if for all E > 0, lim, PM ((x,,) > E) = O. More generally, we denote X,, = (a,))) if Xf,/aa
= Op ( (1) and
we denote X, Ope (an.) if X./aa, opn (1). For random variables X1, X2 with distribution
Pi and P2. we say X, is (first-order) stochastically dominated by X2, denoted as X1 Sa X2, if for all R, it satisfies that P, > 5) < PP2(X2 2).
2.2 Three Data-Driven Methods
We consider three popular approaches for solving
I in a data-driven fashion.
Sample Average Approximation (SAA). SAA simply replaces Eg in (I) with its empirical counterpart.
More precisely, we solve SAS := argmin {fo(w) == 1 En c(w, 2.))
Estimate- Serimate-Teen-Opimmizz (ETO). ETO uses maximum likelihood estimation (MLE) infer 0 by solving Q supoge n
1 Σ log Here pe is the probability density or mass function.
Then we plug hETO into ( FTO : gero argmin v(w, OFTO).
Integrated- Estimation Integrated-EtiimationnO-P Optiiattiiiii-iiiiittaiiioooonnn (IEO). IEO selects the that performs best the empirical cost function (-(-) evaluated at we. More precisely, we solve inface do (wg) and get a solution @IEO Then we use the plug-in estimator w 180 := Wino- argmin U giEO).
Among the three methods. SAA is model-free while ETO and IEO are model-based. ETO separates estimation (via MLE) with downstream optimization, while IEO integrates the latter into the estimation process. Our primary focus is to statistically compare these three data-to-decision pipelines in the so-called locally misspecified regime, which we discuss next.
Throughout the paper we assume certain classical technical assumptions on the cost and the distribution Po to ensure the asymptotic normality of certain -estimatoors under Pa, and the Cramer- Rao lower bound. In particular, they require relevant population minimizers of min Eg, [c(w, 2)]. ming [e, [- logpe(2)) [=)) and ming Egg [c(wa. 2)] are uniquely attained in the interior of the parameter spaces. See Assumptions 3 and 4 in the Appendix for precise statements.
2.3 Local Misspecification
We first explain local misspecification intuitively before providing formal definitions. Recall the ground-truth data generating distribution Q, and our parametric distribution family (i.c., model)

---

{Po: : 0 E O}. At a high level. we assume that for a finite data size Q could deviate from the model a certain "direction". However, as n is sufficiently large, we expect have a more accurate model, and Q will approach a distribution in {Pa O E ) which we denote as Pa, for some Pg E e. In other words, (Po 0 E } misspecified to Q in a "local" sense, and such misspecification will ultimately vanish. From now on, we use Pe. to represent this distribution. Note that neither Pe. nor 0o are known in practice.
To introduce the local misspecification regime, we first formally define a notion called local perturbation for describing the deviation between two distributions. We work with a general form of local perturbation [van der Vaart, 2000, Fan et al.] [2022] Duchi and Ruan [202]] Duchi [021] where the ground truth distribution Q is related Pe. through a general tilt the distribution (we will provide many classical examples in Appendix (.2]::
Definition (Local Perturbation). Consider a scalar function u(z) 2 - R with zero mean Egg [u] 0 and finite second order moment IEg, [x²]. We define a tilted distribution Q: for / E IR with probability density (mass) function qi with respect to the dominated measure (note that Qi is not necessarily in the parametric family {Po 0 E θ} with qo pe, We further assume for all t, is differentiable for almost every 2, as well as the quadratic mean differentiability condition:
/
(Va (va:
2 dz = o(e))
= plu 2
VPo
Note that when 0. go = po.-
1og P. e
log
fog Py e
log
log
Q
(a) First direction
(b) Second direction
Figure 1: Local Misspecification
The local perturbation in Definition Tis is standard in classical asymptotic statistics [van der Vaart 2000) and consists of two crucial elements, the scalar function u(z) and the real value t. Intuitively. we can think of as the degree of perturbation and the function u(z) as a certain direction of perturbation. Figure presents such geometric interpretation. The parametric family (Po : O O could be viewed as a "curve" in the distribution space. Each point on the curve corresponds to a distribution in the parametric family. If we fix the value t and let the direction u(z) range over all possible directions, then Q will lie within a neighborhood of radius t around this curve. In this sense. the perturbation quantity tu(z) acts like the "vector" pointing from P'e to Q where the "length" is t and the "direction" is given by the vector u(z). Below we will discuss the "local" case where the radius vanishes as the sample size goes to infinity. particular, in Figure (b), the "direction" u(z) is tangent to the curve. We will discuss this special case in the latter part of this paper (Theorem 4.
Local misspecification refers the situation where, the sample size n increases, the sequence Qt. where / depends on n. approaches the model. When the ground-truth Q. lies outside the parametric family, this misspecification adds errors the standard inference errors the order ((1/vni,, and when these two error levels coincide at the same order, we call this scenario balanced misspecification. More broadly, we consider local misspecification with to the following definition. For n i.i.d. data { sampled from Q, let Q" ;= QSa be the n-fold product measure of Q denoting their joint distribution, and analogously for Po.-
(1/n") where a E (0, 20), which leads
Definition (Three Local Misspecification Regimes), Let pm pilnn The tilted distribution Qe is defined in Definition Suppose > 0 and the joint distribution of the i.i.d. data is Q" := Q
Qi/na-
5

---

1. (Mild). We call the case when > 1/2 the mildly misspecified regime.
2. (Balanced). We call the case when a = 1/2 the balanced misspecified regime.
3. (Severe). We call the case when < a < 1/2 the severely misspecified regime.
We note that the local misspecification regime described above should not be taken literally to imply that the data-generating process depends on the sample size. Instead, is information theoretic device to analyze and compare the local behavior of estimators in situations where the influence of misspecification is comparable the order of the statistical error. In other words, we are interested in the more realistic setting where both misspecification and statistical enror are small at similar level, instead of assuming vanishing statistical error but fixed misspecification as previous work [Elmachtoub et al. 2023 2025
3 Main Results
We derive theoretical results compare the asymptotic performances of the three methods, SAA, ETO, and IEO, that encompass the three local misspecification regimes in Definition 2 We first list several standard assumptions. We define se, se,(2) Ve log pe, as the score function at Oo mapping from Z + Re Recall that w(w, 0) c(w, z p)po (z)dz.
Assumption (Smoothness). Assume that
1. The function v(w, 0) is twice continuously differentiable with respect to (w, 0) at (we. 9o) with a Hessian matrix, denoted by
Σ
* ,where denotes a matrix that is not of interest. Assume E E Rd xdo is full-rank and V E xdd is invertible.
Σ
2. The function - we is well-defined on a neighborhood of Oo, twice continuously differentiable Ou with full-rank gradient matrix V vowelo=o. E doxdd
Rdd is well-defined and 3. The Fisher information matrix I := Es, (z) invertible.
Note that the matrices above are fixed quantities and are not related to whether the model is well- specified or misspecified. These matrices are critical for characterizing the sensitivity of the target stochastic optimization problem. We also define & V2,0, f c(wa, x)pe, (=)dz 0, -,=0.- The following lemma provides closed-form expressions for the gradient Vowg and matrices E, Φ.
Lemma 1. Under Assumption | it holds that
Vewale= - V-1 = [wr [))),(2)]]
[=) + = -
Next. we introduce the influence function which is a key ingredient in our derived formulas. Originating from robust statistics [Hampel 1974], it is the functional derivative of an estimator with respect to the data distribution. In our context, this refers to the derivative of the decision obtained from the estimation -pptmiiziooiiiiiiiiinnn pipeline. Specifically, the influence functions for SAA, IEO and ETO are respectively
IFSAA (2) -V- V
mn Wap 2),
IF0 (z) V-1 ΣΦΦ Voc(we,, 2),
(z) = V 'I se, (z),
all of which are pdw-valeed. Regarding notations, V Vwc(we, =) is the gradient of the map c(w, z) at w = wa,. and Tec(we, 2) is the gradient of the map + c(wg, z) at 0 = 0g.
Finally, we introduce regret as the criterion to evaluate the quality of decision w. Since we conduct local asymptotic analysis, the definition of regret is slightly different from the classical definition in asymptotic or finite-sane samie- analysis [Lam 2021 Elmachtoub and Grigas 2022], as it needs to account for the changing sample size. We deñie V, and w as follows:
w : argmin u, (w) = Eqn
c(w, 2;)
wes
7

---

In the local misspecification setting, when the sample size is n, the data distribution is given by Q". Hence, 2, (w) represents the ground-truth expected cost, and w interpreted as the corresponding optimal solution at sample size n. Definition 3 (Regret). For any distribution Q" and any Q2 the regret of sample size is defined as
RQ (w) ( Un (n
In the rest of this section, we conduct a comprehensive analysis on the regrets using the three istimation-optimizztion methods, for the three misspecification regimes introduced Definition 2
3.1 Balanced Misspecification
To state our main results, we define, for E {SAA, IEO, ETO,
NO N(0,, var (z))),
6 = Ea, [u( (=)= (=) (ZSAA(2)) E RR=,
ne 1 =|1b71 E
where N is the normal distribution with zero mean and covariance matrix var (ar(IF(2))) Note that unless otherwise specified, var(-) should always be interpreted as var (.). Theorem (Asymptotics under Balanced Misspecification) Suppose Assumptions and hold. In the balanced regime in Definition
2 under Q", for U (SAA,ETO, IEO).
Vn(w w 2 NO +
nRQn (z0 ) 2 (N + +67) T
V (NT+
In terms of bias, =
|SSA 1l S || s | terms of variance, var (NSAA) var (NIEO) var (NVETO)
Theorem I states that when 1/2, i.e.. the degree of misspecification is of the same order as the statistical error, the gap between the data-driven and optimal decisions is asymptotically normal. Moreover, this normal has mean for SAA (note bSAA 0 and RSAA 0), but generally nonzero for ETO and IEO. More importantly, in the asymptotic limit, 6D represents the bias coming from model misspecification as it involves while NO captures the data noise variability. We highlight that the dependence on u in b directly implies that the ETO and IEO estimator are non-regular the sense of van der Vaari 22000] The theorem shows that in terms of bias, SAA generally outperforms IEO which in turn outperforms ETO. On the the hand, the ordering is reversed for variance. As result there is no universal ordering for the overall error in general. The next theorem lifts further to compare the regrets of SAA, ETO and IEO under the balanced regime. Theorem (Regret Comparisons under Balanced Mtsspecificationn. Lel
G = (NP
3 + bB) (N_ + +ba)
denote the limiting regrel distribution of E (SAA, ETO, IEO} in Theorem 1 T + (bP)) Vb
Then E[G²]
E
T Moreover,
E
(NETO)
V
SE 2
(NBOO)
VA
1 (NSM)) V.NSAA
2
(BSM)) T VOSAA
1B0
V6 IBO
( (EET T VITTO
Like Theorem while Theorem 2 shows a lack of universal ordering for regrets, it depicts a decomposition of the asymptotic distribution of the regret into two parts where two opposite orderings emerge. In particular, it suggests that while ETO best in terms of variance, and SAA best in terms of bias, IEO is in between and could potentially induce lower decision error compared to the other two methods.
Another important insight from Theorems andd regards the explicit form of the bias and variance. For this, let us introduce the analogous results for severe and mild misspecification regimes and discuss the formulas along the way.
7

---

3.2 Severe Misspecification
We first formally describe the (1) order of the statistical error via the following assumptions borrowed from Fang et al.] [2023]. The assumption is natural because it says the empirical part deviates from the expected part at the rate O(1/vn)..
Assumption 2 (Statistical Error Order). For i.i.d {zi)==1 with joint distribution Q", let
ghL argmax Eqn deG
log po (zi)
0
argmin BQ" oee
c(wo, zi)
Assume that ||
Woz ||- | wa and 1 w || are all of order Oq- (1/vnn. Moreover, assume that the matrix V ww Un (w))
+ V asn xc.
Theorem 3 (Asymptotics under Severe Misspecification). Suppose Assumptions 7 and hold. In the severely misspecified case, under Q", for O E {SAA, ETO, IEO),
n w L b=
1
20 Ron (1 D, R b
2
In terms of variance, 0 = var (bSAA) = var 100) = var (BETO). The comparison of bias has the same form the regret (stated in the next theorem).
Theorem 4 (Bias/Regret Comparisons under Severe Misspecification,. Under the same setting as in bSAA
Theorem we have
| S [bTTO] and 0 RSAA < RIEO < RETO
Theorems and 4 stipulate that, once the degree of misspecification is larger than the statistical error (0 < a 1/2), SAA will dominate ETO which will further dominate IEO. This is because this regime only the bias surfaces, and this ordering is in line with the bias ordering in Theorems and 2
In both the balanced and the severe misspecification cases, the bias term can be significant relative to the variance. In all cases, the bias has the form b Ee,, [u(z) (IF (z) IFSAA IFSAA(=))], an inner product between the misspecification direction and the difference of influence functions between the considered estimation-opiimizatii
pipeline and SAA. Note that the latter difference is always zero for SAA, which coincides with the model-free nature of SAA that elicits zero bias. On the other hand, for either IEO or ETO, the bias effect can be minimized if the misspecification direction is orthogonal to the influence function difference. While this characterization is generally opaque, the following provides a more manageable sufficient condition.
Theorem 5 (Approximately Impactless Misspecification Direction). Let the assumptions in Theorem hold. If u() (+) E {3 sea (-) 3 E Re,e }, then BETO (and thus b and bSAA) = O.
Theorem 5 states that if the misspecification direction is in the linear span of the score function of the imposed model at Og, the asymptotic bias is zero for all methods. Figure (b) direction, where u(z) is tangential to the model {Po} Pe,. In this case, the interesting direction of misspecification u(2) aligns with the parametric information and couples the influence function of ETO and SAA. As result, ETO can induce zero decision bias by merely conducting MLE to infer O, even though the model is misspecified.
(b)) illustrates such a
Note that the condition in 5 only on the parametric model, but not the downstream optimization problem. Nonetheless, already allows us to understand and provide examples where a model misspecification can be significant, namely shooting outside the parametric model, yet the impact on the bias of the resulting decision negligible. We provide an explicit example as follows.
Example 1. Consider the distribution family (Po R} to be normal distributions with variance 1, (0, 1). where O, = O. We define the tilted distribution (z(z) with density (z) xx (1+12)+e-** =2 In this case, (x(z) satisfies Definition I and the conditions in Theorem 5 but Q {Pe 0 R).
In Example| the parametric family is a normal location family, while at Og = 0, the perturbed distribution family Q, is never normally distributed for all t > 0. The direction of perturbation
8

---

(misspecification) at Ou here is u(z) = se, (z)
In other words, even if the ground truth distribution of uncertain parameters is complicated. model-based approaches under simplified and misspecified parametric family can still be employed with satisfying decision regret performance.
3.3 Mild Misspecification
Finally, we establish results under mild misspecification.
Theorem 6 (Asymptotics and Comparisons under Mild Misspecification) Suppose Assumptions 4nd hold. In the mildly misspecified case, under Q", for ETO, IEO),
vn( - w NNO
NO
nR (w0 ()
- 1
2 (NO) TVN.
Moreover, in terms of bias, all three estimators have asymptotically zero biases. In terms of variance, var (NSAA) N var (NibO) var (NEI)). In terms of regret, holds that
(NETO) T VNNTO In 2
(NIEO) T VVNDO In (NSAA) T VASAA
2
2
Theorem 0 shows that the obtained solutions (consequently also the regret) exhibit asymptotic behaviors in accordance with the universal ordering of ETO best, then IEO, and then SAA. In this regime, the bias is negligible, while the variance is the dominant term, and the regret distribution is related to the variance. The phenomenon also holds in the well-specified regime stated as follows.
Proposition (Asymptotics under Well Spleif-pcifiiniinnn Elmachtoub et al.
2023 )
the well-specified
case where Q Pey. under Assumptions 7 and for DE {SAA, E ETO, IEO), we have
Vn (100 w0
(zi) + opm
Moreover, var (IFS 2 var (I)))(=)) 2 var (I). If = dw and E is a square and full-rank matrix, then var (I[SAA(2)) var (1)))()))
4 Numerical Experiments
In this section, we validate our findings by conducting numerical experiments on the newsvendor problem, a classic example in operations research with non-linear cost objectives. We show and compare the performances of the three data-driven methods the finite-sample regimes under different local misspecification settings, including different directions and degrees of misspecification. The experimental results in the finite-sample regime are consistent with our asymptotic comparisons. carried out personal desktop computer without GPU acceleration.
All computations
The newsvendor problem has the objective function c(w, =a aT (w ())=)* + +d (z w)+ where for each j E [d=]: (1) z05) is the customers' random demand of product j; (2) w(l) is the decision variable, the ordering quantity for product j; (3) a is the holding cost for product j; (4) d() is the backlogging cost for product j. We assume the random demand for each product are independent and the holding cost and backlogging cost is uniform among all products by setting a0)) 5 and
d()) for all [dz]-
We describe the local misspecified setting by using the framework of Example 5 and building a model and generating random demand dataset as follows. We denote the training dataset as
()) where n is the training sample size. The model assumes that the demand for each product
j E [d=] normally distributed with the distribution N(j6, 1) where 0 is unknown and needs to be learned. We first describe the well-specified setting, where the demand distribution for product is (3j, 1). In this case, the probability density function of the random demand of each product 3/)²/2) To describe the local misspecification, we need to specify the direction and degree of misspecification, i.e., the expression of u(z) and a in Section 2.3 We
is (2())) exp(-(=(1
set (1) a 0.1 to denote the severely misspecified setting, (2) 0.5 to denote the balanced

---

setting and a = 2 to denote the mildly misspecified setting. We discuss two types of directions: (1) u(z) = I - (z())² 2 (2) u(z) m (20) 3))²/22 /2.
We show experimental results in Figure 2 Figure B to support our theoretical results in Section 3 using the mean, median, 25-th quantile, 75-th quantile and histograms of the regret. When u(z)
IT (=(0)² 2 and u(z) I (=0) 3)) 2 /2, in the mildly specified case, ETO has lower regret than IEO, and IEO has a lower regret than SAA. However, in the severely misspecified regime, the ordering of the three methods flips. This is consistent with our theoretical comparison results in Theorems 6 and 4 In the balanced regime, experimental results show that IEO has the lowest regret among the three methods. This is also consistent with the theoretical insight in Theorem 2 that IEO has the advantage of achieving bias-variance trade-off in terms of the decisions and regrets.
Mildly misspecified
Balanced
Severely misspecifed
0.0
6.10
SAA
ETO
0.05
SAA
ETO
SAA
IEO
0.32
IED
IE0
5.00
0.94
2/ 000
2 0.0
2 000
0.03
004
6.00
002
002
0.00
0.00
110
MV
29
103
150
sample
training sample
(a) Mildly Misspecified
(b) Balanced
(c) Severely Misspecified
Figure 2:
The direction of misspecification satisfies u(=) = IT (200)
Mildly misspecilied
Delanced
Severely misspec fieed
0.50
SAA ETO
0.50
0.07
ETO
SAA
0.20
SAA ETO
0.28
1E0
IEO
0.35
1E0
regret
megret
0.24
0.04
0.30
0.32
8
3
0.00
0.00
20
42
30
00
300
120
140
105
100
20
40
60
20
100
120
340
260
180
200
20
40 60
120 340 100 training sample size
03
100
100 200
training sample size
training sample size
(a) Mildly Misspecified
(b) Balanced
(c) Severely Misspecified
d= Figure 3: The direction of misspecification satisfies u(=)=I[g
= -11 (30)-33) /2.
5 Conclusions and Discussions
In this paper, we present the first results analyzing local model misspecification in data-driven optimization. Our framework captures scenarios where the model-based approach is nearly well- specified, moving beyond the existing dichotomy of well-specified and misspecified models. We conduct a detailed analysis of the relative performances of SAA, ETO, and IEO, providing insights into their bias, variance, and regret. By classifying local misspecification into three regimes, our analysis illustrates how varying degrees of misspecification impacts performance. In particular, we show that in the balanced misspecification case, ETO exhibits the best variance, SAA exhibits the best bias, while IEO entails a bias-varianc tradeoff that can potentially result in lower overall decision errors than both ETO and SAA. Additionally, we derive closed-form expressions for decision bias and variance. From this, we show how the orthogonality between the misspecification direction and the difference of influence functions can lead bias cancellation, and provide more transparent sufficient condition for such phenomenon in relation to the tangentiality on the score function. Technically, we leverage and generalize tools from contiguity theory statisties to establish the performance orderings and the clean, interpretable bias and variance expressions. Future research directions include extending our framework to contextual or constrained optimization problems, where challenges like fcasibility guarantees and model complexity become increasingly significant.
10

---

Acknowledgments and Disclosure of Funding
We gratefully acknowledge support from the National Science Foundation grant IIS-2238960, the Office of Naval Research awards N00014-22-1-2530 and N00014-23-1-23774 InnoHK initiative, the Government the HKSAR, Laboratory for Al-Powered Financial Technologies, and Columbia SEAS Innovation Hub Award. The authors thank the anonymous reviewers for their constructive comments, which have greatly improved the quality our paper.
References
I. Andrews, M. Gentzkow, and M. Shapiro. Measuring the sensitivity of parameter estimates to estimation moments. The Quarterly Journal of Economics, 15534)1555-1152, 2017.
I. Andrews, M. Gentzkow, and J. M. Shapiro. On the informativeness of descriptive statistics for structural estimates. Econometrica, 88(6):2231-2258, 2020.
B. Armstrong and M. Kolesár. Sensitivity analysis using approximate moment condition models. Quantitative Economics, 12(1):77-108, 2021.
B. Armstrong, Kline, and L. Sun. Adapting to misspecification. arXiv preprint arXiv:2305.14265 2023.
Asmussen and P. W. Glynn. Stochastic simulation: algorithms and analysis, volume 57. Springer, 2007.
S. Barratt. On the differentiability of the solution convex optimization problems. arXiv preprint 188,00880,,,5598, 2018.
P. J. Bickel and K. A. Doksum. Mathematical statistics: basic ideas and selected topics, volumes 1-1l package. Chapman and Hal//CRC, 2015.
M. Blondel, A. F. Martins, and V. Niculae. Learning with fenchel-young losses. Journal of Machine Learning Research, 21(35): 135)):169, 2020.
S. Bonhomme and M. Weidner. Minimizing sensitivity model misspecification. Quantitative Economics, 13(3):945-81,, 2022. doi: 10.3982/QE1930.
F. A. Bugni and T. Ura. Inference in dynamic discrete choice problems under local misspecification. Quantitative Economics, 10(1):67-103, 2019.
F. A. Bugni, I. A. Canay, and P. Guggenberger. Distortions of asymptotic confidence size in locally misspecified moment inequality models. Econometrica, 1744):17 2012. doi: 10.3982/ECTA9604.
L. Candelaria and Zhang. Robust inference in locally misspecified bipartite networks. arXiv preprint arXiv:2403. 13725, 2024.
G. Conley, B. Hansen, and P. E. Rossi. Plausibly exogenous. Review of Economics and Statistics, 94(1):20-272, 2012.
J. Copas and S. Eguchi. Local sensitivity approximations for selectivity bias. Journal of the Royal Statistical Society Series B: Statistical Methodology, 63(4):871-89) 2001.
J. Copas and S. Eguchi. Local model uncertainty and incomplete-data bias (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(4):459-513, 2005.
E. Delage and Y. Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations research. 58(3):595-61) 612, 2010.
K. Dong, Y. Flet-Berliac, A. Nie, and E. Brunskill. Model-based offline reinforcement learning with local misspecification, In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), 2023.

---

P. Donti, B. Amos, and J. Z. Kolter. Task-based end-to-end model learning in stochastic optimization. Advances in neural information processing systems, 30, 2017.
Duchi, few notes on contiguity, asymptoties, and local asymptotic normality, 2021. Accessed: 2024-12-16.
J. C. Duchi and F. Ruan. Asymptotic optimality in stochastic optimization. The Annals of Statistics, 49(1):21-48, 2021.
O. El Balghiti, A. N. Elmachtoub, P. Grigas, and Tewari. Generalization bounds in the predict- then-optimize framework. Mathematics of Operations Research, 48(4):2043-2055, 2023.
A. N. Elmachtoub and P. Grigas. Smart "predict, then optimize". Management Science, 68(1))9-26, 2022.
A. N. Elmachtoub, H. Lam, H. Zhang, and Y. Zhao. Estimate-thennopiimee versus integrated- cstimation-optimizatiio versus sample average approximation: stochastic dominance perspective. arXiv preprint arXiv:2304.0653 2023.
A. N. Elmachtoub. H. Lam, H. Lan, and H. Zhang. Dissecting the impact of model misspecification in data-driven optimization. In International Conference on Artificial Intelligence and Statistics. PMLR, 2025.
J. Fan, K. Imai, I. Lee, H. Liu, Y. Ning, and X. Yang. Optimal covariate balancing conditions in propensity score estimation. Journal of Business & Economic Statistics, 41(1):97-110, 2022.
Z. Fang, A. Santos, A. M. Shaikh, and A. Torgovitsky. Inference for large-scale linear systems with known coefficients. Econometrica, 91(1):299-327, 2023.
Glasserman, Monte Carlo methods in financial engineering, volume 53. Springer, 2004.
Grigas, M. Qi, and Z.-J. Shen. Integrated conditional estimation-oppimization. arXiv preprint arXiv:2110. 123551 2021.
R. Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69(346):333-33) 393, 1974.
N. Ho-Nguyen and F. Kilıng-Karzan. Risk guarantees for end-to-end prediction and optimization processes. Management Science, (8(12))86008869 2022.
Y. Hu, N. Kallus, and X. Mao. Fast rates for contextual linear optimization. Management Science, 2022.
N. Kallus and X. Mao. Stochastic optimization forests. Management Science, 2022.
J. D. Y. Kang and J. L. Schafer. Demystifying double robustness: comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22(4): 523-539 2007. doi: 10.121407--SSS22
Y.-h. Kao, B. Roy, and X. Yan. Directed regression. Advances in Neural Information Processing Systems, 22, 2009.
Y. Kitamura, Otsu, and K. Evdokimov. Robustness, infinitesimal neighborhoods, and moment restrictions. Econometrica, 81(3):1185-1201 2013.
H. Lam. On the impossibility of statistically improving empirical optimization: A second-order stochastic dominance perspective. arXiv preprint arXiv:210S.13419, 2021.
L. M. Le Cam and G. L. Yang. Asymptotics in statistics: some basic concepts. Springer Science Business Media, 2000.
P. L'Ecuyer. A unified view of the ipa, sf, and Ir gradient estimation techniques. Management Science, 36(1 1664-1383,, 1990.
H. Liu and P. Grigas. Online contextual decision-making with a smart trediictteen-optmiiie method. arXiv preprint arXiv:2206.07316, 2022.
12

---

M. Liu, P. Grigas, H. Liu, and Z.-J. M. Shen. Active learning in the predict-then-optiniee framework: margin-baser approach. arXiv preprint TrXxv:3305006884 2023.
Mandi, Demirović, P. Stuckey, and T. Guns. Smart and-oiiiizee for hard combinatorial optimization problems. In Thirty-Fourth AAAI Conference Artificial Intelligence, pages 1603-1610. AAAI Press, 2020.
A. Marsden, J. Duchi, and G. Valiant. Misspecification in prediction problems and robustness via improper learning. In International Conference Artificial Intelligence and Statistics, pages 2161-2169, PMLR, 2021.
W. K. Newey. Generalized method of moments specification testing, Journal of econometrics, 29(3): 229-256, 1985.
A. Shapiro, Dentcheva, and A, Ruszezynski. Lectures on stochastic programming: modeling and theory. SIAM, 2021.
A. W. van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
A. W. der Vaart and J. A. Wellner. Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer, New York, NY, 1996. ISBN 778-0-387-94640-21 doi: 10.1007/978- 1-4757-2545-2.
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the
relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope the local misspecification analysis of three date-driven stochasitic optimization methods.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
14

---

Justification:
The contextual and constrained case can be future research directions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For each theoretical result, the paper provides the full set of assumptions and complete (and correct) proof (some the appendix).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question:
Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The paper has introduced the experiment setup and problem parameters.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides open access the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
wublic/guides//odeSuubuuae
wtweeeeuuubsssisooooooooooee for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/tipss.cc/publi//guue
Codeeuumiii wettsbbmmssiiio ssionpolicyyyiiiic for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g.. data splits. hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper has provided experimental setting and details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper has provided statistical significance of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: The paper has provided information on the computer resources. Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
17

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make consideration due to laws or regulations in their jurisdiction).
preserve anonymity (e.g., if there is special
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper is theoretical and neutrally demonstrates the math theorems.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
18

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
19

---

Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: the core method development in this research does not involve LLMs as any important, original, or non-standard components
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our L.M policy http::/eeerrrpp.ecooooooaaarrr// for what should or should be described.
20

---

A Additional Examples and Techniques Details
A.1 Examples of Data-Driven Optimization in Operations Research wtih Nonlinear Cost Objectives
We now give two canonical examples of stochastic optimization problems in operations research with non-linear cost objectives.
Example 2 Prouti-Product Newsvendor Problem). The newsvendor problem has the objective function c(w, 2) a' (w 2)+ + (z w)), where for each j E [dg]: (1) 20) is the customers' random demand of product j.; (2) w is the decision variable, the ordering quantity for product j; (3) a is the holding cost for product (4) d(i) is the backlogging cost for product and (5) the
goal is to minimize the expected total cost.
Consider another classical problem in operations research, the portfolio optimization problem Kallus and Mao 2022 Grigas et al., 2021 Elmachtoub al. 2023].
Example (Portfolio Optimization). Let dn = d2 + 1 and denote the cost function as c(w, 2) 7 (w 1))² + exp((ww O)). The decision satisfies (w(1), w(22, (.d. E 1
denoting the investment fraction on products 1,2, ........e (i.e, dw - 1) and (dd=) an auxiliary decision variable. The first component represents the risk (variance) of the portfolio and the second component represents the exponential utility of the portfolio.
A.2 Further Examples of Local Misspecification
Quadratic Mean Differentiability (QMD) family). Suppose PD pd for some fixed og. Consider a sequence of vectors in R say {h Suppose the joint distribution of { Q", is also in the parametric family, but is of the form pan exists score function (z)) - de with Eo, (Co, (z)] = such that
Example 4 (Parametric Perturbation:
Both. If there
2
VPo.
00 hr po:
dz ho 1), hn O.
In particular, in our framework. we focus on the case where hn h/n" for fixed vector h. When 1/2, van der Vaart (2000) shows that the likelihood ratio between Q" and pm satisfies:
dQ" (Z1, Zn)
1 h Ih opn -
log dP 21,
(o.
Opm (111)3),
where I : Eg,, Leo e denotes the Fisher information. In other words, under pm
dQ" (2) 2n) P- N - Ih, h Ih),
log (21, 2n)
where the limiting distribution a Gaussian distribution with mean 3 T Th and variance h Th.
In the previous example, the ground truth distribution Q is still in (Pa 0 E θ but is in the local
neighbourhood of Po.- The more common and interesting examples are when Q g (Po 0 E + as discussed in examples below.
Example (Semi-parametric Local Perturbation: Part I). Suppose Pa, is given distribution, and w(=)= Z - IR is an unobserved random variable with Es, [u] = O and a finite variance [n, For scalar in a neighborhood of zero, we define the tilted distribution of Pa,, called Qr, as
dQ{(z)= exp(tu(2)) (x)
C.
where C. (x)((t(((()) (z) oc normalization constant. Clearly Qi=o Pe. Lemma 2 (Log Likelihood Ratio Property in Example 5). Under Definition when = 1/2, i.e.
Q" = 0000 1/ the log- likeliho ratio between Q" and Pm satisfies:
d2 ... Zn)
a
log d. 21, kn)
Σ
u(z))
Ea, [u-2] + Opp (1)
21

---

dQ ( P N(- 2 Fo [u²], Ro, [u²])-
This implies, under P log aPm (3
Example 6 (Semi--araman parametric Local Perturbation: Part II), Consider the random variable u(z) 2 - R with zero mean, Ee, [u] 0, and finite second moment, say Es, [u²]. Now define the tilted distribution:
dQ.(z) = [1 +u())+)]+ dPo where C. / lu(2))+ dPo.
C₁
In particular, in our framework we focus on the case where Q Q 1/n and Q" = Q//n When 1/2, by Duchi 2021]. the log likelihood ratio satisfies
(Q Zn) log dp 21 2nn
1
Σ 1=1
2 u(zi)
Eoo 1uo + 0pm
In other words, under PM
dQ (2) kn) 4 N(- Bo,lu"), [u²),, Ea, Eo, (u²1). log dPP 21, y.., Zn.
Example 7 (Semi-parametric Local Perturbation: Part III), Consider the function g R + 1-1,1 be any three-times continuously differentiable function where g(z) for [-1/2, and g' and the first three derivatives of are bounded. Consider the random variable ==)) : - IR with zero mean Ee, [u] and finite second moment, say Ee, [u²]. Now, for t E. we define the tilted distribution
dQ+(=) = 1+ [(b(((z)) dPo, (z) where C. = 1
=1 / = g(tu(())dPool
Ce
== In particular, in our framework we focus on the case where Q Q//n and Q" == 1// When 1/2, by Duchi and Ruan 2021 the log likelihood ratio satisfies the following Property:
dQ" kn) log dpm 21 2n)
u(z))
Eo. [u²] + tpp (1).
2
1=1
In other words, under Pm
(2)( kn) P. N(- Eo, Eo,lu" [u=]], Eo, (221)
log dPn 21 2nn)
Example 8 (Semi-parametric Local Perturbation: Part IV (QMD Family)). Consider scalar function u(z): Z R with zero mean Ee, [u] 0 and finite second order moment Eg, [u²]. We define a tilted distribution Qe for t E R with probability density (mass) function ge with respect to the dominated measure (note that Qe is not necessarily in the parametric family (Pe θ)) with qo = Po. We further assume the quadratic mean diffcrentiability
1 (Va- vpo tuvpo po. dz o(t2).
Note that when Pe,- In particular, in our framework we focus on the case where Q = Q1/m and Q" = 1/ When 1/2, by Duchi 2022 we have
dQ" .... 2n) log dPn (z) (z,,..., kn)
1
Σ u(z))
Eo, [u + op..
vn
Note that Example 8 is the most general version and includes Example 6 particular examples under some mild assumptions.
22

---

A.3 Additional Technical Details
We introduce standard regularity assumptions for general Meesimaation problems asymptotic statistics |van der Vaart (000), which include our SAA, ETO, and IEO methods as examples.
Assumption 3 (Regularity Assumptions for M M-estimation). Suppose the i..d. random variables { 1z,) 1 follows distribution Q. Suppose the function + my m²(z) is measurable with respect to for all Ç and
sup Eis m<(2:) - E&g (mnc (mc(2))] & 0,
2. there exists 5* argmax, BQ [mç(z)), for all > 0, supells-s-112 BQ [mz(z)] <
IeQ [mc- (2)],
3. the mapping Ç + mz(2)) is differentiable at S for Q-almost every z with derivative Vemg-- (z) and such that for every (1 and (2 in neighbourhood of S and measurable function K with [Q[K(z)
(x))a]
mc. (z) mça (z) ((=
4. assume that the mapping & - Eq[m<(z)) admits second-order Taylor expansion point of maximum Ç with nonsigular symmetric second order matrix Ve-.
If the random sequence En satisfies - m2g (zi) supp Ei=1 me(zi), then &n 3 25 and
Vnl (- -²) Vn Σ
V m + o1
Throughout this paper, we assume Assumption holds.
For SAA, consider m²(z) -c(w, 2) with the parameter (
For ETO, consider mc (z) = log (z) with parameter (=
For IEO, consider m²(2) c(wo, 2) with (= 0.
When we say Assumption 3 holds, it means that Assumption 3 Sholds for the corresponding m²(2) in SAA, ETO, and IEO. Assumption 4 (Interchangeability) For any 0 Q and w E Ω
Ve / V Tw po(x)d( V Vwolw, 2) TVoo,,))))oo((dd) Vepe(z)dz,
V Wwo(w, (acc(w,)poll 2))o(z)dz = Vw ((w,,)po(2)d2))
The interchangeability condition in Assumption is a standard assumption in the Cramer-Rao bound [Bickel and Doksum, 2015). A standard route to check the interchangeability condition is to use the dominated convergence theorem. For instance, we provide a way to check the first interchange equation. (=(z) 22) is continuously differentiable with respect to and there exists real- valued function q(z) such that V Vwo(w, x) x)xxx) [[((z))2 < +oo and || [=)) S 9(z), then have Ve f V w. w. 2) pe (z)dz= J V o w, 2) Vers (z)dz. Other sufficient conditions (more delicate but still based on the dominated convergence theorem) can be found in L'Ecuyer 1990].
Asmussen and Glynn| 2007). Glasserman [2004].
Next, we present some auxiliary lemmas that are helpful for deriving our theorems.
The first is a classic lemma in asymptotic statistics, called Le Cam's third lemma (Example 6.7 in van der Vaart (2000).
Lemma 3 (Le Cam's third lemma). Let pm and Q" be sequences of probability measures measurable spaces (Ln, Fn)) and let Xn, be a sequence of random vectors. Suppose that
dQn
5 N ((
Σ
2²))
X X log dpn
002
r
23

---

then
Xn - N(u + + E).
We now state auxiliary lemma about the directional differentiability of the optimal solutions to stochastic optimization problems.
Lemma (Directional differentiability of optimal solutions: Part I), Consider the distribution Qx(z) in Definition 7
Let
Wp == argmin Eq, [c(w,z)) z)].
wEn
Then under Assumptions
and 4
lim t00
(w; wo) = Ea, (=()))
Equipped with the lemma above, we can get the convergence of n (w wo.) under the three locally misspecified regimes:
lim n n20000
-w, - -wo)) = lim 2-0 [
(w, wa) = Eo, [
Proof of Lemma 4 Note that w is the minimizer of E (c(w, 2)] under Q" while wa, under Pm, We will use the directional differentiablity of optimal solution to derive this fact.
We denote v(w, Qx) as Eq, [c(w, =)]. G(w, t) v v(w, Q+) and that ((w,, 1) = 0 for all By implicit function theorem,
argmin o(w, Qe). Note
8
lim (wr w/) -V w (wwo. 0)] V v Qe)le= t0
2 Vww(w 2c)l==
wow E.o (c( (oww 2)] 0r
V-1 i
Vuolwo )dQ(
-V /
Vwo z) 0t
dQu (z)(())l=00
From Definition almost every z,
we know that for almost every z, 0e log (2ogg(2))een = u(z). Hence, we have for
Oe (())) 1-- (0)2))(2))
(3)
In conclusion,
V-1 / Vwelwes z) d)( 1=
/ V Vinc(wa, 2) (00 (z)u(z)) dz
Since
/ Vwe(WWo:: 2) / (0(()))) (0(2))(z)dz go(2)dz // [oo()))(z)ddz V. 2)] -0
and
/
= (llll))Vwe(Wo,, =)].
we have v-1 / Vwr(wos, 2) dQi(<))e= -V- Vwz z) 1Ba Eao,uu(a (U()))SAAAA
24

---

Therefore,
lim Vn(w" - wo,) = lim n
(w, -wo) = Ea, [=(==)]SAAA t0 t
More generally, under severely and mildly specified regime, we have further
lim n" (w we,) Ee, (u((z (u(z))IFAA(()) (z)]. 2-+00
We note that Lemma holds for Example 5 q: (x()) exp(tu(z))
To be more specific,
C₁ (0(z) where C2 / exp(tu(z))go (z)
Therefore, the derivative of (x) with respect to is
① at (c(2) 20(2) (xp((u(=))vll exp((u(2)))0( (x
[ (x)
[ exp((t((z))gn (z) (z) exp(t(()))
[ exp(tu 2))(
At t = 0, since Eg, [u] = 0, we have for almost every 2,
i
// (0(z)u(z)dz go(z) - (=(=u((=).
(z)
1=0 (z())((z)
also possible to extend the result to other examples under additional regularity assumptions. For Example the result still holds. Recall that
(z) [] (L+)((z)]
g0(2),
C. = (1)+ (0(2)dz.
C2
Hence,
Ct it
[1 + tu(z)]
[1 + tttttt)))))))
0t Co
Ot (z)(z))== 1=0 =z)
07
1=0
2
0
(=()))
tu(2)]
Ct
0l
1=
1==0)
/
(z)))
(u(z) (1 tul (z) 2 0} 1=0
tu(2) 2 0) (0(2)dz
1=
(=)))
[(()) (=))g((z)dz
The result is the same and the conclusion of Lemma 4 still holds.
For Example 7 the result still holds. Recall that
(z) 1 - g(tu(z)) (0(2),
G== / (1 + g(tu(z))) (z))dz
C₁
Hence, by noting that g'(D) = 1,
Ct Ct 0t
0 (1 + g(tu(2))) - (1 + g(tu(2))) a Ct
0t [ qu(z)
C/
1=
i
a
20(z)
(1 g(tu(z)))
0t
== de
((())g(0)) (0) ( (()))((0))o(z)))
(0(2)
[(())) - u(=)q0(z))d
The result is the same as 3 and the conclusion of Lemma 4 still holds. We provide another auxiliary lemma similar to Lemma
25

---

Lemma 5 Directional differentiability of optimal solutions: Part II), Consider the distribution Qe in Definition 7 where Qo Po.. We denote
0 argmax Ee, (lo(opgp(<)) pz
lee
O argmin Eq, [c(wrp 2)!-
o
Then under Assumptions
and
we have
0 kl. =
r-0 1 (0 0o) = Eo, (ttuz)|Ftt)) (IFFFT (z)],
lim
V.Oi = lim (0; 0o) = Eo. [u(2 [I) t-0
Proof of Lemma 5
poollogo(>>>> po (0. == Ver Qi) and (0, Qt). Note that GKL (0KL t) for all t. By implicit function theorem,
We denote u (0, Qe) as Eq,
0eL := argmin e
lim (0kL) 0o) - (Vog (0o. 0) vou Fl (O., Q:)l=
t0
at
d
Voo (log (z)
vou KL (0o 1e)le=
r- ②
Ot / se, (2)d(()))
/
800 (z) 1
r-1
dQi(z))
01
At 0, by
0
0t (()))) =0(2))(2))
In conclusion,
r-1 / s0. (z) 8
at d() -
-I- 1 / so. (z) (90())) u(z)} dz.
Since
I so0 (z) If (o(())(())d) 40(2)dz // co(=)u(z)dzz Es, [z)] = 0
and
/ 80. (=)0())(((d) = Eo (o(((())00 (z)1,
we have
1 [-0 7,
(0.L 0o) -r 80, (z)
2 (2)1 1 - (oz)((z))000 (z)].
lim
01
Similarly, we denote (0, as Eq. [c(wa- 2)}. G (0,tt) == Vov* Q.) and 0; = argmin 8² 00, Q+.Q.). Note that (0, t) 0 for all t. By implicit function theorem,
lim (0) - 0v) = V e (0.. 0) t00
St Vov (Oo, Q:)le=
Voo lc x)
Vor (Oo, Q+)e=
at
σ1. / Vec(wa, z =)dQ; 1=100
$
-1 / Vec(wo z)
0 dQ, 1--
01
26

---

At = 0, by
① Øt 9((2) 1=0 (0(2)u(2).
In conclusion,
④ / Voc(we, 2) ① (z)((()) 1=-
④ -1 / Voc(wrs 2) (((0(())(()))) dz.
Since
/ ())0(()))))) = Eeo
2)],
we have
0 dQ: $ -EEE,, [E,,,u(2)) (oo 2)].
lim (0, 0b) --1 Voc(uro, 80
0t
We remark that Lemma 5laso holds for Qt in Example G
B Proofs
In this section, we supplement the proof of the results in this paper.
Proof of Theorem 6 We first notice the fact that, in the mildly misspecified regime, by defining
hn = 1/(n" 1/2) = (1). we have
dQ" (21,....... 2n)
1 - Eo, [u² Opm (h₂) 0pm (1).
log dPm (21, .... Zn)
(2,)
2
In the mild misspecified case, under p", we have a joint central limit theorem
Vn(wu= dQ" wa.) P"
(10) vare, (IF (z)) 8)
log apm
Using LeCam's third lemma, we change the measure from pa to Q" and get that under Q",
VT(@W= wP.) 2 N(0, varo, (IF (z))).
Using the same technique,
n (w, - wa + Eo, (U(Z)]FSAA()) Vn(w", wa,) + O.
In conclusion,
- w vn(w=-ww)
v wa,) vnn wa.) 2 N(0, vare, (I))
Let us now consider the regret. We use Taylor expansion of the regret with respect to w at w and note that Vw (ww", 0 for every n.
V wow (w;) - wi) + 00 vo00(100 wi |3:
un vn(1))) vn(w)) = (w0 ww;)
2
n(vn(11) - en (w,)) = Vn(w
1
0 wi)) V &n (w Vn - wi) + too= (1).
2
By Assumption that V www (w)) + V. the function f Ω + R with f(-) == (-)
and function sequence fn. Ω IR with fr(()) {()) V ww satisfy: for all sequence {w,).. if w, w for some w Ω. then fa (w,,,) ) + f(w) since continuity is preserved under multiplication. Using the extended continuous mapping theorem (Theorem 1.11.1 in van der Vaart and Wellner 1996)), we have under Q",
n U, (w;))
2- N VVN.
2
Moreover, ETO is stochastically dominated by IEO and IEO is stochastically dominated by SAA.
27

---

Proof of Proposition 7 noting Lemma !
The asymptotic normality result is directly from van der Vaart 2000 by
The asymptotic normality of SAA is by Proposition 2A of Elmachtoub et al. 2023 For ETO and IEO, Proposition 2B and 2C of Elmachtoub al. 2023) shows that
vn @ETO
0o)
+ 1),
vn 1 0o)
M" N(0, $ varo.
vare, (Vac(wo, z)) $ $-1).
Regarding the notation, varg, (Voc(wa, 2))) the variance of the random gradient Voc(wrg 2) at 0 = On. under the distribution Pa.- Note that the subscript 0g under the variance is not variable here. Using the delta method, we have
vn
EETO
0o)
P" (O, V V I Vewa.. = (O, ΣI '
V-1) = NNO vare, (Ff=T((2)))
Vn
(iuEO - 00)
- N (O, V Voweg +b vara. (Vec(wos 2)) -
(O, V
ΣΦ 1 vare. ( 2))$ V-1)
((, vare. (II
The inequality varg, (IFETO & varo, (2) $ vare, (IFSAA is from Theorem of Elmachtoub et al. 2023
Proof of Theorem
We use different decomposition framework this time. We recall
0KL argmax Eqq (log
dce
0 argmin Eq, (c(wro =)],
0ce
W := argmin Eq, [c(w, =)].
wef
We denote in : 1/n0. Note that here wi,, = -, but generally wor. + w;, war, X w this case,
w
ETO
" = (10 ETO Wpm. ) (Wom. w..) -ww, - we.).
w IBO
w. = (w TEO wa; + (wer. wo,) - (w), we.):
i SAA
w SAn w, ) (wi, wo.) (w wa
We already show in Lemma 4 that
n nm",", wa.) Eo, [u[ (ISAA)
Next we give a limit of the middle term, using Taylor expansion. For SAA, the middle term equal the third term. For ETO and IEO, wa, wa; l==0 and wa, - l==0-
Wo; We, V,wa; + o(t) Vowg V.0, +
Wøkt we V.w o(t) Vow, V.O. + ott).
By Lemma we can get V,0; and V.OKI. at t = O:
v, KL. = - I [u(2 so, (z)],
V.0; = -- T -Eo, (112 V 2)].
Moreover,
VeWOKL Vew, T -V-1-11 13 Ee, (u(z)se, (1(2))F (z));
V,w0; Vown V.0" = - -V- ΣΦ " (oc z)1 = Eo, (E(
28

---

Finally, for the middle term,
n² waa Wp,) + Eo, (u( )IF (z)),
(Wer.. we.) Es, (u( (z)(FFTo (z)).
n (wi,. we.) Eo, (u( 22)FFAA (z)).
For the first term, under Assumption Oq (L/vn), then
2 (w Wore ) (wil wo; ) and (20SAA w7, are all
(w ETO Wpye 4 0,
(w
n (w wo; ) P 0,
n² n" (w wi 2 0.
When we multiply n²", the term shrinks in probability to O. In conclusion,
n" n" w 2 6P.
Let us now consider the regret. We use Taylor expansion of the regret with respect to w w and note that Un (ww;) = for every n,
V ww (eriinn(1 (10 wx) + ogm 100 105--wjn will?
Un (1 (
(1i w,
2a (Un (00) - Un (ww)) = 2
(10 - w;;) V Un Un (w) n (w - w + tog (1).
By Assumption that V Uw - V, the function f Ω + R with f(-) == 2.)
and function sequence fn Ω IR with f.(()) (.) V wn (.) satisfy: for all sequence St then fu (w,) f(w) since continuity is preserved under
{w )= if w,, for some multiplication. Using the extended continuous mapping theorem (Theorem 1.11.1 in van der Vaart and Wellner 1996]), we have under Q",
1 (69) vOo. & 2
n n22 (vn(w - Un (wx))
Proof of Theorem 4 Recall the influence function of SAA, ETO, IEO:
(2) -V V (Wo, z),
IE0 (z) = -V E V Σ) VVV--
V w (w 2) -V Px, V wwwlwo 2):
IEE (2) = -V- -I 80, (2) = V Egg (so, (2)80, [=00 (=))(z).
For regret comparison, since SSAA 0, we have RSAA 0. Also, RIEO and RETO 0. By noting that V = Vew [c(wo, z)]. we observe that
Eo, (u(z) ((()))F1B0(()) (z)) VEg, (u( (2) (z)] Eg, (u(2) (u=2) (2)] VEg, (u(2)IFSMM(2)
This is because
Eg, (u(z))) (1)))IIBO0 (z)] VEg (u(2)]]] (2))FI (=)]
Eo, [[(2]] w (wo, 2)} V 1 V- -1 ' V V.
V - (u(2 V. c( 2)1
-Ea, (u(z) cwoo V
ΣΣ Σ '. V- 1----EE 'Ea, [u(z) V aaoo 2)]
-Fa, (u() V c wro 2)
VV E (- -1 -N VV- (u(2) v 2)]
-Eeo (1=))) T VE,, (=>)) 1F (=))
-Ee (1(2) VE,, (u(2 (FSZ) (z))
29

---

Now let us prove RETO_ RREO= 2 0.
2
(Fo)((())(1F)))) - [x))) Eo, [u(z) (1= (=))
-Eo, (1(2 (z)) VI (u(z))FETT(()) (2)]
- -2Ra, (u(2)]FFto VEg ju(2 111 (z)] + Eo, (u(Z)IFSMA())) VEa ([(z)|FSAM(2))
2R
-Eo, (z) (= Vo, [u( (I=E [2)1
-Eo, [u(z)) [u(2))IF15(( [z)] T V VEe, [u( ))IE (=)]
-2Ba, (2(2)) )1 V Ee, (u( (Z))IFFAAA (=)] + Ee, [u(=)]][AAA) (z)] Ve, /u( (())IFSAA(>))
Ee, (1=) (x)] V /u( (IAAII + Ee, (u( (I)AAI()IIAA)) (z)] VIE, /u(2 (1((=()SFSAAA)))) (z)].
Hence,
2RETO 2R
=Ea, = (2) VEa (u( (u=)))=o
- 2Eo. [u() IIF VE,, [u(2) III + Ea. [u(2) (===) (=)] VEo (u(z) (I)
_Eo,lu(z)s0,(2) (2)]
Σ VV
-I- [u(
so, (z)]
+ +2Eo, (0
r
Σ
Eo, [u(2 (222) V w (wo, 2)1
+ B w 2) V
E V 10, [uz) z)]
- _Eo,[u(z)s0
V- r
Eg. (u (z) (=)
+ 2Ea, (u())s (z)]
I - V V Ee, [u(2 V w. (w.. =)1
+ Ea. [u(2 2)) 10 (00, =)
T V Σ E V Σ) ΣTT -TV-'E
Eoo [u( (z) (/lz)(///ii vie((w. =)]
( 1/2 r I'
wd - (1>--()) - 1/2 V V- (e)e,,(()))))))))))){())))))[[ V 2)1
20.
The last equality is from the fact that
- (A ) T (Atx-A-by) A A
Ax 2x y + y A
x
In conclusion, we have
RETO RIEO > ISSAA 0.
By the definition of 6 and RD. we know | v V2R root function, we have 0 = ||S v I
Hence, by the monotonicity of square
||6 v S ||
Proof of Theorem 5 Part (i): We note that when u(z) = BT se, (z) for some B E Rdo,
BETO
= Eo,, [u(x) 1I=))IFETO( (=) - Ea, [u(2))) (FAA)I (z)]
V 'Ea. /V co Wa.. 2)00. (z)T] (Ea, [9a (2)80,
80 (z) Es, [u(=)s) (z)] - V Eoo Vno z]]
=V -V-'RO /V 10 w 2) 20 (z)7] (Eeo |00 (z) 800 (z)Ti
cl -1 Eo, (BT 800 (2) 80 (=)] V EE, 13 T 80. V curo =)]
-Eo, -V-IE -V
/ cn Wa 2) 2) (z) ] (Ea, |00 (z) 80 (z) T))
-1 Eo, |80 (z) 80) (2) 3 V v-'Eo, V n (wa, 2) 2)s0, (z) }
-VR V cl wo, 2) 2)50 (z) ] (Ea, [so, (2)80 (z) T)
-1 EEo, (s0, (z) (2)800 (z) T V V- V Vw V (wo, 2) s0,0, (z)T] 1|0
V Eoo V co (W), 2)80, (z) 118 V-E /V wwwew z)80 (z) T13
0.
30

---

when To prove Theorem 1 we need useful result here. When a 1/2, we can show that the log-likelihood ratio is asymptotically normal characterized by the mean and variance of the perturbation direction. This result is used to convert the asymptotics in pd to Q" by conducting change of measure from pa to Q", and also contributes to the overall asymptotically normal limit of the decision that encompasses the bias term. It will also be leveraged later to prove results in the mild misspecification
case.
Lemma 6 (Log Likelihood Ratio Property in Definition Duchi 2021 ) Under Definition 2
a = 1/2, i.e. Q" Q/// the log-likelihood ratio between Q" and P satisfies:
dQ (EI, zn) log dPm (z), Zn.
u(z;)
Eo, [u-] + tpp
am1 2
Proof of Theorem
! By Lemma aand Proposition under p", we have a joint central limit theorem
Vn(w -we))
pm N
vare, (IF (2) ([ Le, (u²)) Ee, [u( (I) (z) Es, (u(2)) [u(=)]F= (
([-EEo
log JP
dQ"
Es, (u²)
Using LeCam's third lemma, change the measure from pa Q" and get that under Q",
vn(w" we.)
2 N (Ee,, [u(2]IF (=)], varo, (z))).
Next, by Lemma (note that this is not stochastic convergence but deterministic sequence convergence)
Vn(wū, wo,) + Eo, [f(()]IFAA>>>] (z)].
In conclusion,
vī( - wxr)
Vnl 1 wa.) - vn(w", wa.) 2 NbP, vara, (IF (z))).
Let us now consider the regret. We use Taylor expansion the regret with respect to w at w and note that V Vw (w;) = 0 for every n1
1
(w= - w,, VV ww Vw (w wi,) +Qq 1//0/00 10000 -0:--::ll will?
Un (102) vn(w
2
n(v Un (ww"))))
1
Vn(w0= wii) V wwwen (w)) Vn(w - - +
o0p (1).
2
By Assumption 2 that V Vun Unwwnn(;;) (w)) + V. the function f Ω + R with f(-) = VV)
and function sequence fn Ω IR. with (x) {0) V un (n (.) satisfy: for all sequence {w,) )_- if w,, - w for some w E Ω then fm (w,,) + f(w) since continuity is preserved under mulptiplication. Using the extended continuous mapping theorem (Theorem 1.11.1 in van der Vaart and Wellner [996]), we have under Q",
2 1 N
n(vn(w))) un (ww)) 2
Proof of Theorem
Recall that:
Vn( -ww)) m(vn(10)) Un (wi)) 27 GP 2 { T
wx 2 NT_ -N6b,, varg, (IF (2))).
VV-
By denoting 6= as Ee, (u(2) (1(=)( (rSAA(2))) we can rewrite C as
G
(0, vare, (IF (x))) - 8F) v (N(0, vare, (x))) 6d)
2
1
N(O. vare, (IF (z))) TV vare, (IF (IF (z))) 2 (b7) T
VN((0, vare, (IF (=))) + (b7) von]
2
31

---

By taking the expectation, the cross term is zero. Hence,
G
2 NO, vare. (IF (=))) T (O, vare. (IF (2))) + (t") T
von
Since varo, (IFETO (z)) I vare, (I)) S varo, (IFSMA(2) (z)), know the stochastic dominance of the SAA, IEO and ETO, and their corresponding expectation.
(O0 varo, [IF)))()))) VN varo, (IFETO(2))
NO, vare, (IF (=))) VO, vare, (IF (2)))
(O, vare, (IF (z))) VVN vare, (IFSA) (z)))
and
VN E [O, vara, (IF (IF)))(2))) (z))) vara. (IF (z)))
E (O, vare. FFF (z))) VN(O0, vare, (IF
E [[00, vara. (II SAA (=))) (O, vara. SAA
From pervious analysis, we already know
(b V
(bTOO) Vb IBO
(bsma) VoSA
Therefore, E(GP) consist two terms. For the first term, ETO less than IEO, and IEO less than SAA. For the second term, the direction is flipped.
Proposition was essentially established by Ellachhoub al. [203]]. but here, we express the asymptotic behaviors of solutions more explicitly in terms of influence functions. Moreover, these more explicit expressions arise from a new projection interpretation influence functions that allows us to describe the performances geometrically, providing another perspective different from Elmachtoub et al. 20033]
To this end, let P be the projection matrix onto the column span of E with respect to the norm ||>||| y i.e., i.e.
Pæ = argmin |yy - x1
y:yecoll E)
which has closed-form expression P Σ (ΣTV~l)) V Second, define the functional T LooP jdw + L2(Poo) dm as the projection operator onto the linear function subspace (Aso, A E xRd J, i.e., for square integrable function f(z) 2 + Rdw,
/ [lf(z) g(z)l12 pa. (z dz.
Tf
argmnin a-== Aso (#)
Theorem Under Assumptions following projection interpretation.
and 4 the influence functions of IEO and ETO have the
1. (z) - PV cl Woo. 2).
2. IFETO - VVV w c(we, z).
The above theorem points out that the influence functions of IEO and ETO are essentially projections of that of SAA, either in vector or function spaces, shedding light on the ordering of their variances by the contraction properties projections.
Proof of Theorem 7
The fact that
IE0 (z) -V PV 2)
because
IE0 (2) V ΣΦ Voc(wa. 2)
-V-'E V Vou - -Vows Vucluro, 2)
- 1-E
V Vwc(Wo,
= =-V- Vw 2).
32

---

We then show the relationship between [2) and IFSAA(z) (z). Let T (p(,,,) )a + L (po,)) be the projection matrix on the linear function subspace (Ase, (z) A E Rd Xdo ), i.e. for general function f(z) : Rdo,
Tf argmin Age
/
- 9(=)) poo poo (z)dz.
The influence function of ETO is also projection, i.c.,
IF (z) V VVV w c( z).
The reason is follows. For ETO, it suffices prove the following fact:
Tf Ea. (f (
s (z)
To prove the fact, we need to show that A Ea, (f(2)s0, (z) I is the minimizer of the optimization problem
since Σ = E w Sooe(wee we. s)s0.
/ /II((=) Ase. (=)l²² pe. (z)dz.
min AER xdo
Since this is essentially quadratic optimization problem, the stationary point the global minimum. Denote the objective function h(A) and require V ((A")) 0. In other words, for all i, ah (A) = O. For simplicity, we write Pa, (z) as p(z) and se, (=) as s(z). Note that
du
do
M(A)= = (i(()) - Σ
ZEZ t=1
Ø=1
Σ
fi(2)² + E Anjs((2))² 2f((z) Σ Ayss(z) p(z)dz
i=1
We have
0h(A))/0A33= = JEEZ
-2f((z))(()+ + 2 2 A3ys p(z)dz
For all i, we have
J Jzz f1())s;<))((2))
/ Σ (2)
de
Writing in a matrix form, the left hand side is Ea, (f(())(2)) Eg, [A [=={(=)(x))]]== [s0,)( I In conclusion, A = (()()))(z))))= and
T) The write hand side is
Tf = (((()ss (z) ( ( (z).
Proof of Lemmal
7 The first identity follows from
Vo v ww 0) w) 0=00
VoEa V Wwo(w, x)]l0=00 w=
Vo / Vwe(we, z)pe, (z)dz
/ Vero (z) Ve) =) dz
/ (o Vologro. po: (z) (V cl 2)) dz
Ea, (2) (V m0 (Wa, 2)) 7.
33

---

For the second identity, by implicit function theorem and applying Barratt 2018 we can prove the first identity
0 = V "ww"" Di w. wullw=wa, (Vewolo=o,) + V Vov w. 0) ww 0=001 0=0 Vo V Vwo 0) wo 000
Vowa o- -=00 0o
Vom 00) "wo
V
The third identity follows since
VeelLa [c(we.: =)]
Ve Ea, (d(wa, =)] Vewoo 0=0.
= Vows V ww Be, [c( [c(wo,: =)] V Vowg
= (-E V (-
E V
by noting that Eo, [c(wa, z)] 0 since wa is the minimizer of the function w - Ea, [c(w, z)].
Proof of Lemma 2
dQ (Z1, 2nl
exp(u(zi)/VI)
n u(z;) - n log C1/va.
log d.Pm (Z1, Zn)
log
Σ
C1//a
i=1
It now suffices to show that n log C1/v= = Ba, [u2] + Opn (1). From the definition of Ct. we know
C. / exp((t((2))dPo, (z).
Taking the derivative, we have
(C+) 1 [xp((((2))u(()dPPaa)) /
(u(()(
Taking the second order derivative, we have
(c=)" = [epptuule)luubbasaddedeeeeleleee = Es, (u²).
By Talor expansion, we have
C. 1 Eo, /27 + o(t2)
In conclusion,
1
+o (1)
(
/ + ( (1)
1
log Ci/vz nog
1
Ee, (2
Eg 22
n
VT
n
vn
= 2 +
Ro,
34


Reviews:
[{"review_id": "vPNemfFpWM", "reviewer": "Reviewer_JYdJ", "summary": "This work compared SAA, ETO and IEO performances from the bias-variance tradeoff perspectives, under the setting of local model misspecification. They provided the ordering comparison results of bias, variance and regret under balanced, severe and mild misspecification scenarios, respectively.", "questions": "1. This work is more about the asymptotic results on the comparison, and you mentioned you leveraged tools from asymptotic statistics, is it possible to extend the current results into finite-sample regime based on current toolsets, or whether it is an important problem?", "limitations": "None", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. The work has a comprehensive comparison on the performance of SAA, ETO and IEO in data-driven optimization\n2. The work leverage several tools from statistics, which should be uncommon in the literature and is novel in terms of the techniques.\n3. The writting of the paper looks good to me.\n\nWeakness:\n1. Some assumptions can be further rationalized:\n   - Line 173, why do you require the minimizer to be in the interior of the space? I am wondering in the common linear programming cases, the solution should be at a vertex, and the assumption may not hold (also uniqueness due to common nonconvexity in ML objective functions)?\n   - Line 178, you assumed Q could deviate from the model in a certain “direction\", I am wondering whether it is a reasonable assumption, whether it is implementable in real applications, is there any approach to detect the direction.\n2. Comparison with existing work. A closely related work should be Elmachtoub et al. [2025] as you cited, they divided their discussion by the dgree of misspecification $B_0$, and here authors divided the discussion by the magnitude of $t$ (exponent $\\alpha$), which looks similar to me, just like set $B_0=1/n^\\alpha$ (is that true?). So I am confused why do you say \"there is no smooth transition in between that captures the impact of varying the misspecification amount.\" in Line 57. Even though authors mentioned the direction $u$ may take effect, but Theorem 5 seems to work in some specical case. So I hope to have more illustrations on the improvement from the techniques you employed.\n\nTypo:\n   - Line 285, what is $O_{Q^n}$, also why there is no norm for the three errors?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your recognition of our significance and strengths, and your detailed review. We address your comments and questions as follows:\n\nW1.1: Let us respond in three parts:\n1) Our paper focuses on non-linear cost objectives, and our assumptions are in line with the standard statistics [2, 3] and optimization literature [4, 5, 6] that presume openness of the decision space and hence that an optimal solution is an interior point. Note that this assumption can be relaxed by adding equality constraints and inequality constraints -- We expect analogous results hold by utilizing KKT conditions, but the conceptual insights would be similar while adding significant technical complexity, and hence we leave this extension as future work. \n2) The uniqueness assumption is also standard in nonlinear optimization; See [4, 5, 6, 7] (some of them even assume strong convexity). Note that this assumption may be weaker than you would initially think, because the uniqueness applies to the expected cost, not the cost itself. For example, in the newsvendor problem, a classic example in operations research, the cost function itself is piecewise linear and is not strongly convex. However, the expected cost is indeed strongly convex if the distribution of customer demand is continuous and has a lower bounded density, therefore, the optimal solution becomes unique.\n3) In fact, the linear programming setting raised by the reviewer is easier than our nonlinear setting from a statistical perspective. This is because when the cost function is linear, it suffices to predict the expectation of the cost vector, which is a point prediction [1]. Because of this, it is not necessary to fit the whole distribution, as we must. In contrast, our developments require the notion of model misspecification that is at the distributional level, since the nonlinearity of the objective function forces a more complex relation between the randomness and the decision variables.\n\nW1.2: Yes, the assumption on $Q$ is reasonable. The form originates from the local minimax theory and semi-parametric statistics in [2] (Section 25, Eq.(25.13)). In other words, it is a smoothness condition saying that the distribution $Q_t$ is differentiable at $t=0$ with score function $u$ in the $L_2$ norm sense. This condition plays a central role in ensuring that the log-likelihood ratio behaves locally like a quadratic form, which is the cornerstone of the local asymptotic normality (LAN) framework. On the other hand, in practice this misspecification direction is not known (otherwise, if it is known to the modeler, it would have been incorporated in the fitting so that the misspecification is reduced). Nonetheless, we highlight that, as  a strength of our main results, the orderings of the bias and variance of the three methods are universal regardless of (almost) all directions $u(\\boldsymbol{z})$, but only depend on the degree of misspecification. That is, the deviating directions $u$ of $Q$ turn out to be an invariant factor under most scenarios. \n\nW2: First, note that the statement \"there is no smooth transition in between that captures the impact of varying the misspecification amount.\" in Line 57 refers to [5], not [8]. Indeed, [5] divides the analysis into either misspecified or well-specified cases and, unlike the current work, there is no notion of ``local misspecification\" there that allows the ground-truth data distribution and the model to be close in a certain asymptotic sense.\n\nNow, as the reviewer rightly points out, [8] has a similar favor as us in that they consider the degree of misspecification. However, the derived bounds of [8] are opaque and loose. Specifically, their degree of misspecification is denoted by $\\delta$ and $B_0$, which is problem and method dependent, is not interpretable and just for technical convenience. They use the so-called Berry-Esseen bounds, a uniform tool, to compare the performance between ETO and IEO by tail probability bounds, thus making the analysis coarse. Even though they can conclude the partial ordering between IEO and ETO, the involved conditions including the threshold for $B_0$ and $\\delta$ in Assumption 4 and Corollary 1 are difficult to understand. In contrast, and in fact as a remedy to [8], we introduce the notion of local misspecification that allows us to explicitly derive asymptotic limits, dissect the bias and variance, and even understand the impact from the directions of misspecification that are well beyond [8]. \n\nFinally, we cannot set $B_0=1/n^\\alpha$ in [8] because $B_0$ there is assumed (and required by the analysis) to be a fixed number that cannot be changed with respect to the sample size $n$. In our framework, however, we allow  dynamics where, roughly speaking, the distance between the ground truth distribution $Q$ and the model $P_{\\theta}$ has a comparable order of magnitude with the sampling error in estimating the parameter $\\theta$. This, together with our developed asymptotic framework, empowers us to dissect bias-variance accurately, especially in cases where these errors are comparable (the interesting case), as well as directional impacts. \n\nQ1: As explained in our response to W2 above, finite-sample results are interesting, but like in [8], they are anticipated to be intricate and do not offer the clean analysis that allows us to dissect bias-variance and the impact of misspecification directions. We do understand that there are merits for finite-sample bounds (just like in other ML problems), but at the same time it is also the case that these bounds very often are conservative and loosely depend on model factors. A tight enough finite-sample analysis that allows us to dissect bias-variance tradeoff meaningfully would be very interesting. Lastly, please kindly refer to our response to Reviewer JBev on our new experimental results that validate our asymptotic theory and its demonstration in finite-sample manifestation.\n\nTypo: Yes we will add the norm for the three terms in our final version.\n\n[1] Hu, Yichun, Nathan Kallus, and Xiaojie Mao. \"Fast rates for contextual linear optimization.\" Management Science 68.6 (2022): 4236-4245.\n\n[2] Van der Vaart, Aad W. Asymptotic statistics. Vol. 3. Cambridge university press, 2000.\n\n[3] Shao, Jun. Mathematical statistics. Springer Science \\& Business Media, 2008.\n\n[4] Duchi, John C., and Feng Ruan. \"Asymptotic optimality in stochastic optimization.\" (2021): 21-48.\n\n[5] Elmachtoub, Adam N., Henry Lam, Haofeng Zhang and Yunfan Zhao. \"Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective.\" arXiv preprint arXiv:2304.06833 (2023).\n\n[6] Nocedal, Jorge, and Stephen J. Wright, eds. Numerical optimization. New York, NY: Springer New York, 1999.\n\n[7] Qi, Meng, Paul Grigas, and Zuo-Jun Max Shen. ``Integrated conditional estimation-optimization.'' arXiv preprint arXiv:2110.12351 (2021).\n\n[8] Elmachtoub, Adam N., Henry Lam, Haixiang Lan, and Haofeng Zhang. \"Dissecting the Impact of Model Misspecification in Data-driven Optimization.\" arXiv preprint arXiv:2503.00626 (2025)."}, {"review_id": "8iot0bYast", "reviewer": "Reviewer_jkSD", "summary": "This paper analyzes two different data-driven stochastic optimization approaches -- estimation-then-optimize (ETO) and integrated estimation-optimization (IEO) -- in their comparison with a popular paradigm called the sample average approximation (SAA). The formulations of ETO and IEO are standard and follow existing literature and the settings of discussion combines the assumptions by Elmachtoub et al. [2023] with a misspecification model referred to as \"local misspecification”. This model introduces three levels of misspecification: mild, balanced, and severe, which correspond to the presence of \"deviations\" from the true distribution at the rate of $o(n^{-2\\alpha})$ with $\\alpha>1/2$ (mild), $\\alpha=1/2$ (balanced) and $0<\\alpha<1/2$ (severe), respectively. These scenarios extend the discussion by Elmachtoub et al. [2023] in that the latter considers a well-specified case and a generic misspecified case. The paper claims to have explicated the asymptotic tradeoffs among ETO, IEO, and SAA in terms of their variance, bias, and regret. Discussions alike are of significance, as they can characterize the impact of misspecification on decision quality and inform the selection of data-driven approaches.\n\n\nRef: \nA. N. Elmachtoub, H. Lam, H. Zhang, and Y. Zhao. Estimate-then-optimize versus integratedestimationoptimization: A stochastic dominance perspective. arXiv preprint arXiv:2304.06833, 2023.  (Please note that there is a more recent version of this paper made available after NeurIPS submission deadline).", "questions": "Q1. The authors should carefully check whether the structural assumption invoked in Line 901 is actually necessary for the proof of Lemma 4.\n\nQ2. Some improvements in the wording of Table 1 are needed for accuracy. In particular, the description of the variance for ETO, IEO, and SAA in the severe misspecification case as “$\\approx 0$” (and therefore “negligible”) may be misleading. It appears that the variance does not vanish, but rather becomes dominated by other terms --- such as bias --- even though the variance may still be non-trivial. Rephrasing this to reflect the relative scale, rather than suggesting the negligibility of variance, would provide a more accurate interpretation.\n\nQ3. The paper assumes several technical conditions, including the invertibility of the Hessian matrix, twice continuous differentiability, and the full-rankness of $\\Sigma$. The paper can be improved by providing concrete examples from applications where such conditions are satisfied.\n\nQ4. There are multiple notational inconsistencies that need to be addressed. For instance, “$v(w,z)$” in Eq. (2) where $z$ is some parameters v.s. ``$v(w,Q_t)$’’ where $Q_t$ is some distribution in Line 903.\n\nQ5. The paper would benefit from adding a section (such as “Assumptions and Preliminaries”) that summarizes and clarifies key concepts. This section could be used to consolidate notations with subtle distinctions, explicitly state key assumptions used throughout the paper, and discuss useful known results and those that differ marginally from existing findings. \n\nQ6. Please update the information for all the references. E.g., A. N. Elmachtoub [2023]\n\nQ7. Line 1009: typo: “mulptiplication”.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strength:*\n\nS1. The consideration of a new misspecification model, which captures varying degrees of deviation from the true underlying distribution (mild, balanced, or severe), appears to be a potentially original and meaningful contribution. Since misspecification may diminish as decision-makers acquire more knowledge from data, this modeling framework captures a more realistic aspect of data-driven decision-making in practice.\n\nS2. The paper offers analytical comparisons of three important data-driven optimization schemes --- ETO, IEO, and SAA ---  in terms of three key performance metrics: bias, variance, and regret. The results can provide insights for selecting appropriate tools in different contexts of decision-making.\n\n\n*Weaknesses:*\n\nW1. On multiple occasions, the rigor of the analysis and proofs could be improved by adding explanations and justifications. Below are a few examples mostly in the proofs for Lemma 4 and 5, which serve as the bases for many  conclusions in the paper: \n\n   1.a. (Line 901) The proof of Lemma 4 invokes a structural assumption similar to that of Example 5 in the appendix, which may render the proof inapplicable to the statement of Lemma 4. (See Question Q1 below).\n\n   1.b. In line 904, how to obtain $\\frac{\\partial}{\\partial t} \\log q_t(\\boldsymbol z)\\vert_{t=0}=u(\\boldsymbol z)$ from Definition 1 requires additional exposition, although the conclusion in Line 904 seems correct.  \n\n   1.c. The proof of Lemma 4 begins by setting the gradient of the cost function at $w_t$ to be zero. However, this argument is not valid in the presence of constraints $\\Omega$.  \n\n   1.d.  In view of the assumption of an open feasible region $\\Omega$, the analysis seems to have overlooked the possibility that $w_t$ (as well as $w^{\\ast}$  and $w^{\\ast}_n$ ) may not exist.   \n\n   1.e. It is non-trivial to see (i) why $\\lim_{n\\rightarrow \\infty} \\sqrt{n} (w_n^*-w_{\\theta_0})=\\lim_{t\\rightarrow 0}\\frac{1}{t}(w_t-w_0)$ at the end of Line 906 and (ii) how it generalizes to Line 907, particularly in view of the fact that “t” is substantiated differently into $\\sqrt{n}$ v.s. $n$ in (i) and $n^{\\alpha}$ v.s. $n$ in (ii), respectively.  \n\n   1.f. Similar issues (1.a--1.c) from above also apply to Lemma 5.  \n\n   1.g. How Line 966—967 are obtained requires further exposition.\n\n\nW2. The paper can benefit from some careful and thorough editing to improve clarity. Below are a few examples:\n\n  2.a Line 256—257: The formulations of variance, bias, and regret are not explicitly justified in the main text. While their interpretations can be implicitly inferred from the statements of the theorems. Providing clearer definitions or more direct justification would improve readability.\n\n  2.b Critical assumptions (Assumptions 3 and 4) are not discussed until appendix.  \n\n  2.c. Related to 2.b, the justifications of some assumptions are provided within the proofs, rather than being stated upfront. (E.g., Line 969). For clarity, it would be helpful to motivate these assumptions earlier in the main text \n\n  2.d Line 897—898 are pillar results for the rest of the analysis. How they are inferred from Lemma 4 is non-trivial to see, despite that the same reasoning as 897–898 is repeated in the proof of Lemma 4.  \n\n  2.e. Notations with only minor variations are introduced throughout the paper to represent conceptually distinct meanings used thereafter. Sometimes these notations are introduced on the fly as the discussion progresses. (E.g., $w_n^*$ and $v_n$ before Line 250). A more deliberate and consistent presentation of notations would improve clarity. (See Question Q5).\n\n  2.f. The assumptions underlying Lemmata 2, 4, and 5 appear to differ, but they are not explicitly stated alongside the lemmas. It would be necessary to include the relevant assumptions directly in each lemma’s statement.\n  \n  2.g. Proposition 1 appears to be only marginally different from existing results in the literature and is focused on the ``well-specified'' case. In its current form, it would seem more appropriate to move this proposition to a section (See Question Q5) that reviews or introduces known results/concepts, rather than in the section titled “Mild Misspecification.” \n\n 2.h. The wording of Definition 1 should be revised, as it does not explicitly define the term “local perturbation.” Instead, it introduces an assumption in Lines 193–194 without clearly stating the definition. Making the concept and terminology of “local perturbation” explicit in the statement of that definition would improve clarity.\n \n  2.i. The use of the term “local misspecification” is not sufficiently justified. While the paper refers to Definition 1 as standard in asymptotic statistics, e.g., in [van der Vaart, 2000], this specific terminology does not appear to be introduced in that reference. Instead, the same mathematical formulation is presented in Chapter 7 of that reference in the context of local asymptotic normality (LAN). The use of LAN pertains to a somewhat different setting. A clearer exposition of the distinction between the adopted terminology and its original context as well as the justification for its current usage, would enhance rigor and clarity.\n\n\n\n*Ref:*\n\nW. van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.", "was_revised": true, "final_justification": "I would like to thank the authors again for their time and effort in carefully responding to my questions. While my initial main concern was that the paper might contain technical flaws in the proofs, the authors’ explanation clarified that the issues mostly stemmed from unintended oversight or lack of exposition. While my concerns regarding the clarity and rigor of the paper remain, I think many of them could be addressed according to the authors' rebuttal. Therefore, I consider reasons for acceptance to outweigh those for rejection in this case.", "rebuttal": "Thank you for the detailed review and the list of thoughtful comments. In the following, we address all your comments/questions, including those in the proof details in the appendix.\nRegarding many of your presentation concerns, in case of acceptance, we will use the additional page allowed for the camera ready to move more intuition and definitions to the main body of the paper. Specifically, we will add Assumption 3 and 4, the explanation on line 969-970 and the numerical experiments in the additional page.\nBelow, you can find our responses to each concern.\n\nW1.a: No, the structural assumption in line 901 (exponential tilt, essentially Example 5) is NOT necessary to prove Lemma 4. Definition 1 is more general and sufficient. We apologize for adding the exponential tilt assumption and making readers confused.  We will delete this redundant assumption in the revision. \n\nW1.b: The equality comes from the quadratic mean differentiability property in standard asymptotic statistics (Page 94, Section 7.2 of [1]). Specifically, Definition 1 indicates that $\\frac{1}{2}u(z)\\sqrt{p_{\\theta_0}(z)}$ is the derivative of the map $t\\to \\sqrt{q_t(z)}$ at $t=0$ (note that $q_0=p_{\\theta_0}$) for almost every $z$. In this case, $u(z)=2\\frac{1}{\\sqrt{q_0(z)}}\\frac{\\partial}{\\partial t}\\sqrt{q_t(z)}=\\frac{\\partial}{\\partial t}\\log q_t(z)$ at $t=0$.\n\nW1.c: In line 128, we assume that $\\Omega$ is an open set, thus every point in $\\Omega$ lies in the interior. Because of this, it is valid to set the gradient to $0$.\n\nW1.d: In line 866, we assumed the existence of minimizers ($w\\_t$, $w\\^{\\*}$, $w\\_n^\\*$) in the open feasible region in Assumption 3, which is a standard assumption in asymptotic statistics (assumptions in Theorem 5.7 and Theorem 5.23 of [1]). \n\nW1.e: In line 906, once we get the functional limit for $t$, the sequential limit for $n$ is an immediate result (since $1/n^\\alpha$ is a sequence that approaches to $0$ for all $\\alpha>0$) from the sequential criterion for functional limits (Theorem 4.2.3 of [Abbott, 2015]). Please note that the notation for $\\boldsymbol{w}_n^*$ and $Q_n$ already implicitly considers the order $\\alpha$ in line 249-250 and Definition 2.\n\nW1.f: Very similar explanations as the above hold for Lemma 5.\n\nW1.g: Line 966 is the consequence of the former analysis (962-966) on the middle term. Line 967 is the beginning of analyzing the first term based on using definitions of $O_p(\\cdot)$.\n\nW2.a: Thanks for your suggestions. We have provided some descriptions on bias and variance in Theorem 1 and provided the exact definition for regret in Definition 3. We will make it clearer in the final version.\n\nW2.b: We aim to demonstrate our own novel framework and results in the main body. We defer Assumptions 3 and 4 to Appendix (1) for the sake of page length; (2) since they are standard regularity assumptions in the existing asymptotic statistics literature.\nThat said, we will use the additional page for the camera ready to discuss these in the body of the paper.\n\nW2.c: Thanks for your suggestions. We have justified the validity of the assumption by citing previous literature in line 262. We will move further explanations (currently in line 969) here.\n\nW2.d: We hope our explanations for W1.e answers these questions.\n\nW2.e: Thanks for your suggestions. We will display the notations in a more concentrated manner in the final version.\n\nW2.f: Lemma 2 is sufficient to prove using Example 5 and Definition 2 (they are already mentioned in the statement). Lemmas 4 and 5 are both based on the same assumptions in Definition 1 (and interchangeability assumptions). The difference is the result rather than the assumptions, where Lemma 4 focuses on $\\boldsymbol{w}$ while Lemma 5 focuses on $\\theta$. The limit is taken by sending $t\\to 0$ thus also holds by taking $1/n^\\alpha\\to 0$ for any $\\alpha>0$ and $n\\to\\infty$  (and thus for all $1/n^\\alpha$ with $\\alpha>0$).  \n\nW2.g: We recognize that Proposition 1 is mostly an existing result from previous literature [3] (that is why we call it proposition instead of theorem). We aim to demonstrate that the existing result can be extended beyond the well-specified case to the mildly-misspecified case. We will add the reference in Proposition 1 and some clarifying text around it.\n\nW2.h: Definition 1 is to describe the relationship between the ground truth distribution and the parametric distribution (model class). Definition 1 is rather general and we have included many examples in the Appendix. We will make it more explicit by adding the description \"the distribution $Q_t$ is a \\textit{local perturbation} of the parametric family $\\{P_{\\boldsymbol{\\theta}}:\\boldsymbol{\\theta}\\in\\Theta\\}$ at $\\boldsymbol{\\theta}_0$ if it satisfies for all $t>0$, $q_t$ is differentiable and satisfies the following quadratic mean differentiablity condition (line 194)\" in the revision.\n\nW2.i: We have extensively discussed the term \"local misspecification\" in related works section (line 112 to 124) with many classical literature in statistics, econometrics and machine learning, e.g. [2]. Specifically, [2] uses the terminology \"local misspecification\" in Section 2. We have also technically explained the terminology in line 206-217.  We build a universal, general and rigorous local misspecification theoretical framework based on these literature and on the technical tools from [1], despite the fact that the original \"local asymptotic normality (LAN)\" is used for other settings. For these reasons, we believe our use of the term \"local misspecification\" and the relation to past literature are sufficiently clear, but we are happy to revise the exposition accordingly if the reviewer still sees the need.  \n\nQ1: Please see W1.a\n\nQ2: Thanks for your suggestions. The exposition in Table 1 is primarily to give readers a rough sense of our results, before loading them with many technical details. We have stated in the caption that $\\approx 0$ means ``asymptotically negligible\", and we have described in our main results much more accurately how it is in comparison to other terms and how the relative scale may depend on different regimes. Specifically, in the severe case, we build a convergence in probability result. In this case, the limit is a fixed vector (or, say, a degenerate distribution with zero variance). If you feel strongly, we can directly state the convergence result in the table, but it might hurt readability a bit.\n\nQ3: The technical assumptions including the invertibility of the Hessian/Fisher information matrix, twice continuous differentiability, and the full-rankness, mainly indicate the well-posedness of the problem and are often used in standard statistical literature (e.g. Theorems 4.6, 5.23, 5.39 in [1] have discussed the full-rankness, invertibility of Fisher information matrices and second-order Taylor expansion, etc.). We will add further discussions on the satisfaction of these assumptions in our application examples in Appendix A.1 (note that, on the other hand, we have provided many examples in Appendix A.2 regarding local perturbation, as this is the more interesting concept in this paper).\n\nQ4: The notation $v(\\boldsymbol{w},\\theta)$ (not $v(\\boldsymbol{w}, z)$ as you wrote) is a shorthand for $v(\\boldsymbol{w}, P\\_{\\theta})=\\mathbb{E}\\_{P_{\\theta}}[c(\\boldsymbol{w}, z)]$.  The notation $v(\\boldsymbol{w}, Q_t)$ stands for $\\mathbb{E}\\_{Q_{t}}[c(\\boldsymbol{w}, z)]$. They are consistent, but we will make it more clear in the revision. Specifically, we will stop using $v(\\boldsymbol{w},\\theta)$ and instead stick to $v(\\boldsymbol{w}, P_{\\theta})$.\n\nQ5: Please see W2.e\n\nQ6: Yes we will update all references, including [3] for which the reviewer has kindly noted there is an updated version.\n\nQ7: We will fix this typo in the revision.\n\n[1] Van der Vaart, Aad W. Asymptotic statistics. Vol. 3. Cambridge university press, 2000.\n\n[2] J. Fan, K. Imai, I. Lee, H. Liu, Y. Ning, and X. Yang. Optimal covariate balancing conditions in propensity score estimation. Journal of Business \\& Economic Statistics, 41(1):97–110, 2022\n\n[3] Elmachtoub, Adam N., Henry Lam, Haofeng Zhang and Yunfan Zhao. \"Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective.\" arXiv preprint arXiv:2304.06833 (2023)."}, {"review_id": "9QWeP7OQds", "reviewer": "Reviewer_bf4G", "summary": "This paper presents a novel theoretical framework to understand the bias-variance tradeoff in Data-Driven stochastic Optimization (DDO) under local model misspecification, an intermediate regime between the classical well-specified and globally misspecified settings. Specifically, the authors compare three popular methods: Sample Average Approximation (SAA), Estimate-Then-Optimize (ETO), and Integrated Estimation-Optimization (IEO), providing a unified asymptotic analysis that distinguishes mild, balanced, and severe misspecification regimes. The paper introduces an explicit characterization of decision bias and variance using influence functions and tools from contiguity theory. Importantly, it derives interpretable conditions under which model misspecification is approximately impactless, shedding light on when model-based methods can still perform well despite misspecification.", "questions": "1. The regret ordering results depend on specific expressions involving influence functions and score functions. How robust are these findings to changes in the parametric family or cost function structure? For example, would the same patterns hold under non-smooth costs or under heteroskedastic models?\n2. The conclusion mentions potential extensions to constrained or contextual optimization settings. Could the authors clarify what the main challenges would be in applying their local misspecification framework to those more complex scenarios? Would feasibility constraints fundamentally alter the bias-variance tradeoff?", "limitations": "While the theoretical analysis is thorough and well-executed, the paper unfortunately lacks empirical or numerical experiments to illustrate and validate the main findings—particularly the regret comparisons across different misspecification regimes—which limits the practical interpretability and accessibility of the results.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper fills an important gap in the literature by providing a nuanced analysis of the interplay between model misspecification and statistical variability in DDO. This is a first-of-its-kind contribution in this context.\n2. The division of results into mild, balanced, and severe misspecification regimes is both intuitive and insightful. Table 1 effectively summarizes the comparative behavior across these regimes.\n3. The derivation of explicit bias expressions and the concept of \"approximately impactless misspecification direction\" provide valuable intuition and theoretical tools that could inform future method design.\n\nWeaknesses\n\n1. The paper is purely theoretical. While this is not a major flaw, a small numerical experiment (even a synthetic one) illustrating the main theoretical phenomena would be helpful and appreciated by a broader audience.\n2. Some parts of the paper, especially around the statistical theory (e.g., contiguity, local LAN models), may be challenging for readers unfamiliar with advanced asymptotics. Additional intuition or diagrams (beyond Figure 1) could aid accessibility.", "was_revised": true, "final_justification": "This paper presents a novel theoretical framework to understand the bias-variance trade-off in data-driven stochastic optimization. During the author-reviewer discussion, the authors provided detailed response, which addressed my concerns. So I will maintain my score that is inclined to acceptance.", "rebuttal": "Thank you for recognizing our significance and strengths, and for the thoughtful comments. We address your comments and questions as follows:\n\nW1: We understand your comment and have run new experiments. Please refer to our added experiment part in the response to reviewer JBev. \n\nW2: We have already introduced an important lemma, LeCam's third lemma in the appendix. We will add more explanations and references for the contiguity theory, if needed in the appendix. Specifically, we would start from the definition of contiguity for two sequences of measures $P_n$ and $Q_n$. Let $\\{P_n\\}$ and $\\{Q_n\\}$ be sequences of probability measures on a common measurable space. We say that $\\{Q_n\\}$ is contiguous with respect to $\\{P_n\\}$, written as $Q_n \\triangleleft P_n$, if for every sequence of measurable events $A_n$, $P_n(A_n) \\to 0 \\quad \\Rightarrow \\quad Q_n(A_n) \\to 0$. If both $Q_n \\triangleleft P_n$ and $P_n \\triangleleft Q_n$, then we say the sequences are mutually contiguous and write $P_n \\sim Q_n$. We will introduce the criterion for contiguity (Le Cam's first lemma): Let $\\{P_n\\}$ and $\\{Q_n\\}$ be sequences of probability measures on $(\\mathcal{X}_n, \\mathcal{A}_n)$ such that $Q_n \\ll P_n$ for all $n$, and define the likelihood ratio $Z_n = \\frac{dQ_n}{dP_n}$. Then the following are equivalent: \n(1) $Q_n \\triangleleft P_n$, i.e., $Q_n$ is contiguous with respect to $P_n$.\n(2) If along some subsequence, $\\frac{dP_n}{dQ_n} \\to U$ in distribution under $Q_n$, then $\\mathbb{P}(U > 0) = 1$.\n(3) If along some subsequence, $\\frac{dQ_n}{dP_n} \\to V$ in distribution under $P_n$, then $\\mathbb{E}\\_P[V] = 1$.\n(4) For any sequence of statistics $T_n$, if $T_n \\to 0$ in $P_n$-probability, then $T_n \\to 0$ in $Q_n$-probability.\nWe will introduce asymptotic log normality for two mutually contiguous $P_n$ and $Q_n$: $\\log \\frac{dP_n}{dQ_n} \\xrightarrow{d} e^Z, \\quad \\text{under } Q_n, \\quad \\text{where } Z \\sim \\mathcal{N}(\\mu, \\sigma^2)$, then we have: (1) $Q_n \\triangleleft P_n$, and (2) $P_n \\triangleleft Q_n$ if and only if $\\mu = -\\frac{1}{2} \\sigma^2$. We will review quadratic mean differentaiblity and local alternatives, definitions and properties of local asymptotic normality (LAN) in further details.\n\nQ1: Let us respond to four aspects of your question:\n\n1) We emphasize that in Theorems 1-6, while the bias and variance terms depend on specific expressions involving influence functions and score functions, the orderings of the bias and variance are universal among all cost function structures (influence functions) and parametric families (score functions). The regret ordering in the mildly and severely regimes are also universal among all influence functions and score functions. The only undetermined regime is the balanced regime, where the orderings of bias and variance are opposite, thus making the regret (the sum of bias and variance) ordering undetermined. However, we empirically demonstrate the regret ordering in the added numerical experiments and find that IEO may achieve the best bias-variance tradeoff and the lowest regret among all three methods in the newsvendor problem.\n\n2) We only assume the smoothness of the cost function \\textit{in expectation}, instead of the cost function itself. The former assumption is much weaker and general than the latter. That is, the cost function itself can be possibly non-smooth in our framework. For example, the cost function of the newsvendor problem has kinks and is not smooth. Nonetheless, if the distribution is continuous, then the expected cost will be smooth and satisfy our framework. \n\n3) If by \"heteroskedastic\" the reviewer means that the cost function taken in the expectation can have different noise depending on the decision, then yes our framework allows that. In fact, we don't even make an explicit distinction between \"heteroskedastic\" and \"homoskedastic\" in the paper.\n\n4) Our framework focuses on nonlinear objective function, and does not apply to linear programming settings per se. However, we should point out that the latter setting is easier than our nonlinear setting from a statistical perspective. This is because if the cost function is linear, then it suffices to predict the expectation of the cost vector, which is a point prediction [1]. Because of this, fitting the whole distribution like we do is not needed. In contrast, our setting require the notion of model misspecification that is at the distributional level, since the nonlinearity of the objective function forces a more complex relation between the randomness and the decision variables.\n\nQ2: Our framework can be extended to other settings. For contextual optimization, the extension is quite immediate. Specifically, we introduce the feature $\\boldsymbol{x}$ in addition to the random parameter $\\boldsymbol{z}$. We write the joint distribution of $\\boldsymbol{x}$ and $\\boldsymbol{z}$ as $Q(\\boldsymbol{x},\\boldsymbol{z})=Q(\\boldsymbol{x})Q(\\boldsymbol{z}|\\boldsymbol{x})$. In this case we need to estimate $Q(\\boldsymbol{z}|\\boldsymbol{x})$ using parametric family $Q_{\\boldsymbol{\\theta}}(\\boldsymbol{z}|\\boldsymbol{x})$. Note that in the contextual case, the SAA method is not interesting because it  allows to choose any map from context to decision, thus easily overfitting the\nfinite-sample problem. In this case, IEO and ETO are the legitimate approaches. In the mildly misspecified case, ETO is better than IEO while in the balanced and misspecified case, IEO is better than ETO.\nFor constrained optimization, if the constraints are deterministic, then we expect our analysis and insights to continue to hold, but it would introduce additional technical complexity from the KKT conditions instead of using simpler optimality conditions. On the other hand, if the constraints are stochastic, then there is an additional question of feasibility (e.g., in chance constrained settings where random safety conditions need to be enforced with high probability). These extensions to constrained settings warrant future work in our view.\n\n[1] Elmachtoub, Adam N., Henry Lam, Haofeng Zhang and Yunfan Zhao. \"Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective.\" arXiv preprint arXiv:2304.06833 (2023)."}, {"review_id": "pxaTdU3wed", "reviewer": "Reviewer_JBev", "summary": "Prior work focused on either well-specified models or fully misspecified models, no theoretical framework to analyze the intermediate and realistic case where the model is locally misspecified. Besides, SAA, ETO, and IEO are widely used, but their performance differences under slight misspecification were not well understood. \n\nThe key challenge is to develop a unified theoretical framework for analyzing and comparing data-driven optimization methods under local model misspecification, and proved insights into when each method should be applied. \n\nThis paper studies the statistical performance of SAA, ETO, and IEO—under local model misspecification. The authors analyze how bias and variance trade off across different misspecification regimes (mild, balanced, severe). Key contributions include: a unified asymptotic framework capturing these regimes, explicit expressions for decision bias, and sufficient conditions under which misspecification has negligible impact. The results provide guidance on method selection depending on the degree and direction of model misspecification.", "questions": "See my questions in the above “Weakness” part and below: \n\n- Could you provide a discussion on how you expect these asymptotic results to manifest in finite-sample settings?\n- It would be great if the author may consider adding a small-scale numerical experiment, even on a simple, synthetic problem.", "limitations": "See the limitations also in the above “Weakness” and “Questions” part.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The analysis relies on classical smoothness and identifiability assumptions (e.g., differentiability of cost functions, uniqueness of minimizers, invertibility of Hessians, etc.), which seem natural and reasonable to me.\n\n**Strengths:**\n\n- This paper presents a technically solid analysis of data-driven stochastic optimization under local model misspecification.\n- It provides asymptotic results for SAA, ETO, IEO under varying degree of model misspecification.\n- They derive explicit expressions for decision bias and variance, offering insights into when and why model-based methods might fail or succeed. \n- The main contribution is a local misspecification framework, which allows for a better understanding of how these methods perform when the underlying statistical model is \"almost\" correct.\n- The paper appears novel both in problem framing and in technical methodology.\n\n\n**Weakness:** \n\n- Lack of empirical finding of finite sample behavior: The paper’s main results (Theorem 1, 3, and 6) describe the limiting behavior of the estimators as the $n \\rightarrow \\infty$. In practice, devisions are always made with a finite amount of data. This paper lack of evidence to show how well this asymptotic theory approximates the actual performance of SAA, ETO, and IEO for realistic finite sample sizes. \n- It does not provide insight into the absolute magnitudes of bias vs variance. For a given problem, is the bias from a \"mild\" misspecification truly negligible compared to the variance? In the \"balanced\" regime, is the bias-variance tradeoff a dramatic one, or are the differences between the methods marginal? \n- Theorem 5: misspecification in the direction of the model’s score function leads to zero asymptotic bias. This is a powerful results, but again its practical utility is not tested, the lack of empirical validation makes the work feel incomplete. It provides a map of the asymptotic world but offers no guidance on how that map corresponds to the finite-sample territories where practitioners actually operate.", "was_revised": true, "final_justification": "My main concern was satisfactorily addressed. I remain my overall assessment.", "rebuttal": "We sincerely appreciate your detailed review and thoughtful comments on our paper. \n\nW1: We completely understand and have added experiments to validate our theoretical results. We validate our findings by conducting numerical experiments on the newsvendor problem, a classic example in operations research with non-linear cost objectives. We show and compare the performances of the three data-driven methods under different local misspecification settings, including different directions and degrees of misspecification, to validate our theoretical analysis as well as providing additional insights about the effect of model parameters.\n\nThe newsvendor problem has the objective function $c(w,z)={a}^\\top(w-z)^++d^\\top(z-w)^+$, where for each $j\\in[d_z]$: (1) $z^{(j)}$ is the customers' random demand of product $j$; (2) $w^{(j)}$ is the decision variable, the ordering quantity for product $j$; (3) $a^{(j)}$  is the holding cost for product $j$; (4) $d^{(j)}$ is the backlogging cost for product $j$. We assume the random demand for each product are independent. We assume the holding cost and backlogging cost is uniform among all products by setting $a^{(j)}=5$ and $d^{(j)}=1$ for all $j\\in[d_z]$.\n\nWe describe the local misspecified setting by using the framework of Example 5 and building a model and generating a random demand dataset as follows. We denote the training dataset as $\\{z_i^{(j)}\\}_{i=1}^n$, where $n$ is the training sample size. The model assumes that the demand for each product $j\\in[d_z]$ is normally distributed with the distribution $N(j\\theta,1)$ where $\\theta$ is unknown and needs to be learned. We first describe the well-specified setting, where the demand distribution for product $j$ is $N(3j,1)$. To describe the local misspecification, we need to specify the direction and degree of misspecification. We set (1) $\\alpha=0.1$ for the severely misspecified setting, (2) $\\alpha=0.5$ for the balanced setting and $\\alpha=2$ for the mildly misspecified setting.\n\nWe discuss two types of directions: (1) $u(z)=\\sum_{j=1}^{d_z} (z^{(j)})^2$; (2)  ${u}(z)=\\sum_{j=1}^{d_z} j(z^{(j)}-3j)$.\n\nWe show experimental results in Tables 1-3 to support our theoretical results in Section 3, using the mean of the regret for three methods, under three directions and three misspecified regimes. When ${u}(z)=\\sum_{j=1}^{d_z} (z^{(j)})^2$, in the mildly specified case, ETO has a lower regret than IEO, and IEO has a lower regret than SAA. However, in the severely misspecified regime, the ordering of the three methods flips. This is consistent with our theoretical comparison results in Theorems 3, 4 and 6. In the balanced regime, experimental results show that IEO has the lowest regret among the three methods. This is also consistent with the theoretical insight in Theorem 2 that IEO has the advantage of more readily achieving the bias-variance trade-off in terms of the decisions and corresponding regrets.  It also gives us some practical guidelines in choosing among the three methods: if the misspecification error is larger than the sampling error, then SAA should be used; if the model is almost well-specified, then ETO should be used; if the misspecification error and the sampling error are of the same scale, or are generally unknown, then IEO is usually a robust choice.\n\nW2: We note that (and we believe this is precisely what the reviewer thinks as well), as in other asymptotic analyses, our \"negligibility\" is theoretically framed by comparing the orders of different sequences. Consequently, the theory by itself does not reveal the absolute magnitudes of the various quantities; instead it serves primarily to provide comparative insights. While this is a typical limitation of any asymptotic study, through our new experiments we can get a sense of the absolute magnitudes, as shown in Tables 5-7, when $u(z)=\\sum_{j=1}^{d_z} (z^{(j)})^2$ under three different misspecified regimes. We demonstrate the bias and variance terms as scalars by taking the norm of sample mean and sample covariance matrix of $n^{-\\min\\{1/2,\\alpha\\}}(\\hat{w}^\\square-w^*_n)$.  In the mildly misspecified case, the bias term is dominated by the variance term; in the severely misspecified regime, the dominance is flipped; in the balanced case, no significant dominance exists. Moreover, the results are consistent with the bias and variance comparison results in Theorems 1, 3, 4, 6. \n\nW3: In our new experiments, we include the setting when ${u}(z)=\\sum_{j=1}^{d_z} j(z^{(j)}-3j)$, an impactless direction discussed in Theorem 5. In this case, in Table 4 the ordering of the three methods is different from previous scenarios (Table 1-3). This is because the direction of misspecification ${u}(z)$ in this case coincides with the score function itself. Note that the score function at $\\theta\\_0=3$ is $s_{\\theta_0}(z)=\\nabla\\_{\\theta}\\log p\\_{\\theta\\_0}(z)=\\nabla\\_{\\theta}\\log\\exp(-\\sum\\_{j=1}^{d_z} (z^{(j)}-3j)^2/2)=\\sum\\_{j=1}^{d\\_z} j(z^{(j)}-3j)$. Even in the severely misspecified setting, all three methods have asymptotically $o(n^{-\\alpha})$ decision bias and corresponding $o(n^{-\\alpha})$ regret, i.e., $b^\\textup{SAA}=b^{\\textup{IEO}}=b^\\textup{ETO}=0$ and $R^\\textup{SAA}=R^\\textup{IEO}=R^\\textup{ETO}=0$. Intuitively, through this particular direction, regardless of the degree of misspecification, the ordering of the performance of the three methods is consistent with the mildly misspecified setting, where ETO is better than IEO and IEO is better than SAA. The experiments validate our theory and show our theory is robust. Not only the theory holds in the asymptotic regime but also we have seen the same finite-sample behaviours and trends for a wide range of sample sizes.\n\nQ1 and Q2: As described in our responses to your comments above, we have provided and discussed new experimental results that aim to give a sense of the finite-sample manifestation. \n\nWe also want to point out that, in terms of building additional theory that sheds light on finite-sample behaviors, we can likely derive corresponding bounds, but it is unclear that this would reveal better insights than our asymptotic analysis. Please see our response to W2 for Reviewer JYdJ regarding the comparison and motivation of our work with [1] that focuses on finite-sample bounds.\n\n[1] Elmachtoub, Adam N., Henry Lam, Haixiang Lan, and Haofeng Zhang. \"Dissecting the Impact of Model Misspecification in Data-driven Optimization.\" arXiv preprint arXiv:2503.00626 (2025).\n\nNote: the caption stands for the table above.\n\n|n|SAA|IEO|ETO|\n|--|---|---|---|\n|20|7.92e-2|8.74e-3|3.85e-3|\n|40|2.75e-2|4.11e-3|2.13e-3|\n|60|2.53e-2|2.09e-3|1.75e-3|\n|80|1.80e-2|1.74e-3|1.25e-3|\n|100|1.62e-2|2.96e-3|1.49e-3|\n|120|1.45e-2|9.49e-4|4.05e-4|\n|140|9.45e-3|9.17e-4|3.44e-4|\n|160|1.25e-2|1.68e-3|6.49e-4|\n|180|8.50e-3|1.69e-3|1.91e-4|\n|200|8.66e-3|8.12e-4|3.71e-4|\n\nTable1: mild case, first direction\n\n|n|SAA|IEO|ETO|\n|--|---|---|---|\n|20|4.85e-2|1.37e-2|3.30e-2|\n|40|3.57e-2|7.36e-3|1.29e-2|\n|60|2.54e-2|5.51e-3|8.12e-3|\n|80|1.90e-2|3.14e-3|5.84e-3|\n|100|1.22e-2|2.21e-3|6.23e-3|\n|120|1.53e-2|1.63e-3|3.86e-3|\n|140|9.51e-3|1.82e-3|5.26e-3|\n|160|9.45e-3|2.47e-3|3.25e-3|\n|180|6.44e-3|1.70e-3|4.25e-3|\n|200|4.64e-3|1.14e-3|2.66e-3|\n\nTable 2: balanced case, first direction\n\n|n|SAA|IEO|ETO|\n|--|---|---|---|\n|20|4.64e-2|2.76e-2|1.18e-1|\n|40|2.19e-2|2.35e-2|1.03e-1|\n|60|1.49e-2|2.60e-2|1.05e-1|\n|80|8.80e-3|2.16e-2|9.99e-2|\n|100|1.05e-2|2.04e-2|9.86e-2|\n|120|9.34e-3|2.03e-2|1.01e-1|\n|140|5.80e-3|2.02e-2|9.69e-2|\n|160|5.51e-3|2.03e-2|9.50e-2|\n|180|5.46e-3|2.00e-2|9.23e-2|\n|200|5.35e-3|2.02e-2|8.91e-2|\n\nTable 3: severe case, first direction\n\n|n|SAA|IEO|ETO|\n|--|---|---|---|\n|20|7.39e-2|4.94e-3|6.25e-3|\n|40|4.16e-2|8.01e-3|5.45e-3|\n|60|2.76e-2|2.86e-3|1.12e-3|\n|80|1.93e-2|2.58e-3|7.16e-4|\n|100|1.38e-2|1.96e-3|7.24e-4|\n|120|1.64e-2|1.59e-3|1.26e-3|\n|140|1.09e-2|4.95e-4|4.77e-4|\n|160|7.81e-3|1.36e-3|1.02e-3|\n|180|9.29e-3|1.23e-3|5.04e-4|\n|200|6.89e-3|1.09e-3|2.12e-4|\n\nTable 4: severe case, second direction\n\n|n|SAA_b|SAA_v|ETO_b|ETO_v|IEO_b|IEO_v|\n|--|-----|-----|-----|-----|-----|-----|\n|20|4.41e-1|2.63e+0|4.86e-2|4.79e-1|3.44e-2|1.41e+0|\n|40|1.26e-1|2.71e+0|1.84e-1|8.30e-1|1.60e-1|1.48e+0|\n|60|1.50e-1|4.13e+0|5.95e-2|6.88e-1|3.13e-1|1.90e+0|\n|80|1.11e-1|3.30e+0|4.56e-1|1.17e+0|1.99e-1|2.18e+0|\n|100|4.94e-1|3.83e+0|5.07e-2|9.36e-1|3.53e-1|2.05e+0|\n|120|6.96e-1|2.64e+0|1.56e-2|6.68e-1|8.12e-2|1.33e+0|\n|140|3.42e-1|2.34e+0|7.10e-2|7.11e-1|2.38e-1|1.37e+0|\n|160|3.87e-1|3.79e+0|1.33e-2|8.47e-1|1.55e-1|1.72e+0|\n|180|2.38e-1|3.03e+0|1.07e-1|1.03e+0|1.90e-2|1.84e+0|\n|200|4.96e-1|3.99e+0|5.06e-2|8.29e-1|2.73e-2|2.43e+0|\n\nTable 5: mild case (b: bias; v: variance)\n\n|n|SAA_b|SAA_v|ETO_b|ETO_v|IEO_b|IEO_v|\n|--|-----|-----|-----|-----|-----|-----|\n|20|1.22e-1|2.10e+0|1.08e+0|6.37e-1|3.27e-1|1.60e+0|\n|40|5.92e-1|1.85e+0|1.41e+0|6.38e-1|3.75e-1|1.06e+0|\n|60|5.63e-1|2.65e+0|1.28e+0|4.21e-1|4.40e-1|1.71e+0|\n|80|4.66e-1|2.80e+0|9.79e-1|6.69e-1|4.00e-1|1.47e+0|\n|100|1.52e-1|2.62e+0|1.17e+0|1.04e+0|3.87e-1|1.93e+0|\n|120|2.29e-1|2.18e+0|1.09e+0|7.57e-1|3.84e-1|1.60e+0|\n|140|1.72e-1|2.97e+0|1.15e+0|9.11e-1|4.25e-1|1.80e+0|\n|160|2.96e-1|2.37e+0|1.41e+0|4.92e-1|4.17e-1|1.66e+0|\n|180|4.10e-1|2.82e+0|1.38e+0|8.91e-1|4.28e-1|1.81e+0|\n|200|3.41e-1|2.42e+0|1.24e+0|4.45e-1|4.51e-1|1.19e+0|\n\nTable 6: balanced case\n\n|n|SAA_b|SAA_v|ETO_b|ETO_v|IEO_b|IEO_v|\n|--|-----|-----|-----|-----|-----|-----|\n|20|8.84e-2|1.08e-1|6.60e-1|3.56e-2|2.15e-1|9.07e-2|\n|40|3.40e-2|6.37e-2|7.18e-1|1.77e-2|2.21e-1|3.27e-2|\n|60|5.77e-2|6.13e-2|7.13e-1|1.43e-2|2.28e-1|2.66e-2|\n|80|4.82e-2|4.11e-2|7.25e-1|1.17e-2|2.28e-1|2.32e-2|\n|100|1.87e-2|3.52e-2|7.50e-1|6.70e-3|2.31e-1|1.27e-2|\n|120|5.88e-2|3.25e-2|7.25e-1|9.10e-3|2.32e-1|1.49e-2|\n|140|1.45e-2|2.96e-2|7.56e-1|9.10e-3|2.35e-1|1.79e-2|\n|160|3.33e-2|2.06e-2|7.54e-1|5.40e-3|2.35e-1|1.22e-2|\n|180|2.12e-2|2.72e-2|7.40e-1|6.60e-3|2.36e-1|1.55e-2|\n|200|2.25e-2|1.92e-2|7.63e-1|4.45e-3|2.39e-1|9.57e-3|\n\nTable 7: severe case"}, {"review_id": "EXnYSRXFen", "reviewer": "Reviewer_tsVS", "summary": "This paper presents a theoretical investigation into the relative performance of three widely used approaches in data-driven stochastic optimization: Sample Average Approximation (SAA), Estimate-Then-Optimize (ETO), and Integrated Estimation-Optimization (IEO). The authors rigorously analyze the bias-variance tradeoff associated with these methods under conditions of local model misspecification, a realistic setting in which the true data-generating process deviates slightly from the assumed model.\n\nBy employing tools from contiguity theory in statistics, the paper distinguishes itself with a mathematically elegant framework. The analysis reveals that the relative effectiveness of SAA, ETO, and IEO is context-dependent. Specifically, when the degree of local misspecification increases, the tradeoff between variance (due to data noise) and bias (introduced by model misspecification) becomes a critical factor. This highlights that no single method dominates across all regimes; rather, the optimal choice depends on how well the model aligns with reality.\n\nWhile the theoretical result is interesting, it is not clear how the method can be used and applied in practice because there is no experiment, even on synthetic data. I gave the Borderline accept with less confidence.", "questions": "This paper offers interesting theoretical insights that could inspire future empirical or methodological work, but it lacks the experimental evidence necessary to demonstrate relevance or applicability in practice. Adding even a basic synthetic experiment or discussing potential practical guidelines would substantially improve the paper's value to the community.", "limitations": "See the weaknesses", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* Novelty and Theoretical Depth: The application of contiguity theory to analyze local misspecification and derive asymptotic comparisons is intellectually rigorous and adds depth to the existing literature on decision-making under uncertainty.\n\n* Clear Exposition: The paper is well-structured and the theoretical arguments are laid out in a clear, readable fashion. The definitions and assumptions are reasonable and standard in the field, and the logic flows smoothly.\n\n* Conceptual Insight: The identification of distinct sources of bias and variance—stemming respectively from model mismatch and sample randomness—sheds light on why optimization performance varies across methods and provides a fresh lens to view classical problems.\n\nWeaknesses:\n* Lack of Empirical Evaluation: A major shortcoming of the paper is the absence of any empirical validation. While the theoretical results are compelling, it remains unclear how significant the observed effects are in practical settings. Even a simple synthetic example would help to visualize the tradeoffs and enhance the paper’s impact.\n\n* Practical Applicability is Unclear: In real-world scenarios, practitioners often do not know whether or how their models are misspecified. Estimating the degree of local misspecification or the noise level in the data is non-trivial, which complicates the practical utility of the results. The paper stops short of offering actionable guidelines on how a practitioner might choose between SAA, ETO, or IEO based on observable properties of their problem.\n\n* No Discussion on Robustness or Extensions: The paper could be strengthened by discussing how robust the results are to broader forms of misspecification or whether the framework can be extended to incorporate distributional robustness, Bayesian perspectives, or online decision-making settings.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your detailed review and thoughtful comments on our paper.\n\nW1: We completely understand your concern, and have run and discussed new experimental results. Please refer to our response to Reviewer JBev for details.\n\nW2: We very much understand the reviewer's point on practical applicability. As in many other theoretical studies, the goal of our paper is to provide insights, in our case the bias-variance dissection and directional impacts of model misspecification, and also to bring in the new theoretical tools that allow us to derive the results and conclude these insights. There is certainly an important question on practical implementation. To this end, in our mind our work is not a one-time study in this problem domain, instead there will be many follow-up works on other aspects including practical and computational issues.\n\nThus, we are careful not to over-state our contributions (both in the paper and in this response). That being said, our theory does give some clear insights. First, in terms of choosing among the three methods: if the misspecification error is larger than the sampling error, then SAA should be used; if the model is almost well-specified, then ETO should be used; if the misspecification error and the sampling error are of the same scale, or are generally unknown, then IEO is usually a robust choice. Second, our main results reveal that the orderings of the bias and variance of the three methods are universal regardless of (almost) all directions $u(\\boldsymbol{z})$, but only depend on the degree of misspecification. That is, the deviating directions $u$ of $Q$ turn out to be an invariant factor under most scenarios.\n\nW3: We will add discussions on the extensions of our framework to other settings in the paper. These extensions, including contextual optimization, constrained optimization and the incorporation of robustness, range from immediately doable to the need of a potentially substantial expansion of our framework: For contextual optimization, the extension is quite immediate. Specifically, we introduce the feature $\\boldsymbol{x}$ in additional to the random parameter $\\boldsymbol{z}$. We write the joint distribution of $\\boldsymbol{x}$ and $\\boldsymbol{z}$ as $Q(\\boldsymbol{x},\\boldsymbol{z})=Q(\\boldsymbol{x})Q(\\boldsymbol{z}|\\boldsymbol{x})$. In this case we need to estimate $Q(\\boldsymbol{z}|\\boldsymbol{x})$ using parametric family $Q_{\\boldsymbol{\\theta}}(\\boldsymbol{z}|\\boldsymbol{x})$. Note that in the contextual case, the SAA method is not interesting because it  allows to choose any map from context to decision, thus easily overfitting the\nfinite-sample problem. In this case, IEO and ETO are the more legitimate approaches. In the mildly misspecified case, ETO is better than IEO while in the balanced and misspecified case, IEO is better than ETO.\nFor constrained optimization, if the constraints are deterministic, then we expect our analysis and insights to continue to hold, but it would introduce additional technical complexity from the KKT conditions instead of using simpler optimality conditions. On the other hand, if the constraints are stochastic, then there is an additional question of feasibility (e.g., in chance constrained settings where random safety conditions need to be enforced with high probability). These extensions to constrained settings warrant future work in our view. Lastly, to incorporate robustness, a natural consideration is the possibility of distributional shifts. In this setup, we would need to consider the worst-case regret or other decision-theoretic counterparts, and the involved dissection is not only confined to statistical bias and variance but also deterioration of the decision quality coming from the fundamental shifts. This extension is substantial and would be worth another future work.\n\nQ1: As described in our response to W1, we completely understand your concern, and refer you to our new experimental results and discussions in our response to Reviewer JBev."}]

Author rebuttal:

Meta-review: This paper provides a new analysis studying the impact of model misspecification on the asymptotic performance of sample average approximation, estimate-then-optimize, and integrated estimate-then-optimize.

Pros:
- This paper provides a novel and significant view on model misspecification in an important broader context
- The analysis provides interpretable insights that are likely to be useful to practitioners 

Weakness: 
- While the paper is a theory paper, the empirical evaluation could be stronger. The authors added an empirical evaluation focused on finite-time performance during the review process but the paper would be stronger if there were additional investigations.

I'm recommending acceptance because of the novelty, significance, and usefulness of the theory

There was a robust conversation between the reviewers and authors that will improve the paper's clarity, level of rigor, and the strength of its empirical evaluation. Reviewer comments focused on clarifying a number of technical points in the paper and on the lack of empirical evaluation.
Decision: Accept (poster)