Prescribe-then-Select: Adaptive Policy Selection for Contextual Stochastic Optimization
Authors: Caio de Próspero Iglesias
Kimberly Villalobos Carballo
Dimitris Bertsimas
Venue: Accepted by TMLR ()

Abstract: We address the problem of policy selection in contextual stochastic optimization (CSO), where covariates are available as contextual information and decisions must satisfy hard feasibility constraints. In many CSO settings, multiple candidate policies—arising from different modeling paradigms—exhibit heterogeneous performance across the covariate space, with no single policy uniformly dominating. We propose Prescribe-then-Select (PS), a modular framework that first constructs a library of feasible candidate policies and then learns a meta-policy to select the best policy for the observed covariates. We implement the meta-policy using ensembles of Optimal Policy Trees trained via cross-validation on the training set, making policy choice entirely data-driven. Across two benchmark CSO problems—single-stage newsvendor and two-stage shipment planning—PS consistently outperforms the best single policy in heterogeneous regimes of the covariate space and converges to the dominant policy when such heterogeneity is absent. All the code to reproduce the results can be found at https://anonymous.4open.science/r/Prescribe-then-Select-TMLR.

Published in Transactions on Machine Learning Research (01/2026)
Prescribe-then-Selet:: Adaptive Policy Selection for Con textual Stochastic Optimization
Caio de Próspero Iglesias
caiopigl@mitt.ck edu
Sloon School of Management
Massachusetls Institute of Technology
Kimberly Villalobos Carballo
Nimberly.o@ryye edu
Tandon School of Engineering
New York University
Dimitris Bertsimas
dbertsim edu
Sloon School of Management
Massachusetts Institute of Technology
Reviewed on OpenReview: https:// oww.reveee.......... net/ forum? id- 1FEsAF2170
Abstract
We address the problem of policy selection in contextual stochastic optimization (CSO), where covariates are available as contextual information and decisions must satisfy hard feasibility constraints. In many CSO settings, multiple candidate policies-rrssing from dif- ferent modeling paradigms-exhilee heterogencous performance across the covariate space, with no single policy uniformly dominating. We propose Prescribe--thnn-Sleed (PS), modular framework that first constructs library of feasible candidate policies and then learns meta-policy to select the best policy for the observed covariates. We implement the meta-policy using ensembles of Optimal Policy Trees trained via cross-validation the training set, making policy choice entirely data-driven. Across two benchmark CSO problems- single-stage newsvendor and two-stage shipment planning PS consistently out- performs the best single policy in heterogencous regimes of the covariate space and converges to the dominant policy when such heterogeneily absent. All the code to reproduce the results can be found htttp////ttaabb..oo//ggeasaaaaas
1 Introduction
Optimization under uncertainty ariscs in numerous applications, including supply chain management, ergy markets, and financial planning, and remains important research area in the optimization commu- nity BBirge & Louveaux 2011: Shapiro et al. 2021 Conejo et al. 2010 Snyder 2006 Snyder & Shen 2019)
In these problems, decisions are made under uncertainty about future outcomes, with the objective of min- imizing total cost. wide variety of data-driven methods have been developed to address such problems,
including sample average approximation (Shapiro 2003 Shapiro & Nemirovski 2005 Kleywegt et al. 2002)
stochastic approximation algorithms (Robbins & Monro 1951
Nemirovski al.] 2009), robust optimiza- Bertsimas et al. 2018a and distributionally
tion techniques (Bertsimas el al.] 2018b; Ben-Tal Tal al. 2009
robust optimization formulations (Delage & Ye! 2010 Calafiore & Ghaoui 2006). These approaches differ in how they model and mitigate uncertainty: some aim to minimize expected costs (e.g,, sample average or stochastic approximation methods), while others focus on guarding against worst-case outcomes (as in robust optimization) or balance the two by minimizing worst-case expected cost. under a range of plausible distributions (as in distributionally robust optimization).
In many real-world settings, however, the decision-maker also observes auxiliary lata-known as covariates, context, side information- prior to making a decision. For example, in transportation, real-time weather

---

Published in Transactions on Machine Learning Research (01/2026)
and traffic conditions can help reduce uncertainty in travel times; in retail, promotions and seasonal sig- nals can inform demand; and in healthcare, patients' comorbidities and clinical histories provide valuable information about health outcomes. When such covariates are available, they can help infer relevant charac- teristics of the underlying uncertainty, narrowing the set of plausible scenarios and enabling more informed decisions. This observation has led to the development of contextual stochastic optimization (Sadana el al. 2025 where the goal is to learn a policy that maps covariates to decisions.
A diverse set of methodologies has emerged for solving contextual stochastic optimization problems. One
approach is to fit parametric decision nule-suchhe
linear function (Ban & Rudin 2019 Bertsimas &
Kallus 2020), kernel-basod model (Bazier-Matte & Delage 2020 Ban & Rudin 2019 Bertsimas & Koduri
2022 Notz & Pibernik 2022 Bertsimas & Carballo 2023 or neural network (Orooj et al. 2020:
Huber et al. 2019 Zhang& & 2 by minimizing empirical cost (potentially regularized) over historical data. These methods are computationally efficient at deployment since no optimization is needed once the policy is trained. However, their performance is often sensitive to the choice of function class and may suffer if the true policy is poorly approximated within the chosen function space. Importantly, these approaches are not suitable for constrained CSO problems, where feasibility constraints potentially even integrality constraints-mis be satisfied for every decision.
An alternative family of methods, commonly referred to as prediet-chen-optimiiz, follows a two-stage ap- proach: first, the conditional distribution of the uncertain parameters estimated given the observed covari- ates; then, this estimate is used to approximate the expected cost and solve the corresponding optimization problem (Bertsimas & Kallus 2020 Ban & Rudin 2019 Kannan et al. 2024 Deng & Sen 2022). Such two-stage approaches offer modularity and interpretability; yet, they risk suboptimal decisions when ac- curate predictions do not translate into good prescriptions particularly when the loss function used in training is misaligned with the downstream cost. To mitigate this, recent work has focused on end-to-end training schemes that integrate optimization objectives with predictive modeling (Bengio 1997 Donti et al. 2017 Kallus & Mao 2023 Qi et al.) 2021 Elmachtoub & Grigas 2022). These methods take into account the optimization task loss during predictive model training. However, they introduce new challenges, such as increased computational burden during training. In addition, they have been shown often achieve substantially slower regret convergence rates compared to the former non-integrated approach (Hu et al.
2022
Each of these paradigms presents trade-offs between statistical guarantees, computational complexity, and decision quality. In particular, when multiple methods or policies are available perhaps developed using different paradigms, modeling choices, or feature sets-an an important question arises: how should one select among a set of candidate policies? Tn this paper, we address this question through the following contributions:
1. We formalize the problem of policy selection in contextual stochastic optimization, where covariates are available as contextual information, and decisions must satisfy hard feasibility constraints.
2. We introduce Prescribe-then-Seeee SSee-eeett, modular framework that first generates diverse library of feasible candidate policies and then uses supervised learning to train a meta-policy that maps each covariate to the best candidate policy.
3. We conduct extensive computational experiments on two benchmark CSO problems in operations man- agement: newsvendor and shipment planning. Results show that the proposed meta-policy improves over the best single policy in settings where different policies perform best across different regions of the covariate space, and matches the best single policy otherwise.
2 Related Work
Our work relates to several strands of literalure on selecting among a set of candidate decision policies in the presence or absence contextual information. While there extensive research on policy ensem- bling, mixture-of-xxerrts architectures, and divide-and-conquer strategies in reinforccment learning (RL), the problem has received far less attention in the setting of contextual optimization under uncertainty within operations research, where feasibility constraints and explicit cost function minimization are critical.
2

---

Published in Transactions on Machine Learning Research (01/2026)
One line of work in reinforcement learning examines policy ensembling in context-free settings, where multiple policies are combined to improve stability, robustness, or exploration without conditioning on the current context (Wiering& & Van Hasselt 2008 Duell & Udluft 2013). These methods typically aggregate policies by averaging their action outputs or using majority voting, aiming to reduce variance, mitigate overfitting, and hedge against the weaknesses of individual learners. While effective in improving average performance, these methods do not adapt policy choice to specific states, tasks, or other contextual information, and therefore cannot exploit scenarios where different policies are optimal in different contexts.
To address such heterogeneity, RL has developed context-dependent policy selection mechanisms, most prominently through mixture-of-experts architectures. Here, meta-policy-oftee referred to a gating function maps the current context to a distribution over expert policies. In early work (Doya et al. 2002 Samejima et al. 2003; Van Seijen et al. 2008), the context was simply the state of the environment, leading individual policies to specialize in different regions of the state space; the meta-policy would then either combine their actions (soft selection) or choose the highest-scoring policy (hard selection). More recently, Gimelfarb et al. (202T) extended this approach contextual policy transfer, in which the context incorporates higher-level task descriptors in addition to the state, allowing the meta-policy to select from library of policies that were trained on different tasks but may transfer knowledge to the current task. Related approaches in divide-and-conquer RL partition the state space and assign a policy to different regions Ghosh et al. 2017 Goyal et al., 2019). While these techniques achieve context-dependent specialization, they are typically studied in online settings and operate in unconstrained action spaces where policies can be aggregated without violating feasibility. Moreover, partitions of the contextual space are often learned to optimize surrogate objectives that are disconnected from the true cost. function of interest in the rewnstream optimization problem.
In contrast, contextual optimization under uncertainty has seen almost no work on combining or selecting from multiple policies. The only example we are aware of is Cui et al. (2025), who recently proposed learning a fixed linear combination of candidate policics to improve decision quality. Their method uses historical data to estimate weights for each policy and forms a weighted sum of their putputs, demonstrating performance gains in the unconstrained newsvendor problem (Khoujal 1999). While promising in unconstrained settings, this approach is not generally applicable when decisions must satisfy hard fcasibility constraints, since averaging feasible policies does not necessarily produce another feasible policy. Therefore, there remains need for methods that. leverage contextual information to select. the best policy from feasible candidates.
In this paper we study the problem of policy selection in single-stage and two-stage contextual optimization under uncertainty, where decisions do not affect the underlying uncertainty and must satisfy hard feasibility constraints. We propose general framework which first generates a library of feasible candidate policies, and then learns supervised selection model to choose exactly one policy for each instance based on its context. By tailoring the policy assignment to cach observed covariate, the approach can outperform any single candidate policy on average, especially when different policies are optimal in different regions of the covariate space. This enables flexible reuse of hirh-ppeforring policies, adaptation to heterogencous contexts, and robust performance across complex decision environments.
The remainder of the paper is organized as follows. Section g introduces the general formulation of contex- tual optimization under uncortainty, and Section g presents the proposed Prescribe-then-Selee framework. Section 5 reports the results of our computational experiments, and Section offers concluding remarks.
3 Problem Setting
We consider the problem of contertual stochastic optimization (CSO) Sadana et al. 2025), in which a random vector of covariates X E X' C Rd= capturing context such as holidays, promolions, or seasonal trends-is is observed as side information in a stochastic optimization problem. After observing X = æ, the decision-maker sclects an action z from a fcasible set 2 C R&d reprosenting all constraints that the decision z must satisfy. The objective is to minimize the expected cost with respect to some unknown random variable Y E y' C RRd. We assume that cost. function e: Z y - R. is provided, and that decisions do not affect
3

---

Published in Transactions on Machine Learning Research (01/2020)
the uncertainty Y. Specifically, the problem can be written as
v² (ax) min E[c(z, X =x] x),, Tt (a) ZEZ
arg min B[((2, #E2
Y) X x).
(1)
where of is the optimal value function and n= is the optimal policy function. Given dataset DN {(;; 1i) with historical obsorvations, the data-driven approach to solving equation consists of using these observations lo approximate the conditional expectation in the objective.
Two classical baselines illustrate the spectrum of data-driven approaches. At one extreme, the sample- avergge-appaazimatoon (sce, o.g.. Kleywegt et al.) (2002) ) method ignores covariates altogether, replacing the conditional expectation in equation with its unconditional counterpart and approximating it via the empirical average. While simple to implement, sample-average approximation produces the same decision for all contexts, even when covariates could help narrow down the realized value of the uncertainty Y. At the other extreme lies the
approach (see, e.g., Bertsimas & Kallus (2220); ), which fits a predictive model (x) to estimate E[ X = ær). and then substitutes this prediction into the objective, converting the problem into arg mingez c(z,AN (z)). This method tailors decisions to the observed context and is computationally efficient, but reduces the entire conditional distribution to a single point est estimale, ignoring its variance and how the cost function responds to uncertainty beyond the mean.
An important obstacle in CSO problem is that X and Y can be highly variable across regimes. For instance, in retail sales, routine weekdays, promotional weekends, and holiday peaks may each exhibit very different conditional demand distributions fr z). Such heterogeneity in the covariate space may also induce heterogencity in policy performance, with different policics performing best in different covariate regimes. For instance, low-variance regions often reward the precision of point- pontt-preiitton, whereas high-variance regions may benefit from approaches that explicitly account for uncertainty approximation. Likewise, many other existing approaches the CSO problem may perform well in certain regions of the covariate space but not average across the entire space. Figure SigureDillsse Iilustrates this phenomenon for the classical newsvendor problem, where different candidate policies dominate in different segments of the covariate space, an empirical result examined more thoroughly in Section 5.4.1 These observations motivate the use of mult candidate policies, each designed to perform well under a specific regime, logether with a selection mechanism that assigns the most cest-effective policy to each context. The remainder of the paper develops and analyzes this approach.
c(z, Y) such as sample average
Segment A
Segment B
Segment C
15000
10000
10000
12500
0800
ANNrage proftt
12000
Aveer Awwwragp jreffitt
Averaage preffit
9500
-O- S4A
9V/0
-C- PP -NN
11500
9000
+ PP-37
+Q- PPH-E
L1800
C-040
9030
8500
000000
000000000
1000
8000
D00000000
10300
1000
2000
3000
4200
3000
1000
2030
3000
4000
5000
1000
2000
3000
400C
3000
Training Size
Treining Size
Training Size
Figure 1: Newsvendor: segment-wise profit vs. training size (average and 95% CIs shown).
4 Methodology: Prescribe-then-Seees
This section presents our proposed method for addressing covariate heterogencity in contextual stochastic optimization by combining multiple decision policies within a unified framework. Instead of relying on a single model across the entire covariate space, we construct a library diverse candidate policies and use a meta-policy to select one among them basod on the obscrved covariates. Each candidate policy maps contextual features to feasible decisions under distinct modeling assumptions and inference mechanisms. They may differ in the type of predictive estimates they produce, their complexity, or their inductive biases.
4

---

Published in Transactions on Machine Learning Research (01/2026)
The meta-policy then leverages this diversity by jointly learning partition of the covariate space and deciding the most effective policy for each region.
We illustrate the methodology using a few representative examples of individual policies that often exhibit heterogencous performance across the covariate space. The framework, however, is readily extensible and can incorporate policies developed under other approaches to the CSO problem. We obtain the meta-policy by training an ensemble of policy trees, which are depth--onstrrinee decision trees over the covariate space that aim to minimize the empirical cost of the CSO problem over historical data. We refer the proposed method Prescribe-thens then-Select, reflecting its two-stage structure: first prescribing set of candidate policies, then selecting the most suitable one for cach region of the covariate space. The following subsections describe each component of this method in detail.
4.1 Prescribing: Developing a library of candidate policies
The first stage of our framework constructs a library of prescriptive models, each representing a policy that makes feasible decisions given the observed covariates. We here consider three families of prescriptive meth- ods: sanulle-vengg-appoxinnatoo policies, point prediction policies, and predictive prescriptive policies. However, we highlight that our framework could include any other feasible policies to the CSO problem.
4.1.1 Sample Average Approximation (SAA).
The sampl-averagg--ppoxiaaaoii method (Keeywegt et al. 2002 is a classical approach for solving stochas- tie optimization problems by replacing the true expectation in equation) [ with an empirical average computed from a finite sample. Given N historical scenarios {y each representing a realization of the uncertain parameters, SAA approximates the expectation as E[((2,,)) as by solving the following optimization problem:
N E c(2,ys) and finds a prescription
SAA
arg min Σ c(z,
(2)
ZEZ
Unlike other CSO policies, SAA does not condition covariates and instead treats all observed sce- narios equally. As result, it is well suited to settings where covariate information is absent or irrelevant. Solving equation reduuees the problem to single deterministic optimization problem, making SAA straight- forward to implement.
4.1.2 Point-Prediction Policies (PPt).
The point-prediction framework first fits a predictive model (a) & E| X x= deterministic optimization problem where uncertainty is collapsed to its predicted mean:
= æ] and then solves a
x
PPT (x) = arg min ((z,Ai(()) sEZ
(3)
This approach is efficient and interpretable, and often performs well when the cost function is approximately linear and the conditional distribution of Y has low variance. We illustrate this approach with two predictive modols:
(a) PPI-KNN: Given data point x E X, the k-nearest neighbor model (kNN) (Cover & Hart 1967
identifies the k training samples {(x,, y:)} closest to x under a fixed distance metric. Denoting by No. the set of indices i for these neighbors, majorily vote in the case of calegorical uncertainty.
(xx(x) Eiens(a) Yi in the case of numerical uncertainty,
Random forests (RF) are ensemble models that aggregate predictions from multiple decision trees, each trained on a different boolstrap sample of the data and typically using randomized feature selection at each split (Breiman 2001). This procedure, known as bagging (bootstrap aggregation). reduces variance and enhances generalization. Let Ti...., Tв denote the trees in the forest, and let Ls(x) be the set of training indices falling in the same leaf as in tree Tù. The predictions for given input x are then given by (x) (x) 1/ Et [[([]]] Eiecs(0) yi.
(b) PPt-RF:
5

---

Published in Transactions on Machine Learning Research (01/2026)
(c) PPt-NN; A feed-forward neural network (NN) (Gooffello et al. 2016 is defined nonlinear map So X' + Rdd that recursively applies affine transforms and non-linear activation functions. Starting from (0)) = x, each layer computes h = (. (6-1) + b()) where Φ is the activation function (e.g., ReLU, tanh) for all & E [L] and finally outputs fo(x) = o(h) where a is the identity function for regression tasks or the softmax function for classification tasks. Parameters 0 {W(0), (,1) )EEL) are typically fit via empirical risk minimization with stochastic gradient methods. The PPt prescription sets px( = fo()).
4.1.3 Predictive-Prescriptiv Policies (PP).
The predictive prescriptive framework (Bertsimas & Kallus 2020 Bertsimas & Dunn 2019 integrates local statistical estimation with cennario-based optimization, and can be interpreted form of weighted sample- wverage-aprooximation (Sadana el al. 2025 Given a predictive model that assigns weights wv,; (2: to each
training pair (at, y.), this approach approximates the conditional expectation in equation ! as
Efc(2, Y) X 2)
Σ (N) (z..9)). and solves 7 (<"()) E arg min ZEZ
Σ WN (x)
(4)
These normalized weights (Eiz (x) = =) capture the local distributional information implied by each model and are used to generate prescriptions tailored to the covariate æ. Using the PP framework, the kNN and RF predictive models would lead to the following weighting schemes:
(a) PP-KNN: In this method, the nearest neighbors data point are not used to estimate E[Y | X = x). but rather they are used to estimate a local approximation of the conditional distribution frix(: (: x). Specifically, the weights the PP approach are given by:
1 E. if
KNN (x)
0, otherwise.
(b) PP-RF: For a given data point E X, each tree in the forest routes æ to a leaf node. The collection of training samples retrieved from the leaf nodes across all trees forms local, ensemble-base approximation to the conditional distribution fvix(-1 æ). As with kNN, this distribution can be used to support scenario- basod optimization. The PP approach in this case defines the weights as:
B
1 E Cx(2)
B
[s(z))
4.2 Selecting: Learning the best policy for each context
The second stage of our framework learns meta-policy that chooses the best candidate policy for the given context. Let the library of candidate policies be denoted by II = 1 each mapping covariates x E X to a feasible decision z E 2. To learn the meta-policy we adopt Optimal Policy Trees (OPT) (Amram et al. 2022), which partitions the feature space into axis-aligned regions and assigns a fixed policy index to each region. While extensions that allow for more general splits such as linear combinations of features do exist, we do not explore them in this paper.
OPTs solve a treatment assignment problem from observational data. The OPT method learns a decision free that maps covariates to the treatment that optimizes expected oulcomes. Importantly, we do not use the OPT to obtain a feasible policy for the CSO problem, since just like parametric decision rules, the OPT could not be tractably optimized to satisfy hard constraints on the decisions. We instead leverage OPTs to select among a set of feasible candidate policies based on the observed covariates. other words, we consider each candidate policy as a treatment; and train the OPTs to learn which treatment best given the contextual information.
Formally, a policy tree is decision tree T(x: θ), parameterized by θ = {(Rg. (5)))))))m where {R, is disjoint partition of X' and 7j E [M] denotes the policy index assigned to region Ry, with [x] = (1.....n) n). The policy induced by this tree is x (x))))))), where T(x; θ) = Ey=r' N 1fx E Ry). The policy tree itself does not produce decisions, but rather selects which candidate policy to invoke for given input. Since all
6

---

Published in Transactions on Machine Learning Research (01/2026)
policies H" produce feasible decisions, does n (x))))))) The goal is to find the policy tree that solves the following minimization problem:
Ô arg min E (X)TT
(5)
By construction, the optimal policy tree cannot perform worse than the best single policy in the library. Moreover, there exists a region of the covariate space where another policy offers a sufficiently large improvement in conditional expected cost, the optimal policy tree will achieve a corresponding improvement in overall performance. The following lemmna formalizes this property.
Lemma 1 Suppose that X,Y are obtained from a joint probability distribution, and let m² be the inder of the policy with the smallest overall expected cost (x(="(()) from the library II {- Ld m== Suppose
there exists a region C with Pr(X E R) > O and policy m o m² such that E [c( th (X), x) XER] s
E [c(TT" (X), Xe R] -6, for some 0. Then E ( S W[((TT (X), r) P-Pr E Rt.
The proof follows because we can construct a tree that assigns policy m to R and policy m to its complement R. The expected cost of this tree is
Pr(X E [(T | R] Pr(X e B o( (X). x E R
Subtracting the cost of always using m² yields an improvement of least 8 Pr(X E R) > Since the optimal policy tree T(.: : Ô) cannot do worse than this construction, the claim follows.
Although evaluating the expectation and handling the non-convex objective in equation make computing e challenging, several heuristic algorithms have been proposed to address this problem. In particular, we adopt the Optimal Trees algorithm for its strong empirical performance in similar policy selection tasks and its ability to scale to large dalasets (Amram et al., 2022). This method uses a coordinate descent. approach to minimize a regularized empirical risk version of equation 5 At each iteration, the current tree structure determines the best prescription for each leaf; these prescriptions are then evaluated in the objective, and the result guides the next coordinate descent step.
4.3 End-to-End Pipeline: Training and Inference Phases
components into unified end-to-end framework Prescribe-teen-Seeee (PS). The process consists of two stages: a training phase, which constructs both the candidate policies and collection of policy trees for selecting the best among them, and an inference phase, which uses the ensemble of these trees to form meta-policy that generates context-specific prescriptions.
We now integrate the Prescribe (Section and Select (Section 4.2
Training phase
Inference phase
K-foid crass-validation for OPT training
R OPTs trained on held-out fold
New test datopoint
Creato sach solit
Select
GPTS
OPT
Creade such pollcy
Select
Oreats euch palay
OFTIV+ PH
OFT
Ensemble OPTS (majonty
Compute cost of (i)))) (z)) using
Select
Croade such policy
CFT
Se Selected policy (1e)
Full-training
entire
Trained for each policy
Final decision (2
Figure 2: End-to-end pipeline. Left: K-fold, R-replicate training produces an ensemble of OPTs, each trained on a distinct held-out fold using its corresponding cost table. Right: At inference, a new context is routed through the OPT ensemble, and the majority-vote policy is applied to produce the final decision.
7

---

Published in Transactions on Machine Learning Research (01/2026)
Training phase. Let Durain ((z,, 1 denote the training set, and let {I(1,
I I(K) be random
K-fold partition of the index set [N]. For given fold k, the training indices [(-k) [N] I(k) are used
to create the candidate policies x ....., T according to the procedures in Section
For each held-out
instance i I(() and each candidate policy m E [M], we assess the quality this policy on this observation
via the realized out-of-sample cost: C.m c (=*"()), v.). Arranging these values for all E I(A) yields
the cost table ((k) R | where
c(T (ai,),
c(T (x;,) (
((k)
ft f .200)))) = I(k).
c( (1) (22 1200)) )r y 1200s)
c(T (20 (203s)) yi Fi
We use this k-fold cross- volsdvaliottion approach to prevent overfitting, since deploying the candidate policies out of sample prior to OPT training allows the OPT to identify which policies tend to generalize best in different regions of the covariate space. For each fold k, we train Rt independent OPTs ( ((k.n) ",.,,,,,.,,,,.,, which take imput the pairs (æ,, C. for I(A), solving equation 5 with R different random seeds mitigate the fact that the learning algorithm is stochastic heuristic and may not always find a global optimum. After repeating this procedure for all K folds, we have total of K x R policy trees. Finally, all candidate policies w!...., n are refit on the full training set Derain lo maximize the performance on the inference phase. A detailed, per-fold view the procedure appears in Figure while Algorithm | in the appendix summarizes the complete training phasc.
Inference phase.
Upon observing a covariate X, each OPT T(k.)) outputs a policy index ((k.)) (x) e
[M], and we aggregate those outputs by majority vote y(x) mode(f (+rr) k E [K], r [R])), break- ing ties uniformly at random. The final decision made by the meta-policy is obtained by applying the selected
policy x>(=)(x) (see Figure 2 This procedure is explained in detail in Algorithm in the appendix.
We highlight that PS does not use additional data beyond what each individual policy uses. All candidato policies and the meta-policy use the exact same data for model development and testing.
Z(-A)
Z(R)
Create each policy
Select
Use (at, si) to train ML models
Make decisions (2z;) Vi Iik). Vinw è [M]
Compute test using the true yn
Train policy
weights
K Nearest
Preesriivee- KNN
Nelghbours
preaiicion
Point Prediction KMN
C,y cff""(an))
Random
Predictive Prescriptive RF
Forest
point-presictton
Point-Predde Point-Predictio
Figure 3: Detailed view of one fold in the training phase. In fold k; models trained on -(- produce
sample weights point predictions, mapped to decisions x" (ax)) for each I(k) and m II. Evaluating these using the true outcomes yields the cost matrix ((k), which, together with (æ, : E [kk),, }, trains R Optimal Policy Trees. Across all folds, this produces R K OPTs. We show an example with M = 4 policies (predictive prescriptive kNN, point-prediction kNN, predictive prescriptive RF,, and point-prediction RF).
5 Computational Experiments
In this soction we evaluate Prescriee-then-Sloct on two classical contextual stochastic optimization tasks that are widely used benchmarks in the operations research literature (c.g. Bertssmas Kallus (2020):
Bertsimas et al. (202)): Huber et al. 2019): Ban & Rudin (019)) Bertsimas al. (008b))) The first. task
8

---

Published in Transactions on Machine Learning Research (01/2026)
is two-stage shipment-planning problem, where initial production decisions made multiple facilities before demand is realized, followed by additional production and shipment decisions once demand becomes known. The second task is a single-stage newsvendor problem, a classic inventory management setting where stocking levels must be chosen in advance under uncertain demand. For both tasks, we introduce context heterogeneity by generating synthetic datasets with distinct demand segments for example, holiday spikes versus routine woekday or weekend cycles-cade following different distributions.
5.1 Multi-Product Newsvendor
The pulti-product newsvendor problem is classical model in inventory management (Khouja 1999 where decision-maker must determine order quantities for multiple products before uncertain demand is realized. It is widely used in retail, wholesale, and mamufacturigg to balance the costs of understocking (lost sales) and overstocking (excess holding or disposal). In the multi-product setting, items compete for a shared resource such as storage space or budget, and the meeee seeks to allocate this capacity as to maximize the expected profit. We consider the contextual version in which the decision-maker observes covariales as side information (e.g., calendar or seasonality indicators) before ordering.
Let P, C, E R4 denote prices, costs, and storage requirements for d products, which share total storage capacity S >>>0 0. The random vector Y E IR represents the (unknown) demand, and the observed covariates. The decision maker chooses stocking levels q to maximize expected profit, formulated as
max EE P min ( c q x === m]] s.t. 8 T 920
where the minimum min(Y, q} is applied component-wise. In our experiments we set d = and S = 1200. Prices p, costs c, and storage coefficients s are set such that higher-price items carry proportionally higher unit costs and larger storage requirements while maintaining positive margins (p > c). This yields comparable profit magnitudes across products and nontrivial trade-offs under the knapsack constraint s S. The exact parameter values for prices, costs, and storage coefficients are provided in Appendix B Moreover, since E p² min(Y, q} X = æ| = E Ps EJ min(Y, 9.) | æ (by linearity of the expectation). the implementation of both the PP and PPt policies for this problem estimate the distributions fyzjx(: xx)
for all j E [d] instead of f-(x(-1 x).
5.2 Shipment Planning
Shipment-planning is relevant problem to manufacturers, distributors, and logistics providers socking to minimize transportation and production costs while meeting demand requirements (Bertsimas & Kallus, 2020). In two-stage setting, the planner commits to an initial production plan before demand is observed. and then adjusts additional production and plans shipments once demand becomes known. We again assume the decision-man maker has access to contextual information prior to making decisions.
Formally, production facilities, I demand locations, and covariates E X available as side information. In the first stage, before demand is observed, the planner chooses u E _ with unit cost Pt- After demand R. at price P2 > P1 produced, and shipments U2 from location
Y E RV is realized, additional items f to location f at per-unit costs efé 2 0 must be decided. Each product yields per- unit revenue >> 0 and full demand must be satisfied. The cost minimization CSO problem can then be written as
æ]. min Pi Fu + E[Q(uu))) X =x], u.ER
where
Q(u:y)) =
min
ΣΣ
Cft f2 + P2 The s.t. U 2 e U2,, - u
UZEREXL CERF /=18=1 t=1
In our experiment, we set F L = 4, P1
5, P2 10, a 90, and draw fixed shipping costs via
Cye = 20 + 2( 1) + Eye with Ege ~ Unif(0,3).
9

---

Published in Transactions on Machine Learning Research (01/2026)
5.3 Implementation Details
Dala Generation: Our data-generation process is designed to simulate demand data with explicit covariate- driven heterogeneity to create distinct demand segments, following common piecewise eataagennrattinn ap- proaches 4Amram et al, 2022 Zhou ct al.) 2023 Bertsimas et al. 2019). In the multi-product newsvendor setting, segments capture patterns such as holiday-driven spikes, smooth seasonal variation with weekday effects, and short-term disruptions. In the shipment-planning setting, segments reflect operational patterns such as contracted replenishment periods, event-driven surges, and routine demand with calendar effects, In both cases, regime activation is determined by calendar covariates, and we add gaussian noise to in- troduce variability across products or locations. Within each segment, demand is generated using smooth covariate-driven functions combined with i.i.d. Gaussian and Bernoulli distributions Citribbutions-consissttttt with stan- dard practice in the literature (e.g. (Bertsimas & Kallus 2020: Kallus & Mao 2023 Bertsimas et al. 2023 Ban et al. 2019)). ). Full specifications, inchuding functional forms, parameter values, and segment definitions, are provided in Appendix and [ directly from the training data, without access to the ground-truth segments used for data generation.
We highlight that each OPT in the PS framework learns its own partition
Software Implementation: Our implementation combines Python for the machine- components with Julia for the optimization layer. The main packages are Gurobi (v12.0.1) for mathematical programming, the OPT framework from Interpretable AI (v3.2.2) for model selection, and in Python, scikit-learn (v1.6.0) and pandas (v2.2.3). Gurobi is accessed through Julia, while preprocessing and evaluation run in Python.
Implementation Setup: We use K = 5 folds and R = 10 repetitions, yielding K x R = 50 trees per ensemble. PS chooses among the base policies (SAA, PPT-RF, PP-RF, PPN- PP-KNN, PPT-NN}. We evaluate training sizes N E {250, 500, 750, 1000, 1250, 2000, 3000, 5000); for each N draw 100 independent training samples and assess strictly out-of-sample performance on fixed test horizon. For the prediction models, we use kNN with 5 neighbors and RF with number of trees B = 5 and a feed-forward neural network with hidden layers of sizes (16, 32, 16), ReLU activations, and early stopping.
Evaluation: We evaluate policics on data not used for training or meta-policy construction. Let I be the index set of held-out test points and Drest = ((xs; (() : Ztest.. To evaluate a policy т, we compute problem, evaluating c(T() y:) requires solving the second-stage optimization problem defining Q(T(<:);; This guarantees feasibility of the second-stage decisions U2 with respect to the realized demand yı and first-stage decision (x:),, ensuring that all policy evaluations satisfy the full constraint structure of the two-stage model.) We report this quantity for the candidate policies {n²m MM J==1 as well for the meta-policy x7() To quantify variability as function of the training-set size N, we draw S 5=100 independent training samples for each of the sizes considered and run the full pipeline in Section 4.3 yielding base policies { JM
its average test profit cost(T) [ ExErome (I(Zz,) y:) (notice that in the
and meta-policy 2(0) N.s for E [S]. For policy in sample s with data-size N, its average test cost is then cost(m ). We estimate the mean of (cosl(T)) (,)))) over samples s and construct a two-sided 100(1 a confidence interval (CI) using the Student-t approximation:
8
AN
1 S
cost(TTM..
cost p
CI N
am AN + 02- S-1
S -
VS
5.4 Results
In this section, we report performance on the test set as described in Section research questions. Unless stated otherwise, each curve in the plots shows mean profit across repeated samples with two-sided 95% t-based confidence intervals. We report profit as the negative of total cost for shipment planning and revenue for the newsvendor problem, such that higher values are better in all figures.
1 to address three main
5.4.1 Segment-wis wise heterogeneity
RQ1: Do the candidate CSO policies exhibit heterogencous out-of-sample performance across the covariate space, does single policy dominate the rest? To answer this question, we
10

---

Published in Transactions on Machine Learning Research (01/2026)
compute out-of-sample performance function of training size for six candidate policies (SAA, PP-KNN, PPN-KNN, PP-RF, PPT-RF, and PPr-NN) and three data regimes, and we analyze the results below.
Newsvendor, As shown in Figure 1 we indeed observe heterogeneity across segments:
Segment A (small variance for holiday-influennce products). For small N, PPT-RF attains the highost profit, consistent with low-variance setting where point prediction works well. As N grows, PP RF matches PPT-RE suggesting improvements in the local distribution estimates of the PP method.
Segment B (medium variance for products with seasonal smooth demand). PP-KNN is best for small N, which expected as the averaging effect tends to work well for continuous functions. As N increases (c.g., 3000-5000), PP-RF becomes the best method.
Segment C. (high-variance products with discontinuous summer demand). PP RF dominates for all train- ing sizes. This is expected as kNN averaging effect fails to capture abrupt changes and noint-prediction variants be very inaccurate when the prediction falls on the wrong side of the demand discontinuity.
Segment A
Segment B
Segment C
36400
13900
13800
Altrag: profE proftt
Averape proftt
Average peragee prrffit
21000
13700
-C- PP
PP-ENN
200
20000
31400
23900
PP--F
13900
PP:-NN
25030
13490
000
25700
23000
32
1000
2000
3000
4000
3000
1000
2000
3000
4000
5000
1000
2000
3000
4000
3000
Training Size
Tteining Size
Training Size
Figure& 4: Shipment: segment- wise mean test profit vs. training size (95% Cls; profit net cost).
Shipment Planning. Heterogeneity is also observed in this task, as shown in Figure
products with random discontinuities). PP-KNN yields higher profit across most training sizes. Forests were observed to wrongly place calendar splits that affected performance in these segments, increasing leaf variance and harming the weights in the PP method; while kNN neighborhoods remained fairly stable.
Segments A/B (small-uariance
Segment C imediun-uariance variance for products with seasonal smooth demand). PP-RF delivers the best performance, as the problem involves a complex continuous demand function. By contrast, kNN fails to capture the segment structure, lacking the flexibility of RF to adapt effectively. PPT-NN attains intermediate performance: improves with training size but remains below the RF methods for all N.
Across both tasks the answer to RQ1 is affirmative: test performance heterogeneous, and no single policy dominates uniformly. Which candidate policy is best depends the segment and the training size. With limited data, the best policies differ by context (e.g., PPT-RF in low-noise segments; PP-KNN where local averaging reduces variance), whercas in segments with abrupt regime changes PP-RF is consistently superior at all N. As N grows, PP RF closes the gap with point-prediction in simpler segments and usually overtakes other methods. We also observe cross-segment spillovers: because models are trained on pooled data, noise introduced by other segments (e.g., latent holiday holiday/event effects) can make tree leaves noisier and depress RF performance even in otherwise regular segments, while kNN's locality is less sensitive to such noise.
5.4.2 Benefit of Prescribe-then-Seees
RQ2: Does Prescribee-then-Select outperform the best single policy, is the improvement sta- tistically significant, and how does it vary with training size? To answer this question, we conduct experiments on both benchmark problems, comparing PS against the best single-policy bascline across a range of training sizes. Figures 5a and 5b show average profit as a function of the training size with two- sided 95% (-based confidence intervals for PS and all candidate policies.
11

---

Published in Transactions on Machine Learning Research (01/2020)
22440
-9-
10000
-Q-
19435
22420
10000
Neerage Profft
22400
Avaragg
22300
2130
Average
31000
0300
3000
0--0-0--00
2020
0::0-0
1000
3000
2000
4003
3000
Training Sizee
Teitning Size
(a) Newsvendor.
(b) Shipment Planning.
Figure 5: Profit training size for PS and the best single candidate policy in the (left) newsvendor and (right) shipment tasks. (95% CIs; profit net cost.)
Newsvendor. For small samples (N E [750, 1500]), PS is statistically significantly better than all individual policies (non-overlapping or barely overlapping CIs at multiple N), reflecting its ability to route contexts toward PPr RF (best in Segment A for small N) and PP KNN (best. in segment for small N). The magnitude of the gain in this regime is modest absolute terms but comparable the improvement peedictive-prescrpplle
obtained when moving from point-prediction (N> 1500), PP-RF dominates and PS rapidly converges to PP-RF (by N =2000).
baselines. As data size increases
Shipment Planning. PS gains become apparent once the training set exceeds N 1000, mirroring the paltern observed in 5.4.1 Segments A/B favor PP kNN, whereas Segment C favors PP RF in this data size regime. Improvements strengthen at N E (3000, 5000} where the 95% Cls only slightly overlap. Since the problem two-stage with costly recourse (p2 > Pi), choosing the wrong policy is highly expensive in terms of cost, which affects average gains even PS increasingly routes A/B to PP-KNN and C to PP-RF.
Across both tasks, PS yields significant gains by learning partitions of the covariate space from cross- validation alone and finding the best candidate policy for each region. Notably in the shipment setting, policies that are very weak on average and would be disregarded in general frameworks such as PP-KNN and PPT-kNN become useful: PS deploys them segments where they excel (A/B) and defaults to PP-RF RF clscwhere (c.g., C), thereby improving overall performance. Illustrative examples of the OPTs being trained are in the Appendix D We note that some of the policies we consider such PP-kNN are asymptotically optimal under standard assumptions: their expected loss converges to the optimal loss the sample size grows [Bertsimas & Kallus. 2020). However, these experiments demonstrate that, with limited data sizes, no single policy from our library achieved the optimal loss during testing, highlighting the benefit of PS.
5.4.3 Uniform dominance and convergence
RQ3: In the absence of substantial segment heterogeneity, does PS effectively revert to the best base policy, does the selection step risk choosing an inferior policy? In other words, want to evaluate whether, in regimes where single base policy is uniformly superior across the covariate space, PS asymptotically selects that policy and thus incurs negligible regret relative to the best single policy, In the newsvendor task (Figure 5a for N Z 2000 the mean profits of PS and PP-RF are statistically indistinguishable within two-sided 95% t-based confidence intervals, indicating effective convergence PS to the dominant policy. Because PS is trained solely via cross-validee valddaaddd folds and never observes test data, this demonstrates PS low-risk default: when uniform dominance emerges, PS matches the best single policy, while at heterogeneous cases it retains the performance gains.
5.4.4 Summary of findings and computational considerations
Our results show that (i) out-of-sample performance is segment-dependeen with no single candidate policy dominating the others; (ii) PS delivers statistically significant improvements over all single policies in het-
12

---

Published in Transactions on Machine Learning Research (01/2026)
erogeneous regimes; and (iii) when single base policy becomes uniformly dominant, PS converges to it. Computational Cost. With K cross- vross-vali folds and M candidate policies, each ML model is trained K times, and is evaluated for all M variants each fold-yielding K M sets of optimization problems to solve. Even though creating the meta-policy introduces additional step, its cost is small compared to solving the optimization models themselves (see Appendix DD Thus, policy selection adds minimal overhead relative to standard CSO workflows where multiple policies are typically evaluated to identify the most suitable onc. The only additional cost specific to PS is training the meta-policy, an ensemble of K x R OPTs, which adds modest training time and memory overhead and incurs negligible inference latency.
6 Conclusion
We introduced Prescribe-then-Seeett, a modular framework for adaptive policy selection in contextual stochastic optimization. PS generates diverse set of candidate policies and learns a selection model that matches each context to its most effective policy, enabling flexible integration of diverse policies while pre- serving constraint feasibility. Our experiments on two benchmark problems showed that PS consistently exploits heterogeneity in the covariate space to improve upon the best single policy, while converging to the dominant policy in homogeneous regimes. These findings position PS as a practical, low-risk approach for decision- makion environments where diverse, high-quality prescriptive models are already available. Future work includes extending P$ multi-stage settings, exploring other selection models beyond policy trees, and evaluating performance in large-scale, real-world applications with high-dimensional and partially observed covariates. An additional direction is to study cases where policy averaging is feasible and to consider the development of meta-policie that learn how to combine candidate policies instead of selecting only one.
13

---

Published in Transactions on Machine Learning Research (01/2026)
References
Maxime Amram, Jack Dunn, and Ying Daisy Zhuo. Optimal policy trees. 2741 2768, 2022. doi: 10.1007/1109944-22--00222
Machine Learning, 111(7);
Gah-Yi Ban and Cynthia Rudin. The big data newsvendor: Practical insights from machine learning. Operations Research, 67(1):90 1089900-08,-000 2019.
Gah-Yi Yi Ban, Jérémie Gallien, and Adam J Mersereau. Dynamic procurement of new products with covariate information: The residual tree method. Manufacturing & Service Operations Menagement, 21(4):798 815, 2019.
Thierry Bazier-Matte and Erick Delage. Generalization bounds for regularized portfolio selection with market side information. INFOR: Information Systems and Operational Research, 58(2):374-401, 2020.
Aharon Ben-Thr Tal, Arkadi Nemirovski, and Laurent El Ghaoui. Robust optimization. 2009.
Yoshua Bengio. Using a financial training criterion rather than a prediction criterion. International journal of neural systems, 6(04))433-444) 1997.
D. Bertsimas and J. Dunn. Machine Learning Under Modern Optimization Lens. Dynamic Ideas LLC, 2019. ISBN 9781733788502. URL https://books. wwwpookks.ooggl...ooooo http.///bbbooo/dddd333..........
Dimitris Bertsimas and Kimberly Villalobos Carballo. Multistage stochastic optimization via kernels. arXiv preprint (xXi::2303005111 2023.
Dimitris Bertsimas and Nathan Kallus. From predictive prescriptive analyties. Management Science, 66 (3):1025-100) 1044, 2020. doi: 10. 10.1287/ 0007//mno.20011 2010.m02.0.2.255....
Dimitris Bertsimas and Nihal Koduri. Data-drives optimization: A reproducing kernel hilbert space ap- proach. Operations Research, 70(1):454 471, 2022.
Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. Data-driven robust optimization. Mathematical Programminng 167(2))235 292, 2018a.
Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. Robust sample average approximation. Mathematical Programming, 171(1):217-28) 2018b.
Dimitris Bertsimas, Jack Dunn, and Nishanth Mundru. Optimal prescriptive trees. INFORMS Journal on Optimization, 1(2):164 183, 2019.
Dimitris Bertsimas, Christopher McCord, and Bradley Sturt. Dynamic optimization with side information. European Journal of Operational Research, 304(():634-651, 2023.
John Birge and François Louveaux. Introduction Stochastic Programmming. Springer, New York, 2nd edition, 2011. ISBN 978- 1-2664--2266--8
Leo Breiman. Random forests. Machine learning, 45(1):5-33 2001.
Giuseppe Carlo Calafiore and L El Ghaoui. On distributionally robust chance-constrained linear programs. Journal of Optimization Theory and Applications, 130(1):1-22) 2006. 2006.
Antonio J Conejo, Miguel Carrión, Juan M Morales, et al. Decision making under uncertainty in electricity markets, volume 1. Springer, 2010.
T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1):21-27, 1967. doi:
Xiangyu Cui, Nicholas G. Hall, Yun Shi, and Tianyuan Su. Collective wisdom: Policy averaging with an application to the newsvendor problem, 2025. URL httpss//aarxx...rr//a///200. 17638
14

---

Published in Transactions on Machine Learning Research (01/2026)
Erick Delage and Yinyu Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations research, 58(3):595-611 6126 2010.
Yunxiao Deng and Suvrajeet Sen. Predictive stochastic programming. Computational Management Science, 19(1):65-995 98, 2022.
Priya Donti, Brandon Amos, and J Zico Kolter. Task-based end-to-end model learning in stochastic opti- mization. Advances in neural information processing systems, 30, 2017.
Kenji Doya, Kazuyuki Samejima, Ken-ichi Katagiri, and Mitsuo Kawato. Multiple model-based reinforcement learning. Neural computation, 14(6):1347-1369, 2002.
Siegmund Duell and Steffen Udluft. Ensembles for continuous actions in reinforcement learning. In ESANN, 2013.
Adam N Elmachtoub and Paul Grigas. Smart "predict, then optimize". Management Science, 68(1):9 26, 2022.
Dibya Ghosh, Avi Singh, Aravind Rajeswaran, Vikash Kumar, and Sergey Levine reinforcement learning. arXiv preprint 1rxix:1711.000 09874, 2017.
Divide-and-conquer
Michael Gimelfarb, Scott Sanner, and Chi-Guhn Lee. Contextual policy transfer in reinforcement learning domains via deep mixtures--f-eeperrs. In Uncertainty in Artificial Intelligence, pp. 1787-1797. PMLR, 2021.
lan Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT Press, 2016.
Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng, Sergey Levine, and Yoshua Bengio. Re- inforcement learning with competitive ensembles of information-constraindd primitives. arXiv preprint arXiv:1906.10667, 2019.
Yichun Hu, Nathan Kallus, and Xiaojie Mao. Fast rates for contextual linear optimization. Management Science, 68(6):4236 42046,4225442-2 2022.
Jakob Huber, Scbastian Müller, Moritz Fleischmann, and Heiner Stuckenschmidt. A data-driven newsvendor problem: From data to decision. European Journal of Operational Research, 278(3):904 -9159099-1., 2019.
Nathan Kallus and Xiaojie Mao. Stochastic optimization forests. Management Science, 69(4):1975-100 1994, 2023.
Rohit Kannan, Güzin Bayraksan, and James R. Luedtke. Residuals- based distributionally robust optimization with covariate information. Mothematical Programming, 207(1):369 425, 2024.
Moutaz Khouja The single-period (news-vendor) problem: literature review and suggestions for future research. omega, 27(5):537 553, 1999.
Anton J Kleywegt, Alexander Shapiro, and Tito Homem-de Mello. The sample average approximation method for stochastic discrete optimization. SIAM Journal on optimization, 12(2):479 502, 2002.
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approxima- tion approach to stochastic programming. SIAM Journal on oplimization, 19(4:11574 1609, 2009.
Pascal M Notz and Richard Pibernik. Prescriptive analytics for flexible capacity management. Management Science, 68(3):1756 1775, 2022.
Afshin Oroojlooyjadid, Lawrence V Snyder, and Martin Takáč. Applying deep learning to the newsvendor problem. Jise Transactions, 52(4))4444446 463, 2020.
Meng Qi, Paul Grigas, and Zuo-Jun Max Shen. Integrated conditional estimation-optiiizatio. arXiv preprint ar Xix:222110 10. 12351, 2021.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Herbert Robbins and Sutton Monro. A stochastic approximation method. The onnals of mathematical stalislics, pp. 400-407, 1951.
Utsav Sadana, Abhilash Chenreddy, Erick Delage, Alexandre Forel, Emma Frejinger, and Thibaut Vidal. A survey of contextual optimization methods for decision-making under uncertainty. European Journal of Operational Research, 320:271-289, 2025. doi: 10.1016/1.coo::22244....222
Kazuyuki Samejima, Kenji Doya, and Mitsuo Kawato. Inter-module credit assignment in modular reinforce- ment learning. Neural Networks, 16(7):985- -944, 2003.
Alexander Shapiro. Monte carlo sampling methods. science, 10:353 1255,-425,-0 2003.
Handbooks in operations research and management
Alexander Shapiro and Arkadi Nemirovski. On complexity of stochastic programming problems. In Contin- wous oplimizalion: Currenl. trends and modern applications, pp. 111- Springer, 2005.
Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszezynski. Lectures on stochastic programming: modeling and theory. SIAM, 2021.
Lawrence V Snyder. Facility location under uncertainty: review. IIE transactions, 38(7):547-55)) 564, 2006.
Lawrence V Snyder and Zuo-Jun Max Shen. Fundamentals of supply chain theory. John Wiley & Sons, 2019.
Harm Van Seijen, Bram Bakker, Leon Kester, et al. Switching between different state representations reinforcement learning. In Proceedings of the 26th TASTED International Conference on Artificial Intelligence and Applications, pp. 226 231, 2008.
Marco A Wiering and Hado Van Hasselt. Ensemble algorithms in reinforcement learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38(4):930-936 2008.
Yanfei Zhang and Junbin Gao. Assessing the performance of deep learning algorithms for newsvendor problem. In International conference on newral information processing, pp. 912-921. Springer, 2017.
Zhengyuan Zhou, Susan Athey, and Stefan Wager. Offine multi-action policy learning: Generalization and optimization. Operalions Research, 71(1):148 183, 2023.
16


Reviews:
[{"review_id": "KyIV0oe5Pu", "reviewer": "Reviewer_xgRy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Prescribe‑then‑Select (PS), a modular framework for contextual stochastic optimization that first builds a library of feasible candidate policies (e.g., SAA, point‑prediction and predictive‑prescriptive variants with kNN and Random Forests) and then trains a meta‑policy (ensembles of Optimal Policy Trees) to choose, per context, which candidate to deploy. Using synthetic shipment‑planning and multi‑product newsvendor benchmarks with deliberately heterogeneous demand regimes, PS improves over any single policy when heterogeneity is present and effectively collapses to the dominant policy when one exists.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The paper’s central claims that (i) candidate CSO policies perform heterogeneously across contexts, (ii) a learned selector can improve over any single policy in such regimes while (iii) reverting to the dominant policy when heterogeneity fades, are supported by clear experimental evidence on two standard CSO benchmarks (multi‑product newsvendor and shipment planning). The evidence is internally consistent though it is limited to synthetic data.\n- The authors present segment‑wise performance curves for five canonical CSO policies. Different policies win in different segments and at different data regimes directly supports the intuition that “one size doesn’t fit all.”\n- The paper argues that averaging policies can violate hard constraints, whereas selecting one feasible policy preserves feasibility. This is a good motivation for hard selection in constrained CSO and is consistent with the chosen benchmarks.", "audience_yn": "Yes", "audience_explanation": "The work sits at the interface of learning‑to‑decide and robust prescriptive analytics, reusing strong prescriptive models while adding a learned gating layer that respects feasibility. This modularity is attractive to practitioners who already have trusted policies but face context heterogeneity. It complements approaches such as “predict‑then‑optimize” by learning when each paradigm works rather than committing to one.", "requested_changes": "1. (Nice to have) Compare against a projected policy‑averaging baseline where averaged decisions are projected onto the feasible set (when projection is well‑defined), or explain carefully why such a comparison is invalid for the studied problems. \n2. (Nice to have) Report compute metrics (training wall‑clock per N, number of optimization solves, memory, and inference latency for the meta‑policy and base policies) and discuss scalability as M, K, R grow.", "broader_impact_concerns": "In high‑stakes domains (e.g., healthcare logistics, loan default predictions), context‑conditioned selection can encode spurious correlations. Mis‑gating could degrade service for minority contexts.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5Yje9iJR4u", "reviewer": "Reviewer_e4ii", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary\n\nThis paper studies the contextual stochastic optimization (CSO) problem,\nwhere upon receipt of side information, a decision from among a\ncontinuous set must be chosen to minimize a (stochastic) cost. Towards\nthis end, the authors introduce Prescribe-then-Select (PS), a framework\nfor combining a set of \"base\" decision policy learning algorithms to\nproduce a superior policy that performs uniformly well across the\ncontext space. The approach uses an ensemble of Optimal Policy Trees\n(OPTs)—a decision tree method over policies—together with an majority\nvote to determine which base policy to deploy for any given context.\nThrough experiments on synthetic datasets, the paper shows that PS\nindeed outperforms all base policies.\n\n## Strengths\n\n1.  The paper, for the most part, is very nicely written and a joy to\n    read. The authors took great care to make every statement precise,\n    which I greatly appreciate.\n2.  Similarly, the implementation details are given in very nice detail;\n    even the submitted code is very well documented.\n3.  The paper states its research questions / hypotheses very clearly,\n    and gives precise conclusions in support of their hypotheses.\n4.  While not a result about the proposed algorithm, I really\n    appreciated the added touch of RQ1 and the corresponding\n    experimental results, which demonstrates that existing methods do\n    indeed exhibit highly context-dependent performance (and the ranking\n    of these methods fluctuates across context space).\n\n## Weaknesses\n\n1.  The paper says PS constructs a \"diverse library of feasible\n    candidate policies\", but this feels like a stretch to me. The\n    candidate policies are all very simple; one of them is\n    context-independent, and the remaining 4 are actually just two\n    distinct methods instantiated across two function approximators (kNN\n    and random forest).\n2.  The paper also claims to conduct \"extensive computational\n    experiments\" on benchmark CSO problems. Again, this feels like a\n    stretch. Firstly, only two benchmarks are tested. This isn't a huge\n    deal in my opinion because the two benchmarks have quite different\n    properties. But also, these benchmarks are synthetic, and I\n    personally can't appreciate how relevant they are as someone that is\n    not involved in CSO / operations management research. That said,\n    \"extensive computational experiments\" lead me to expect experiments\n    on real datasets.\n3.  Despite the very clear writing and the otherwise nice related work\n    section, it is not actually clear to me what the main innovation of\n    this paper is, given that OPT already exists.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical claim appears to be correct, albeit simple. The\nempirical claims appear to be correct as well, and are supported by\nmany appropriate experimental results.", "audience_yn": "Yes", "audience_explanation": "The CSO problem is quite relevant to the ML research community. The\nsuccess of fixing context-dependent performance via a meta-policy will\nlikely be interesting to the TMLR audience.", "requested_changes": "1.  **Critical**. Currently, the policy library consists of very simple\n    policies based on classical ML techniques. I would be really curious\n    to see if the influence of PS is as major when the library consists\n    of more modern policy instantiations, such as deep neural nets. If\n    this is infeasible, it should at least be discussed. I mention\n    neural networks primarily because these are useful tools for\n    ingesting contexts and implementing complex context-dependent\n    decisions, so perhaps a single policy of this nature can render PS\n    obsolete.\n2.  **Critical**. I believe the benchmarks need further justification\n    here. At the very least, the method of sampling the synthetic data\n    should be discussed / motivated more. Are these synthetic datasets\n    standard? Does the CSO / operations management community understand\n    / care about these benchmarks? Ideally, experiments on a real\n    dataset would be very nice.\n3.  **Critical**. Related to the first point, I believe some important\n    baseline methods are missing here. Many are mentioned in the related\n    work, but I don't think any of those are tested. I think again this\n    is especially relevant since stronger baselines could be\n    incorporated into PS, and I'd like to see that PS is still useful\n    even with stronger base policies.\n4.  **Non-critical**. Please discuss OPT in more detail, particularly to\n    elucidate how your approach is an extension of it.", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "Below are some comments I had while reading the paper; mostly just\nelaborating a little more on some of the requested changes.\n\nIn section 4.1, how does one typically perform the optimization over\n$\\mathcal{Z}$, which is a continuous space? Is there an assumption that\n$c$ is convex in the first argument? Particularly with regard to PPt, I\nwould expect that $c$ should be convex in both arguments.\n\nIt would help if you could explain more what the difference is between\nyour method and OPT. From my understanding, PS is OPT applied directly\nto CSO. What were the challenges involved in this application? Why don't\nthe results from OPT trivially carry over to this setting?\n\nMy hunch is that the major difference comes from the 2-phase approach,\nwhere you build an ensemble of OPT trees in the first phase. But this\nconcept doesn't seem specific to CSO to me. Could you have used the same\ntechnique in the exact setting of the OPT paper to achieve better\nperformance?\n\nIs there any reason why you can't use e.g. neural network policies\nwithin PS? Is it just due to difficulties of realizing feasible\npolicies? If that's the case, are there any meaningful benchmarks you\ncan run where it is easy to represent feasible policies with neural\nnetworks / other function approximators?", "extra_scores": {}, "extra_text": {}}, {"review_id": "xfBmXEVDRu", "reviewer": "Reviewer_jKrC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a method that learns to choose the best Contextual Stochastic Optimization policy from a set of candidate policies, depending on the observed covariates. This is enabled by first training candidate policies on data representative of the entire covariate space, and then learning a policy-labeled partition of the covariate space. Given trained policies, the policy-labeled partition of the covariate space is realized via the Optimal Policy Trees (OPTs) framework. Given a covariate, the an ensemble of OPTs then routes this query to the respective candidate policy.\n\nThe authors show theoretically that the optimal policy is never worse than the best single policy, which is validated empirically on two tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Strengths**\n- The need for the method is clearly motivated (Section 5.4.1).\n- The method is explained in a clear manner, the exposition is easy to follow.\n- The theoretical result (Lemma 1) is sensible and the empirical evaluation supports the findings nicely.\n\n**Weaknesses**\n- The method comes with a significantly larger training cost, as $M \\times K + M$ individual policy and $R \\times K$ policy tree training routines are required. Whether this is prohibitive depends on the training resources required for the individual policies.\n\n**Questions and Comments**\n\n1. Why is the k-fold CV method chosen to train the OPTs? Specifically, why is each OPTs only fit to the respective held out data, instead of the entire data set? Was this tried and overfitting was observed? Why is it necessary to deploy the candidate policies out of sample in the OTP fitting procedure?\n\n2. Section 5.4.1 provides a nice motivation for why a single policy is insufficient. Perhaps a teaser for this, e.g., in form of a figure, could be shown earlier in the paper. \n\n3. Are all relevant candidate policies considered in this work? Is there truly not a single policy that can capture $f_{Y \\mid X}(y \\mid x)$, thought the covariate space? This would be relevant to determine the significance of this work.\n\n4. Final paragraph of Section 4: These [...] $\\rightarrow$ This [...]", "audience_yn": "Yes", "audience_explanation": "Finding the optimal policy in CSO is a relevant problem and the paper provides a useful contribution.", "requested_changes": "See above.", "broader_impact_concerns": "This is fundamental research, there are no immediate ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Y2CPCfxUqT", "reviewer": "Reviewer_ChFf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers contextual stochastic optimization with covariate x, decision variable z, random noise y, and objective function of the form $\\min_{z} E[c(z, y)|x]$. This paper focuses on learning a meta policy to select among a set of candidate policies $\\pi(z|x)$ given the covariate x. This process is called prescribing (candidate policies) and selecting (meta policy). The meta policy is represented as a policy tree. Using the newsvender and shipment planning problems, the authors demonstrate that the meta policy effectively selects the best policy for any covariate space partition when no single candidate policy dominates all partitions.", "claims_and_evidence_yn": "No", "claims_explanation": "The results show that the proposed prescribe-then-select method is effective, which is the mainly claim of the paper. Description of the methods is relatively clear. But some of the design choices seem a bit arbitrary.", "audience_yn": "Yes", "audience_explanation": "Contextual stochastic optimization is a topic of interest to the optimization and control community. The proposed method, which is essentially a two-staged learning of mixture-of-experts policy, is potentially of interest to some.", "requested_changes": "My main comment is that some of the design choices in the paper seem a bit arbitrary.\n* In section 4.3, you referred to the process described in section 4.1 as training policies. But what you are actually training, i.e., the KNN and RF, are predictive models of the form $P(y|x)$ or its simplification. Can you confirm this is the case, i.e., you don't directly train a mapping $\\pi(z|x)$ and at inference time you also solve $\\min_{z}$ rather than querying $\\pi(z|x)$?\n* Why did you choose KNN and RF as your predictive model classes? What about other models like generalized linear models, Gaussian process, or neural networks?\n* Why is it fundamentally not possible (or considered) to train a globally optimal policy? With the amount of work or compute used by your method, you can presumably improve your probabilistic predictive model such that the resulting policy is better globally then the meta policy or any individual policy in your set?\n* Can you explain more clearly the notion of partitioning by the policy tree, segments in the experiments section, and their relationship? I thought the covariate partitioning is supposed to be discovered automatically by the policy tree. Is that how you identified the segments or the identified partitions somehow overlapped with ground truth segments?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)