Decision Making under the Exponential Family: Distributionally Robust Optimisation with Bayesian Ambiguity Sets
Authors: Charita Dellaporta
Patrick O'Hara
Theodoros Damoulas
Venue: ICML 2025 spotlightposter (2025)
Keywords: Distributional Robustness
Bayesian Inference
Stochastic Optimisation
Primary area: probabilistic_methods

Abstract: Decision making under uncertainty is challenging as the data-generating process (DGP) is often unknown. Bayesian inference proceeds by estimating the DGP through posterior beliefs on the model’s parameters. However, minimising the expected risk under these beliefs can lead to suboptimal decisions due to model uncertainty or limited, noisy observations. To address this, we introduce Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) which hedges against model uncertainty by optimising the worst-case risk over a posterior-informed ambiguity set. We provide two such sets, based on the posterior expectation (DRO-BAS(PE)) or the posterior predictive (DRO-BAS(PP)) and prove that both admit, under conditions, strong dual formulations leading to efficient single-stage stochastic programs which are solved with a sample average approximation. For DRO-BAS(PE), this covers all conjugate exponential family members while for DRO-BAS(PP) this is shown under conditions on the predictive's moment generating function. Our DRO-BAS formulations outperform existing Bayesian DRO on the Newsvendor problem and achieve faster solve times with comparable robustness on the Portfolio problem.

Decision Making under the Exponential Family: Distributionally Robust Optimisation with Bayesian Ambiguity Sets
Charita Dellaporta" * 12 Patrick O' O'Hara" *3 Theodoros Damoulas 2.4
Abstract
DASpa
02
BASe
Decision making under uncertainty is challeng- ing as the data-gener process (DGP) is often unknown. Bayesian inference proceeds by esti- mating the DGP through posterior beliefs on the model's parameters. However, minimising the expected risk under these beliefs can lead sub- optimal decisions due to model uncertainty or limited, noisy observations. To address this, we introduce Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) which hedges against model uncertainty by optimising the worst-case risk over posterior-informed am- biguity set. We provide two such sets, based on the posterior expectation (DRO-BASPE) or the posterior predictive (DRO-BASpp) and prove that both admit, under conditions, strong dual formu- lations leading to efficient single-stage stochastic programs which are solved with sample aver- age approximation. For DRO-BASpE, this covers all conjugate exponential family members while for DRO-BASpp this is shown under conditions on the predictive's 's moment generating function. Our DRO-BAS formulations outperform existing Bayesian DRO on the Newsvendor problem and achieve faster solve times with comparable robust- ness on the Portfolio problem.
DD30
P. EGP
10
15.0
17.6
20.0
25.0
27.5
30,0
Figure I. Visualisation of our BAS and BASpp ambiguity sets BDRO (Shapiro et al., 2023) for Normal likelihood with Normal-Gamma posterior. BAS contains distributions with ex- pected (under the posterior KI. divergence from the model of most c. BASpp contains distributions with KL divergence at most from the posterior predictive distribution. BDRO minimises the expected worst-case risk between each KL-based ambiguity set based on posterior samples μt, o² (orange crosses).
from the data-generating process (DGP). Given the observa- tions, parametric model-based inference estimates the DGP with distribution Pe indexed by parameters 0 E θ in pa- rameter space θ S R In a Bayesian framework, the data is combined with prior to obtain posterior beliefs about and the decision maker can now minimise the risk under the Bayesian estimator.
However, the estimator is likely different from the true DGP due model and data uncertainty: the number of observa- tions may be small; the data may be noisy; and the prior or model may be misspecified. Minimising the risk under the Bayesian model inherits any estimation error, and leads to overly optimistic decisions on out-of-sample scenarios even the estimator unbiased: this phenomenon is called the optimiser's curse (Kuhn al., 2019). For example, if the number of observations is small and the prior is overly concentrated, the decision will likely be overly optimistic.
1. Introduction
Decision-makers regularly attempt to optimise an objective under uncertainty with only finite sampling access via in- dependently and identically distributed (i.i.d) observations
"Equal contribution 'Deparmeent Statistics, University Col- lege London, UK 'Department of Statistics, University of Warwick, Coventry, UK 'Department of Computer Science, University of Warwick. Coventry, UK 'The Alan Turing Institute, London, UK. Correspondence Charita Dellaporta <h. wewlaporta@uucl..ee @ucl.ac.uk. Patrick Hara <Patrick. H-O-Hara@ warwick.ac.uke.
To hedge against the uncertainty of the estimated distri- bution, the field of Distributionally Robust Optimisation (DRO) minimises the expected objective function under the worst-case distribution that lies in an ambiguity set.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

DRO with Bayesian Ambiguity Sets
Diserepancy-basse ambiguity sets contain distributions that close to nominal one in the sense of some diserepancy measure such as the Kullback-Leibler (KL) divergence (Hu & Hong, 2013), Wasserstein distance (Kuhn et al., 2019)
instances faster than Bayesian DRO with comparable out-of-sample robustness.
2. Background
or Maximum Mean Discrepancy (Staib & Jegelka, 2019).
For example, model-based methods' (Iyengar et al., 2023;
Let x E A be a decision- decision-making variable that is chosen to minimise a stochastic objective function / A' x E -+ IR, where X' _ PR is the set of feasible decisions and E _ I P is the data space". Let IP E P(E) be the data tatt--enerating process (DGP) where P(E) is the space of Borel distribu- tions over E. We are given n. i.i.d. observations 'D - fi: Pr. Model-based inference considers a family of models Pe & {Po 8 E θ C P(E) where each Pe has probability density function p(g]0) for parameter space O C RR In a Bayesian framework, data D is combined with a prior x (0) to obtain posterior beliefs about 0 through (I)) Under expected-vale -aauue risk, Bayesian Risk Optimisation (Wu et al., 2018 8) solves stochastic optimisation problem:
Michel et al., 2021; 2022) consider a family of parametric models and create discrepancy-
ambiguity sets centred on the fitted model. However, these works do not capture un- certainty about the parameters which can lead to a nominal distribution far away from the DGP when the data is limited. The principled framework for capturing such uncertainty is Bayesian inference. Existing DRO methodologies like Bayesian DRO (BDRO) introduced by Shapiro et al. (2023) that are informed by parameter posterior belief', do not cor- respond a worst-case risk objective and hence do not give rise single, interpretable worst-case risk distribution (see Figure 1).
We introduce DRO with Bayesian Ambiguity Sets (DRO- BAS): a framework for robust decision mabaiinnnmmiing under un- certainty based on posterior-informee ambiguity sets. Our contributions are:
(BRO)
min EE- [Eg=Polla(EE]]. xE
(1)
where la(() - (xx,E) is the objective function. How- ever, decision-makers sometimes seek worst-case protection. The closest work to ours, using parametric Bayesian infer- ence to inform the optimisation problem, is Bayesian DRO (BDRO) by Shapiro et al. (2023). BDRO takes an expected worst-case approach under the posterior distribution. More specifically, let (Po) - (Q E P(E) dxi. (QI S }} be the KL discrepancy- distribution Po. where dxu, the KL--divergence and param- eter e E [0, oc) controls the size of the ambiguity set. Under the expected value of the posterior, BDRO solves:
1.
We define Bayesian Ambiguity Sets (BAS) which lever- age posterior beliefs about the parameters of interest. We provide two distinct formulations of BAS: one con- taining distributions with small KL divergence from the posterior predictive distribution (DRO-BASpp) and an- other one including those with small expected KL diver- gence from the candidate model (DRO-BASpE) BASPE). Sim- ilarly to discrepancy- baasdd DRO, the decision maker solves a single worst-case risk minimisation problem.
ambiguity set centered on
2.
We show that PR attains a dual formulation which is an efficient single-stage stochastic program. Additionally, for models within the conjugate exponen- tial family, we demonstrate that the dual program of DRO-BASPE is also single-stage and can be solved ex- actly when linear objective function and a Gaussian likelihood are used. When the objective function is non-linear but convex, the duals can be solved with sample average approximation (SAA) (also known as Monte Carlo). Finally, we provide the worst-case risk distribution form as well as finite-sample results on the optimal tolerance level.
(BDRO)
min B ZEX
sup (PB
Bewalfa(E)]
(2)
where EB-M(D & Is Y(0)II D) dl denotes the expectation of random variable Y : θ + R with respect to (T(0) D). decision maker often interested in protecting against and quantifying the worst-case risk, but BDRO does not correspond to worst-case risk analysis. Moreover, the BDRO dual problem is a two-stage stochastic problem that involves double expectation over the posterior and likelihood. To get good approximation of the dual problem using SAA, a large number of samples are required, which increases the solving time of the dual problem.
3. On the Newsvendor problem, we show that DRO- BAS Pareto dominates existing Bayesian DRO for- mulations (Shapiro et al., 2023) when evaluating the out-of-sample mean-variance trade-off of the objective function when the number of SAA samples is small. On the real-world Portfolio problem, DRO-BAS solves
Gupta (2019) introduced Near-Optimal Ambiguity Sets, based on Bayesian guarantees. This method aims to find the smallest convex ambiguity set that satisfies a Bayesian Posterior Feasibility guarantee. The authors also provide
FFrr convenience, we assume the decision variable E X' and the random variable e E both have dimension D. but our work is not restricted to this case.
'For discussion of empirical DRO vs model-based DRO methods, see Appendix E.3.
2

---

DRO with Bayesian Ambiguity Sets
an upper bound on the posterior value-at-risk, constructing the ambiguity set according to the framework of Bertsimas et al. (2018). In this approach, the ambiguity set is chosen by searching over space of potential sets and selecting the one that meets a predefined guarantee based on the posterior distribution. In contrast to BDRO, the ambiguity set is con- fined to members of the model family, and posterior beliefs are not used to inform the anbiguity set itself, but rather to guide the search for feasible sets.
risk minimisation, controlled by e. Expected-value risk in (1) is equivalent to minimising the expected risk under the posterior predictive Pn E P(E) that has probability density function defined by:
Pn(E5 (D) 2 1 p(E" 10)d1((
|
D).
(3)
First we propose Bayesian Ambiguity Sets with the posterior predictive (BASpp):
Alternative DRO formulations often incorporate nonpara- metric Bayesian methods. Wang et al. (2023) suggested using a nonparametric prior over the DGP P* to induce distributional uncertainty. This nonparametric prior is cho- sen to be Dirichlet Process (DP) prior with prior centring measure P(E). The authors suggest inducing distri- butional robustness by minimising the worst-case loss over an ambiguity set for F. Bariletto & Ho (2024) also address DGP uncertainty by combining DP-based posterior beliefs about the DGP with a theision framework that preferences, arising from
(. {Q E P(E) d cc
(4)
where Q P(E) is distribution in the ambiguity set and dRL : P(E) x P(E) - R is the KL divergence. The ambiguity set is informed by the posterior predictive IP,:: it consists of all probability measures Q E P(3) which are absolutely continuous with respect to P,, i.e. Q < nn and have Kl.-divergence to P, less than or equal to e. Thus, e dictates the desired amount of risk in the decision. For a given decision x, the worst-case risk is
models smooth ambiguity- the economic decision theory literature (Cerreia- Vioglio al., 2011). 1). This framework was recently generalised by Bariletto al. (2024) to handle heterogeneous data sources through hierarchical DPs. Unlike the previously mentioned BDRO, these methods are not well-suited for situations where the decision-maker has parametric model and seeks to incorporate parameter posterior beliefs.
sup ES-Q [f)(E)] (B.( (F..(F.)
(5)
The posterior II(0) D)) targets the KL minimiser between the model family and IP (Walker, 2013), hence it is natural to choose the KL divergence in (4). This is because oo the posterior collapses to 0 _ arg mingee dxa (P* Po) and the posterior predictive is equal to Pn 1 Thus, for a well-specified model family, which occurs when P Pg- E Pe. we have that the limit of infinite observations, (F,,) a KL-based ambiguity set with P the nominal distribution. The goal of the decision maker is to choose that minimises the worst-case risk, yielding the problem:
+
3. Bayesian Ambiguity Sets
The goal of this work is formulate a family of optimisation problems that, based on parameter posterior beliefs, produce decisions with worst-case risk protection. We introduce Bayesian Ambiguity Sets (BAS) which achieve this through two ambiguity set constructions. We first propose a BAS based on the posterior predictive (BASpp) which forms a distributionally robust counterpart the BRO objective in (1). To overcome some of its drawbacks, we further propose BAS based on posterior expectation (BASpF). The difference between the two lies in the position of the posterior expectation in the constraint. The two ambiguity sets, for tolerance level e, consider probability measures Q E P(E) with density function q(E) such that:
(DRO-BASPP)
min (B.(
(6)
Ambiguity sets of the form B. based on the KL- divergence have been previously suggested in a non- Bayesian DRO context by Hu & Hong (2013), who studied the optimisation of this problem for a general nominal dis- tribution P. For the dual of the worst-case risk Rs. (r) (fx) to be well-defined, the following property of the nominal distribution P is required.
Property 3.1 (Finite moment--eneratie function of fa).
0))TT()) 1D) se. dkl. (((E). / p(E) P(E1O))TT(
Let P. E P(E). For all x E X, there exists / > 0 such that:
(BASp))
Eqar [exp(tf=({))] < co.
(7)
[ dRi (a(E).(()) ()) (TT( D) S.
(BASPE)
If If, satisfies this property, we have the following result. Proposition 3.2. Assume Pn satisfies Property 3.1 for all D C E". Then, for any e 0, the worst-case risk in (5) is:
3.1. BAS via Posterior Predictive (BASpp)
Given posterior II and risk tolerance level e E [0, oc) we extend the Bayesian risk objective in (1) to a worst-case
falel
RB.(P.)(fa) inf nc + y In E, (p.(())))) 720
6
(8)
3

---

DRO with Bayesian Ambiguity Sets
The proof can be found in Appendix B.I. This result offers strong duality and the resulting optimisation problem is single-stage stochastic program. Unlike BDRO, the problem in (6) not only provides a worst-case approach but also requires sampling from only a single expectation when the posterior predictive is available.
tradition, instead of BDRO's expected worst-case formula- tion (2). The goal is minimise the worst-case risk:
(DRO-BASPE)
min #
(m
(12)
We derive an upper bound for the worst-case risk in (11):
Assuming that Property 3.1 holds in Proposition 3.2 requires the noment-generatie function (MGF) of (E") under the posterior predictive ,,, to be finite. This is a standard as- sumption made in KL-based DRO methods (Hu & Hong, 2013; Shapiro et al., 2023) ensuring that the dual objective in (8) is well-defined. However, this assumption can be violated when e.g. the model is a Normal distribution with conjugate Normal-Gamma prior, leading to a Student-t posterior predictive (see e.g. Murphy, 2023) which has an infinite MGF. A similar problem occurs when the Expo- nential distribution with conjugate Gamma prior is used. Thus, the use of such an ambiguity set can be limiting in the model and objective function choices. A way to overcome this limitation is to approximate the primal problem in (6) using the ambiguity set (. based on a finite sample approximation P,, of the posterior predictive where:
Proposition 3.3. Assume Pg satisfies Property 3.1 for all
O. Then the worst-case risk satisfies:
[x( e 7
(fu) S inf ye + En y In Ep 720
(13)
Proof sketch. We introduce a Lagrangian variable Y 2 O for the constraint Ra (Q, Po)] S 6 from (10) and write the dual in terms of the convex conjugate (En (-IP)) (fz) of the expected KL-divergence. The upper bound follows by Jensen's 's inequality because the conjugate is convex function. The result follows from the convex conjugate of the KL-divergence (Bayraksan & Love, 2015; Shapiro et al., 2023). See Appendix B.2 for a detailed and full proof.
While Proposition 3.3 provides a weak duality upper bound applicable to general Bayesian models, its primary purpose is to motivate the need for more tractable formulations. In particular, this bound highlights the challenges of working with expected d-divergences, like the KL. To address this, Section 3.3 focuses on exponential family models, where we can move beyond the weak duality bound and derive strong duality result in Theorem 3.6. This analysis requires the closed-form expression of the expected KL divergence (Lemma 3.5), which plays a central role in obtaining the convex conjugate and deriving novel results on tolerance levels, worst-case distributions, and tractable optimisation.
M
iid SI:M ~ PRI
PA.
P
1 M
(9)
Although for finite M, the dual formulation can be applied with PP,, M, the resulting optimisation problem is always SAA and does not solve the original primal in (6) to optimality. Hence, we offer an alternative BAS formulation based on a posterior expectation.
3.2. BAS via Posterior Expectation (BASpE)
Consider Bayesian ambiguity sets with the Posterior Expec- tation (BASpg) defined as:
3.3. DRO-BASPE for the Exponential Family
A.III 2 [Q P:: (d.
(10)
The convex conjugate (En (ydga (-)) of the expected KL-divergence in the proof of Proposition 3.3 is not easy to obtain for a general Bayesian model, but if we can find an exact expression for this convex conjugate for specific mod- els, then the worst-case risk (11) can be computed exactly. In this section, we derive such an expression for exponential family models with conjugate priors. When the likelihood distribution is a member of the exponential family, there exists a conjugate prior that also belongs to the exponential family (see e.g. Diaconis & Ylvisaker, 1979). Exponential families include many of the most widely used probability distributions, making them versatile for modelling across diverse settings. Additionally, they are highly appealing due to their extensive theoretical foundation and range of valuable properties (see Diaconis & Ylvisaker, 1979; Gutiérrez- -Pññ et al., 1997; Gutierrez-Pane 1997, for a de- tailed overview.). We use the conjugate exponential family
The ambiguity set considers all probability measures Q e P(E), which are absolutely continuous with respect Po (Q & Pa)) for all 0 E θ and have at most expected (with respect to II) KI. divergence to Pp. The shape of our ambiguity set is posterior-driven: we emphasise this point by taking the posterior II as an argument to the ambiguity set (TT). This is contrary to standard discrepancy- baased ambiguity sets (-) as in (4) which are centred on a fixed nominal distribution. We will later prove that BASpF can be reduced to ambiguity set of the form (-) for exponential family models, allowing for efficient computation. For fixed decision I, the worst-case risk is:
R.AM (Jz) A sup Be- [S))
(11)
(I)
Similarly to the BASpp carst-case risk in (5), our formula- tion (11)& is still a worst-case approach, keeping with DRO
4

---

DRO with Bayesian Ambiguity Sets
and the convexity of the log-partition function derive the strong dual of the worst-case risk problem in (11). The following definition is from Murphy (2023).
Theorem 3.6. Suppose the conditions of Lemma 3.5 hold and 2 Gmin (n) as in ( (18). Furthermore, assume plE fi) satisfies Property 3.1 for all n E Q. Then the worst-case risk (T. in (11) is equal to:
Definition 3,4 (Exponential family with conjugate prior). Let p(D 7) be an exponential family likelihood with nat- ural parameter 77 E Q S R² scaling constant h + R. Log-partition function : - I such that _ {1 E RK A(r) < oo), and sufficient statistic s: = + R with
inf 7(E GG(F,)) + + Ppcin 720
e
(19)
Observe that this dual formulation (19) closely mirrors the dual of the problem in (8), albeit with different tolerance level and expectation law. However, in this case, assuming Property 3.1 requires (C) to have a finite MGF with re- spect to a member of the model family, namely p(5 ii). Contrary to the posterior predictive case, this is satisfied for exponential family models with many objective functions such as the linear and piecewise-
likelihood function:
p(D n) (D) exp (n sDD) nA(1))
where hDD = II M(;) and
sDD) = - sEE). We
consider the prior distribution:
TT T 17 m(())
{ [. D(T,
exp
(14)
ones we use in our experiments. The same model assumption is imposed by BDRO (Shapiro et al., 2023).
where v V are prior hyperparameters and 2(F, is the normalising constant. The conjugate posterior 11(77 7,
Using this theorem, we can derive an immediate connection between BASpE (10) and KL-based ambiguity sets of the form (.) such as BASpp:
has the same form as (14) with parameters 7 = 7 + sDD (DD and v + n.
Corollary 3.7. Let é é (-G(F Then in the exponen- tial tial family case, (I)) = BB.((a))
We start with Lemma before presenting the main result. The proof can be found in Appendix B.3.
Lemma 3.5. Let P(E 1) be an exponential family likeli- hood function with natural parameter n and T(n T T,,), I( R, D) conjugate prior-posterics pair as in Definition 3.4. Let i) A E BuCri,,,,1 and define G: T + R as:
Hence, in the exponential family case, the two BAS formula- tions are different as they correspond to KL-based ambiguity sets with distinct nominal distributions. Additional insight on their relationship are discussed in Appendix D.
It is worth noting that although the dual formulations of the DRO-BAS formulations and the PDRO method of Iyengar et al. (2023) with the KL divergence are both model-based, they are fundamentally different. Our ambiguity sets are posteriori informed, integrating prior beliefs and data evi- dence. Consequently, the resulting nominal distribution (IP,. for DRO-BASpp and F4 for information from our posterior beliefs, including their un- certainty quantification, unlike the point estimator approach of PDRO. This allows to propagate uncertainty from the posterior beliefs about the parameters to the ambiguity set.
G(F,D) A (A - A(f))
for hyperparameter space T. If for almost all v) E Ω the partial derivatives 0 and V of the posterior p.d.f. II:
Ω x T
+ I2z0 exist for all (D, F) E T then:
contains all the
i i= -= (-I)
(15)
à ( (- D) (f) 20
G(F,D) D)
(16)
and the expected KL-divergence under the posterior is:
3.4. Tolerance Level Selection
E 1) [dkz (ddz((Q((P))) (Q[[I,)) dxz (Q. + D). (17)
To guarantee that the DRO-BAS objectives upper-bound the expected risk under the DGP, the decision-maker aims to choose large enough so that IP* is contained in the ambiguity sets. For BASpE with the exponential family, Lemma 3.5 yields a closed-form expression for the optimal radius (n)( in the exponential family case, by noting that:
It is straightforward to establish the minimum tolerance level emin required to obtain a non-empty BASpF. Since
the KL divergence is non non-negative, under the condition of Lemma 3.5, for any Q E P(E):
En [dkL (P,) = de.dd22.p +(F,D)
(pg(n) Ey [dKL (PP" [P,))] = dki. (P" Pa) + GGF
2 G 2) 2 Emin (n).
(18)
(20)
We are now ready to present our main result, the proof of which can be found in Appendix B.4.
If the model is well-specified, and hence F and P, be- long to the same exponential family, is simple to obtain
5

---

DRO with Bayesian Ambiguity Sets
(n) based on the prior, posterior and true parameters (see Appendix A for examples). In practice, since the true param- eters are unknown, we can empirically approximate (n)(n) from data D. For any 2 (n) 2 (n), we have that IP* E Ac (II, thus the worst-case risk upper bounds the true risk under the DGP:
We can solve the infimum over y in (19) to obtain
(fa)= x+ 2 C(a,))) o'
Importantly, the resulting minimisation dual for DRO- BASpE over decision I is closed form and we do not need to sample from either the posterior or the likelihood. Thus, the number of variables in the dual is O(D) (the dimension of 2) and the dual input size is ( (the size of Σ)
RE (v((E) & sup (E).
(e))
(21)
For BASpp, the inclusion of the DGP is achieved for: <pp(n) A (IP", P.).
Notably, BASpp does not admit such closed-form due to the absence of a finite MGF under the Student-t distribution, which in this case corresponds to the posterior predictive law. As discussed in Section 3.1, this is limitation of DRO- BASpp which leads to a violation of Property 3.1 and does not allow an exact dual formulation in Proposition 3.2 for many exponential family cases.
(22)
A possible approach would be approximate this using the observed dataset and samples (ET.... (xx the posterior predictive via dKL( n Σi= og., M Ei= dc;). However, this can become computationally expensive in higher dimensions if the KL is not available in closed form. Alternativlly, we can use the approximation of (ne) which is available in closed-form in the well-specified case, since (p)(n) S (n). This follows by Jensen's 's inequality since the KI. divergence convex both arguments:
~ Pn, from M
3.6. Worst-case BAS Distribution
The understanding of the structure of the proposed BAS sets is facilitated through the worst-case distributions maximis- ing the objectives in (5) and (11). Indeed, by following the argument in Hu & Hong (2013) (Eq. 8), it can be shown that, for some 7 (ar) > 0 minimising the dual formulation in (8), the worst-case distribution maximising the DRO-BASPP objective in (6) has probability density equal to:
(pp(n) = dkL((")) En [Po) & En (dka [[Fo] = (n).
follows that for any 2 (n)( the risk under the DGP is upper-bounded by (5).
p' p5 ( = Pn((E)) exp
[=(E") (a)))
3.5. Computation
E'EE.
When the objective function (x(() is convex in x, the dual of DRO-BASpp and DRO-BASPE is a convex optimisation problem that can be solved using off thel solvers. Both DRO-BAS duals are single-stage stochastic programs which can be solved in practice using SAA. In contrast, the BDRO dual is two-stage stochastic program where samples must be taken from both the posterior and likelihood Shapiro et al. (2023). The size of the dual programs- in terms of the num- ber of variables and size of the input data- - is summarised for DRO-BAS and BDRO in Table 1. To give an exam- ple for PRO-BASPp. the size of the input of is () because this is the size of the samples f1,..., EMrr Pn; the number of variables for DRO-BASPr is O(D + Mye) because there are 1) decision variables for each dimension of I, Mpp epigraphical variables for each sample E ~ P.,. and one Lagrangian variable.
Ep...
exp
f.(EE
This means that the distribution attaining the worst-case risk is proportional to the posterior predictive and has the same support. Similarly, it can be shown that, in the exponential family case, the distribution that attains the worst-case risk according to the DRO-BASPE objective in (11) is:
p'(E', = p(5")
E'EE
where F(E) = h(g) exp (f)((E) - A(()) + +z((")/n(a)))
and y* (z) minimises the dual problem in (19).
Note that for BDRO, single worst-case distribution does not exist because it considers expected worst-case ap- proach (see (2), Figure 1) rather than a worst-case approach, as advocated by DRO methods. If we sample 0, ~ 1II from the posterior for BDRO, then one can obtain a worst-case distribution for each Po, via the argument in Eq. (8) of Hu Hong (2013). However, notice that general, the minimiser of an expected objective is not the same as the average of the minimisers of the individual objectives. Hence, even looking at the posterior mean or mode of the worst-case minimisers of the inner worst-case objectives in (2) would not necessarily give us a single distribution p that yields a worst-case risk of the form Ef (z. E)] corresponding to the risk minimised by BDRO.
Next, we consider the special case of DRO-BASPE when the objective is linear function ( <" xx the likelihood is multivariate Gaussian (V(E ju, E), and the posterior is Normal-inverse-Wishae Wishart. Posterior parameters j,, Σ can be derived from (15) for ń (see Appendix A.3). By the MGF of a Gaussian, we have
5Tx:]
xT Ex
E (Nrela,5))
exp
exp
+
7
7
272
6

---

DRO with Bayesian Ambiguity Sets
Table The number variables and the size of the input for two cases the dual program terms of the dimension (of and æ) and the number of likelihood samples M,, posterior samples Me. and posterior predictive samples Mpp. Variables include decision, L Lagrangian, and epigraphical. We note the results DRO-BASPE refer to exponential family case.
Property 3.1 holds for (f. P)
Linear f: P. N(,, Input size
Algorithm
P
Number of variables
Input size
Number of variables
DRO-BASPE
Po
O(D + Me)
(M
O(D) O(D + Mpp O(D + Ma)
O(D*)
DRO-BAS
Pn
O(D + Mre)
(M
((Mr O(MaD²)
BDRO
Pa
O(D + Ma)
O(MoM:
M=25 with Exponential DGP
M=100 with Exponential DGP
M=900 with Exponential DGP
180
BASpr
Gat-dfeaappl meam, m/e)
DRO-BASpP
160-
DDRO
140-
1.0
120
10
0.00D
6000
8000
10000
6000
8000
10000
6000
8000
10000
.1 25 with 5D Normal DGP
M- --100 with 5D Normal DGP
M M-900 with 5D Normal DGP
150
C
140
Ont-d--sumple
130
120
...6..
DRO-DASPE
DRO-BASpr BDRO
2000
4000
6O00
8000
2000
4000
6000
8000
2000
4000
8000
M 25 with Truncated Normal DGP
M=100 with Truncated Normal DGP M=900 with Truncated Normal DGP
DRO-BAS,x. DRO-BASPP
O-s-of-accppee mie)
BDRO
1..5
1.0e
1.0
34-
0.07
033
099
32
500
350
600
700
500
550
800
650
700
500
530
600
650
700
Out-of-sumple vullamos, m(f)
Out-of-sample varbonce,
Out-of-eumple vartamos, ofe)
Figure The sut-of-sample mean-variance tradeoff for exponential DGP (top row), SD multivariate normal DGP (middle row), and truncated normal DGP (bottom row). We vary when the total number of SAA samples is 25 (left), 100 (middle). and 900 (right) and plot the markers in bold. For illustration purposes, blurred markers/lines show other smaller values M: for example, the right column shows M 900 in bold and M 25, 100 in blurred. Each marker corresponds to single value of e, some of which are labelled (e.g. e = 0.1, 0.5, 3.0). Filler markers indicate the given e lies on the Pareto frontier.
4. Experiments
our analysis on Bayesian formulations of DRO, however, we discuss empirical DRO and provide additional experi- ments in Appendix E.3. For DRO-BASPE. if < <min (n), then the problem in Theorem 3.6 unbounded and we
We evaluate DRO-BAS against BDRO on two classical prob- lems: the Newsvendor and the Portfolio problems. We focus
7

---

DRO with Bayesian Ambiguity Sets
ignore this configuration. For DRO-BASpp use fi- nite sample approximation Pu,Ms of the posterior predic- tive Pn, as in (9). For given E, we calculate the out-of- sample (OOS) mean m(c) and variance v(c) of the ob- jective function f across J different datasets. Further de- tails and experiments can be found in Appendix E. The code to reproduce the experiments is available at https: / //githubs com wwwp/Patrickooaaaaacoaa dro-
0.007
G+ Ie
DRO- PA
DRO-BASPP
DDRO
0.005
Cor-af-wompple mosts
G 0.00
0.005
100005
4.1. DRO-BAS on the Newsvendor Problem
0.001
The goal of the Newsvendor problem is to choose an inven- tory level æ E 20 of D perishable products with unknown customer demand ş E IR P that minimises the cost function [E( h max 0, x - {) + b max(0, E - x); where h, are the holding cost and backorder cost per product unit respectively, Following Shapiro al. (2023), we set h = 3 and b- 8. We run DRO-BASpL. DRO- and BDRO across 24 different values of e ranging from 0.001 to 1. For DRO-BASPE and DRO-BASPP. we approximate the expec- tations over P i)) and (= (D) respectively with M samples; for BDRO, we approximate the double expecta- tion with Mo posterior samples and Mg likelihood samples for each posterior sample such that Mg x MM M. We evaluate the methods on two well-specified settings (an ponential and multivariate normal DGP) and one univariate misspecified setting (truncated normal DGP/normal model). For random seed, j = I,..., 500, we sample n 20 train- ing observations from Pr, then sample T = 50 test points.
0.00
0.003
G+
1.0
0.0005
0.0010
0.0015 oat-ll- variance
0.0020
0.0025
Figure 3. Mcan-variance tradeoff of the Portfolio objective across all OOS time windows. Markers are filled similarly to Figure
4.2. DRO-BAS on the Real-world Portfolio Problem
The goal of the Portfolio Problem is to choose the weight- ing of a portfolio of stocks that maximises the return. Our dataset and experimental setup follows Bruni et al. (2016) who provide weekly return data for the DowJones index on D = 28 stocks across 1363 weeks. Given n = 52 weeks of training data D.,. we fit model to the return & with Normal likelihood and conjugate Normal-inverse seaed-iiverreWishaarr prior. The test dataset Dr contains the T = 12 weeks of returns im- mediately following the training dataset. We minimise the linear objective function fz (E) = - T x (cquivalent to max- imising the return) such that æ; 2 for all i = 1,..., , D and Σ Ty 1. The J 109 train and test datasets are con- structed using sliding time window (see Appendix F.2).
Figure 2 shows the OOS mean- variance tradeoff across all three DGPs. The middle row shows that, in the well-specified multivariate setting, both instantiations of DRO-BAS dominate BDRO in the sense that DRO-BAS forms a Pareto front for the OOS mean-variance trade- off of the objective function f. That is, for any 61. let (e) and (E)RO(E1) be the OOS mean and variance respectively of BDRO; then there exists 62 for DRO-BAS with OOS mean (B) and variance (B2)(E2) such that MB))((2) < (1) and (B))(c2) < (G) On the top row of Figure 2 (well-specified Exponential DGP) PRO-BAS dominated BDRO for EBDRO 2 0.5 when M = 25, 100, whilst all methods lie on the same Pareto front otherwise. In the misspecified case (Truncated Normal DGP with Normal likelihood), DRO-BASpE Pareto domi- nates BDRO and DRO-BASpp when M- 25, 100 while all methods perform similarly when M = 900. Finally,
Since the objective is linear and the likelihood is Normal, the dual objective for DRO-BASPE is available in closed form (see Section 3.5). For DRO-BASpp. we sample from the posterior predictive Pr, where we set M = 3600 (we found DRO-BASSp needs a large M for good OOS perfor- mance). For BDRO, the dual program Appendix C) and we need only sample M = 900 covariance matrices from an inverse-Wishart distribution.
be simplified (see
Figure 3 shows that DRO-BASPE and BDRO have similar OOS mean-variance tradeoff, whilst DRO-BASpp is Pareto dominated by the other two methods. Figure 7 demonstrates that the cumulative return of DRO-BASpE and BDRO are broadly comparable and the returns are generally larger than DRO-BASpp The discrepancy from DRO-BASpE and BDRO to DRO-BASpp is likely because the dual programs of DRO-BASPE and BDRO can be simplified using the MGF of the Normal likelihood. The simplified dual for DRO- BASpE and BDRO both yield a closed-fome expression for the mean, whilst DRO-BASA also obtains the covariance
for fixed M, the solve times for DRO-BAS and BDRO are broadly comparable (see Table 3 in Appendix E). How- ever, BDRO requires more samples M than DRO-BAS for good oft-ol-sample performance, likely because BDRO must evaluate a double expectation over the posterior and likelihood, contrary to the single expectation of DRO-BAS.
8

---

DRO with Bayesian Ambiguity Scts
Moreover, pRO-BASpp allows for the use of models beyond the exponential family, as long closed-form posterior predictive is available. On the other hand, the dual formu- lation of DRO-BASPE takes advantage of the properties of the exponential family models since the underlying nom- inal distribution is also a member. This is in contrast to BASpp which can lead to general forms of posterior predic- tive distributions, outside the exponential family, leading to assumption violations in the derivation of the dual prob- lem (Section 3.1). This advantage of BASpE further offers more efficient optimisation objective for the commonly used (see e.g. the Portfolio problem of Section 4.2) linear cost functions with a Normal likelihood (see Section 3.5).
Table 2. Average solve times in seconds (with associated standard deviation) and sample times in seconds x 10 on the Portfolio Problem. N.A. means "not applicable" due to no sampling.
ALGORITHM
SOLVE TIME
SAMPLE (x10*)
DRO-BASPE
0.01 (0.01)
N.A.
DRO-BASPP
1.34 (0.22)
3.48 (0.91)
BDRO
4.47 (0.62)
44.45 (3.11)
in closed form but BDRO must sample the covariance from the Inverse-Wishart (see Appendix C). In comparison, DRO- BASpp samples from the posterior predictive without any closed-form expression for the mean or covariance.
Future work will aim to generalise DRO-BASPE to models beyond the exponential family as it is possible that this will lead to differently shaped ambiguity sets, beyond KL-based ambiguity sets with a fixed nominal distribution. These might exhibit benefits in higher dimensions where the size of KL-based ambiguity sets can grow very fast with the tol- crance level, leading to over-conservative decisions. More- over, it would be valuable to theoretically examine the effect of the Monte Carlo sampling size on the out-of-sample cost, similar our empirical analysis in Section 4. Such an analy- sis has already been performed for the Wasserstein distance and the x² >ddivergencee by Iyengar et al. (2023) but has not yet been extended to the KL-divergence.
Table 2 shows that the solve and sample time of DRO-BAS is orders of magnitude faster than BDRO on the Portfo- lio problem. The average solve time of DRO-BASPE and PRO-BASpp is 0.012 and 0.213 seconds respectively, whilst BDRO takes 3.381 seconds. Similarly, the average time taken for BDRO to sample posterior covariance matrices takes 100 times more than for DRO-BASpE to calculate the parameters J and Σ. and 28 times longer than DRO-BASpp to sample from a Student-t distribution. Given that Fig- ure 3 shows comparable OOS mean-variane tradcoff and cumulative returns between DRO-BASPE and BDRO, we conclude that DRO-BASPE is the more suitable method for achieving low OOS mean-variance in this example due to its elimination of sampling and faster solve time.
Extending the BAS framework to general d-divergences, which have been previously used in DRO (e.g. Bayraksan Love, 2015; Duchi & Namkoong, 2019), represents another promising direction for future work. In particular, analysing DRO-BASPp would require deriving the dual formulation using convex conjugate results for ø-divergences, though obtaining results on the tolerance level and worst-case dis- tribution may prove more difficult due to the properties of the posterior predictive and the divergence function. The DRO-BASPe setting is even more challenging, as involves characterising the expected d-divergence under the poste- rior in closed form, which is critical for deriving its convex conjugate and establishing strong duality. Such derivations, as seen with the KL divergence, are key to obtaining re- sults on tolerance level selection, tractable objectives in specific settings, and the form of the worst-case distribution. We believe addressing these challenges could lead to new, principled BAS formulations based on broader class of divergences.
5. Discussion
We proposed an approach to decision-making under un- certainty through two DRO objectives based posterior- informed Bayesian ambiguity sets. By employing exponen- tial family models, the resulting problems benefit from a dual formulation which allows for the optimisation of worst- case risk criteria. This property, combined with generality across the entire exponential family, makes the approach particularly attractive for decision-making problems with statistical models. Since our dual is single-stage stochas- tic program rather than two-stage program, our frame- work offers computational advantages, making it feasible for larger-scale applications.
Understanding the growth of the ambiguity set volume as a function of the tolerance level would aid the interpre- tation of the robustness properties of these sets and is left for future work. Moreover, although both DRO-BAS for- mulations showcased improved empirical performance and computational advantages compared to existing methodolo- gies, a theoretical analysis to identify conditions, beyond what we discussed in this work, favouring one formulation over the other would be highly beneficial.
Finally, while DRO-BAS achieves worst-case robustness, it remains inherently tied to the Bayesian posterior. This dependence makes it vulnerable to severe model misspeci- fication. Future work will investigate alternative posterior formulations that are more robust to model misspecification. to enhance the BAS construction and produce more robust posterior
rosterior-informed informed ambiguity sets.
9

---

DRO with Bayesian Ambiguity Scts
Acknowledgments
Cerreia-Vioglio, S., Maccheroni, F., Marinacci, M., and Montrucchio, L. Uncertainty averse preferences. Journal of Economic Theory, 146(4):1275-1330) 2011.
CD acknowledges support from EPSRC grant [EP/T51794X/1] as part of the Warwick CDT in Mathematics and Statistics, EPSRC grant [EP/Y022300/]] and the G-research travel grant. PO and TD acknowledge support from UKRI Turing Al acceleration Fellowship [EP/V02678X/1] and a Turing Impact Award from the Alan Turing Institute. For the purpose of open access, the authors have applied Creative Commons Attribution (CC-BY) Y) license any Author Accepted Manuscript version arising from this submission.
Diaconis, P. and Ylvisaker, D. Conjugate priors for expo- nential families. The Annals of statistics, pp. 269-281, 1979. Diamond, S. and Boyd, S. CVXPY: A Python-embeddeed modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1-5, 2016.
Duchi, J. and Namkoong, H. Variance-based regularization with convex objectives. Journal of Machine Learning Research, 20(68): 2019.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must specifically highlighted here.
Endres, D., Pabst, K., Eckert, A.-L., and Schween, R. A collection of identities for variational inference with exponential-famil models. Technical report, Philipps- Universität Marburg, 2022.
References
Gao, R. and Kleywegt, A. Distributionally robust stochastic optimization with Wasserstein distance. Mathematics of Operations Research, 48(2):603 655, 2023.
Agrawal, A., Verschueren, R., Diamond, S., and Boyd, S. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
Gotoh, J., Kim, M. J., and Lim, A. E. B. Calibration distributionally robust empirical optimization mod- els. Operations Research, 19(55:1630-16) 2021. doi: 10. 10.1287/0pre.2220.2041 URL httpt::t///oot org/ 10. 10.1287//oi 00122///prr... 2w2.:ppp.................
Agrawal, R. and Horel, T. Optimal bounds between divergences and integral probability metrics. Journal of Machine Learning Research, 22(128):1-59, 2021.
Bariletto, N. and Ho, N. Bayesian nonparametries meets data-driven robust optimization, arXiv preprint arXiv:2401. 15771, 2024.
Gupta, V. Near-optimal bayesian ambiguity sets for distri- butionally robust optimization. Management Science, 65 (9):4242-4260, 2019.
Bariletto, N., Nguyen, K., and Ho, N. Borrowing strength in distributionally robust optimization via hierarchical dirichlet processes. arXiv preprint 1331,0 2024.
Gutierrez-Pena, E. Moments for the canonical parameter an exponential family under a conjugate distribution. Biometrika, 84(3)777-772, 1997.
Bayraksan, G. and Love, K. Data-driven stochastic programming using phi-divergences. In The operations research revolution, pp. 1-19. INFORMS, 2015.
Gutiérrez-Peña, E., Smith, A., Bernardo, J. M., Consonni, G., Veronese, P., George, E., Girón, F., Martínez, M., Letac, G., and Morris, C. N. Exponential and bayesian conjugate families: review and extensions. Test, 6:1-90, 1997.
Bertsimas, D., Gupta, V., and Kallus, N. Data-driven robust optimization, Mathematical Programming, 167:235-292, 2018.
Hu, Z. and Hong, L. J. Kullback Leibler divergence con- strained distributionally robust optimization. Available at Optimization Online, 1(2):9, 2013.
Brown, L. D. Fundamentals of statistical exponential families with applications in statistical decision the- ory. Lecture Notes- MonsorMonggg Series, 9:i-279, 1986. ISSN 07492170. URL http:://///......... /ttp:://www... www.wjsto..... jsttr.oorgrr.................... stable/435555.
Iyengar, G., Lam, H., and Wang, T. Hedging against complexity: Distributionally robust optimization with parametric approximation. In International Conference on Artificial Intelligence and Statistics, pp. 9976-10011. PMLR, 2023.
Bruni, R., Cesarone, F., Scozzari, A., and Tardella, F. Real- world datasets for portfolio selection and solutions of some stochastic dominance portfolio models. Data in brief, 8:858-88 862, 2016.
Kuhn, D.. Esfahani, P. M., Nguyen, V. A., and Chafieezadeh-Ahddee S. Wasserstein distributionally
10

---

DRO with Bayesian Ambiguity Sets
robust optimization: Theory and applications in ma- chine learning. In Operations Research & Manage- ment Science in the Age of Analytics, chapter 6, pp. 130-166. Informs, 2019. doi: 10.1287/educ:2019 0198. URL https: http://pbboollooooo ne. informs. 12w.11277/// orr//aollAbbb/1...LL///eeeee 0019.099..
Markowitz, H. Portfolio selection.
The Journal of doi: https://doi.or//10 10.
Finance, 7(1):77-91, 1952.
URL https: //onlinelibbaay.wwiie.... wileyy hatn//dii/bb///// 1111/3. 6261. 1952.1952..00 tb01525.
Michel, P., Hashimoto, T., and Neubig, G. Modeling the sec- ond player in distributionally robust optimization. arXiv preprint ArXiv:2103, 10282, 2021.
Michel, P., Hashimoto, T., and Neubig, G. Distributionally robust models with parametric likelihood ratios. arXiv preprint arXiv:2204.063400 2022.
Murphy, K. Probabilistic machine learning: Advanced topics. MIT press, 2023.
Petersen, K. B. and Pedersen, M. S. The matrix cookbook, October 2008. URL hitp: /www2. imm. atu. /k/ pubáb/p. patccb/ppppp?200 phpppppp?2 2274. 4. Version 20081110.
Shapiro, A., Dentcheva, D., and Ruszczynski, A. Lectures on stochastic programming: modeling and theory. SIAM, 2021.
Shapiro, A., Zhou, E., and Lin, Y. Bayesian distributionally robust optimization. SIAM Journal on Optimization, 33 13044 2023.
Staib, M. and Jegelka, S. Distributionally robust optimiza- tion and generalization in kernel methods. Advances in Neural Information Processing Systems, 32, 2019.
Walker, S. G. Bayesian inference with misspecified models. Journal of statistical planning and inference, 143(10):
1621-1633, 2013.
Wang. S., Wang, H., and Honorio, J. Learning against distri- butional uncertainty: On the trade-off between robustness and specificity. arXiv preprint arXiv:2301.1335 2023.
Wu, D., Zhu, H., and Zhou, E. A Bayesian risk approach to driven stochastic optimization: Formulations and asymptotics. SIAM Journal Optimization, 28(2):1588- 1612, 2018.

---

DRO with Bayesian Ambiguity Sets
Supplementary Material
The Supplementary Material is organised follows: Appendix A provides details for all the conjugate exponential family examples used in the experiments, while Appendix B contains the proofs of all mathematical results appearing in the main text. Additional computational details are discussed in Appendix C and further details on the relationship between DRO-BASPE and DRO-BASpr are provided in Appendix D. Finally, Appendix E provides additional experimental details and results.
A. Special Cases of Exponential Family Models
We derive the values of G(F, ()) and i appearing in Theorem 3.6 for different likelihoods and conjugate prior/posterior pairs used in the experiments.
A.1. Gaussian Model with Unknown Mean and Variance
This section considers a Bayesian model that estimates the unknown mean and variance of a univariate Gaussian distribution. We consider the natural parametrisation of the Normal distribution with unknown mean and unknown variance, written in its exponential family form and derive the conjugate prior and posterior parameters well and G(F, 2) in Theorem 3.6.
Consider the Gaussian likelihood for mean μ and precision A. Then the likelihood of dataset D is:
pD (, = (2x) exp
(E: - 11)
(23)
The natural parameters and seg-pattiionn function of the normal distribution are
17
m
A(n)
./.
1 In([-20 22p2)
!/2
4122
2
respectively. Before proceeding, we revisit the gamma and digamma functions.
Definition A.1. The gamma function I' N + R and digamma function N + R are
[(=) (= 1)!
(x))
In [(=).
dz
We are now ready prove the following Corollary, which is special case of Theorem 3.6.
Corollary A.2. When the likelihood is Gaussian distribution with unknown mean p and precision À as in (23), and the conjugate prior and posterior are normal- normal-gamma ggamma distributions with parameters (ji, Kc, Ot, B) and (ū, R, respectively. D) = (In (a)
then Theorem 3.6 holds with f) = [(R+1)
R(R+1)
and G(F
2,3
4
Proof. The prior distribution which leads to a conjugate posterior is the Normal-Gamma distribution with parameters hyper-parameters 24, k. a,
π (p, A K, = .GG( A ",
= -N(H M (KA)) G)(A (
- exp
-BX+
- (p -1)²
ZNG
2
where the normalisation constant ZNG given by:
- _-12
ZNG
[(() 2T.
12

---

DRO with Bayesian Ambiguity Sets
Following the form of the conjugate prior in Definition 3.4 write the prior for 1j as:
x(n D)
exp
Kj kipe
f11
K
(0(-2122
"
1
+ 2/3
712
4
TT 1l EA(n))
exp
M(T,V)
where we set ā a x+1 and it follows that:
.2
T1
KM
2 K. Z(F,D)-
T2
TI
V
T(
+
Jv 2T.
(24)
T2
Fijs
+ 2/
2v
2
By conjugacy it follows that the posterior distribution for with parameters statistic s(D) (E=z Ei, Ei is:
F s(D) and i. = + n, for sufficient
ITi D.F.D)=
f n fn
exp
2(F
We derive G(F,D) D) for the univariate Normal with Normal-Gamma prior using Lemma 3.5. First,
)
i In lF
+ 1
T2
+ 1
In
+
Inl
00
02
2
2
2
2
2
T1
1
In
T2
+
D D+1
22
1
1
D+1
2
2v
27
2
72
11
2
27
2
2
T2
TI
V
+ 1)T
1
1
1/
2
2y
20 20
2
27
2
(R + 1)R 2-2
1
1
R + 1
3
2 1/
2
2k
2
2R(23R)
1
(R + 1)
K +
In
1
2k
2
4B
Furthermore, we have:
1 +1
T2
С+1 1
ń a - ((F
In
Im 2
ln
2
21
2
(D +1)
DID +1)
F2D
71
2(720 - 71)
p0R + 1)
R+1
2
49
Plugging this into the definition the log- eag-paritionn function we obtain:
22
1/
+1)
K x+1
A(ñ)
In( 22pz)
In
48
2,3
13

---

DRO with Bayesian Ambiguity Sets
and from Lemma 3.5 it follows that:
G(F,
0
(-1m2(7.D)
-A(i)
07
2
2
In
3
(R + 1)
R ++1 1
M
(R
+
1)
In
+
1
2
e
2x
2
2
~ 2,3
43
48
1
2R
Furthermore, for our implementation we can reparametrise o back to standard parameterization to obtain:
ip μ.
== -212 i
3 X
Tolerance level e In the well-specified case, where we assume that 1sM - P& for some 0 E O, it is easy to obtain the required size of the ambiguity set exactly. Let (n² x 1 and Ip & N(A A 11 Using Corollary A.2 we obtain:
PE (n) = N.A.NGGUAARRRR....
[dKi IN(",,N(E 14, (,,>-1))]
dxL
P'
H4,
B
1
In ā x(aa)
2
R
x B
x-1 1
1
1
In
+
+ In v())
2 3
2
2
x=1 + (pr
1 1 14
In
x+
((a)
R
Posterior predictive distribution needed for the implementation DRO-BASpp. Under the setting of Corollary A.2, the posterior predictive distribution the following Student-t distribution
For completeness, we remind the reader of the posterior predictive distribution, which
3( + 1)
P. tza
El
aK
A.2. Exponential Likelihood with Conjugate Gamma Prior
We now consider the natural parametrisation of the Exponential distribution, written in its exponential family form and re-derive the conjugate prior and posterior parameters as well as r) and G(F,)) 2) in Theorem 3.6.
Consider the Gaussian likelihood dataset for rate parameter O:
p(D A) exp exp
Σ & (nnnnnd
mn
The natural parameter and log-partition function of the Exponential distribution are:
" ーー A(n) In(nn)
respectively, We prove the following Corollary.
FFor a full derivation see example https:: //////wwww /twp///wwccc cs. ubc htt.://.wt....mmmmppppp////////s .pff
14

---

DRO with Bayesian Ambiguity Sets
Corollary A.3. When the likelihood is an exponential distribution with parameter X and gamma prior and posterior pair is used with parameters (a, 3) and () respectively, then Theorem 3.6 holds with řj = and G(F,D) = l ((a)-
Proof. The prior distribution which leads to conjugate posterior the Gamma distribution with hyper- arper-parameters a, B:
a(> a) = Za
cxp ( (ā In
where the normalisation constant Zo = l(a). By following the form of the conjugate prior in Definition 3.4 can write the prior of 1y as:
x (1 D. ~(
exp (n- - (--AA())
where 7 = 13,0=
= a- and Z(F, 1) =T T( + 1). By conjugacy, it follows that the posterior distribution for 1 with parameters 7 + sD) and n. for sufficient statistic s(D) -E Eús is:
[T D,7,D)= 7(7,0)
exp (fn- RA(T))
We ready derive i and G(F,D) D) for this example. Firstly,
0 0 In Z(F
00 (00 + In - In + 1))
92
In 7 v1 +1)
= In (()).
Furthermore, we have
n A 0
In ((7 2)) = 2+1
07
3
and
A(î) = - -n(--)) -
Using Lemma 3.5 obtain:
0 ( (-In 00
(
G(F D)
A(i) 1n ((a) In
In â (()).
Furthermore, for our implementation, we can reparametrise ό to the standard parameterization and obtain:
= n á
3
15

---

DRO with Bayesian Ambiguity Sets
Tolerance level e In the well-specified case, where we assume that IP* - IP for some 0* E O it is easy to obtain the required size of the ambiguity exactly. Let 0* be the true rate parameter, i.e. p. - Exp(O*). Using Corollary A.3 obtain:
6PE (pg(n) EE
[dxI. (IxI,(P" Exp(O)]
Ca(M)).B)
dKI.
1p-
Exp
+ (())) In():)
3
=In(0") - In
α
+
á
+ (/)) - In(a).
B
30
Posterior predictive distribution For completeness, remind the reader of the posterior predictive distribution, which is needed for the implementation of DRO-BASpp Under the setting of Corollary A.3, the posterior predictive distribution is the following Lomax (Parcto type II) distribution:
P Lomax (a,, B).
A.3. Multivariate Normal Likelihood with Normal- Prior
Now assume the random variable ş is multivariate such that E RDD where D is the dimension of the random variable. The definitions of the likelihood, prior, and posterior in terms of the natural parameters can be found in Chapter 3,4.4.3 of Murphy (2023). The likelihood is multivariate normal distribution N(5 μ. Σ) with f E R and E E S7 (where set S is the space D x D positive semi-definite matrices):
P(E μ. 2 = (2x) - exp exp (- 4) T -12)
(25)
The natural parameters of the multivariate normal distribution are
Σ
77
7/2
vec
Σ
(26)
where vec is function that converts matrix into a vector For convenience, abuse notation and treat 7/2 and T2 as matrices instead of using their vectorised form.
We prove the following Corollary.
Corollary A.4. When the likelihood is multivariate Gaussian distribution with unknown mean μ E B. and covariance matrix Σ E &Q as in (25), and the conjugate prior and posterior are normal-Invrsee -nverrs-WWWharrr distributions with parameters (p,, K, L, W) and (p., R, 2, ()) respectively, then Theorem 3.6 holds with
(R - D 2) f2
î =
-T
[K D -2
and
D In(k D 2)
-1
7n- D 2)
GG, - 2
- In
V
+ 2
(R
p.
Proof. The conjugate prior to the likelihood is the Normal-invers--Wihheee (NIW) denoted by NIW [,
The hyperparameters of the NIW have the following interpretation (Murphy, 2023): ju RD is the prior mean for and
'Strictly speaking, since the upper diagonal of the covariance matrix is the same as the lower diagonal, our vectorised exponential family representation is not minimal. It is, however, more convenient and easier to work with.
16

---

DRO with Bayesian Ambiguity Sets
R quantifies how strongly we believe in this prior; matrix v/ E 84+ is (proportional to) the prior mean of >, and 2 dictates how strongly we believe in this prior. We can re-write the prior π - i.R. in the form of an exponential family x17 F, b): starting from the definition of the normal-inverse-Wishaee distribution, we have
p4, 1 1 x
NIW p. H. V A N 1
IW
E v
Σ - 3
exp
Ri
(1
- ;0)TE>
(p ((p-xk) 11)
x |Σ| 1
exp
1 Tr
(-ー!))
1)
ZNIW
2
2
exp
2 1
+ Kf 2 1 /r
2 1t
2p
(1 (++ 2) |n/2ا|
IF
11222
ZNTW
2
exp
4 122
mn + kJ
já +
Kj
1/2 +
2 (e
D + 2) 1 + Tr
7NIW
Kj
711
1
exp
v
In| 1122
4 " 122 1
ZNIW
+ 1
1/2
T
= x |.. T - --mA(()))
exp
Z(F
In the above, we recognise that A(r) = - In| 2122 - 11 1/2 2/2111 mm is the log partition function of the multivariate normal likelihood; we set v ī D 2: and we define
7
T T
(27)
T2
vec
V
(similarly to 72, we abuse notation and treat 72 as a matrix). The normalisation constant ZNIW for the NIW distribution may be written in terms of F and as:
NIW A 2ID/2TDD (1/2) (2T k.) 1/2
2 -22)-D
1 wー D (2T/i)
1 _T -TIT
(7-
D-2)/2
(28)
T2
2
where DD is the multivariate gamma function with dimension D. The posterior distribution is
TTT
fn (1-BA(m))
11 D)- 11(m (T,)- Z(F
exp
The posterior update is 7 = +D) and N vे +n where s s(D) = (E Eu Σn T have the same form, the normalisation constant ( 2) is defined in the same way as (28).
T.
Since the prior and posterior
17

---

DRO with Bayesian Ambiguity Sets
Our goal is to derive the function G(F,D) !)) from Lemma 3.5. First, we derive
@ 85
In ((.
We have
(- in (-
D
(0- - D - 2)
D
(v - D - 2) - D
In 2T
80
2
2
2
D
1
1 TT
T
+
In 1 +
-0 D - 2) In
T2
= T171
2
2
D 1n2
1/ V- D 2
D
1 -
VD
+
+
In
ワ2 -
2
2
2
21
2
+ +2
( D - 2)
TF (
T1T1
2 TT
D
3 )イ)
1
D
R k- D 2
D
n²
In
+
+
In
p0n kpppp
+
v
up/e
2
2
2
2k
2
k
-1
2
R D
2
T T
T
T
T
+
Tr
kii + 1/-
uif1
pill
2
k
h²
D 2
D - 2
D
1
D
2
T
1
DD
+
In
V
+
p
1/
j..
2
2
2
The first equality above holds by definition of Z(7, D) and by log identities. The second equality holds by chain rule for differentiation and the identity [(m||A]) Tr(A-10A) from (43) of Petersen & Pedersen (2008). In the third equality, we
-1 ToT T
have substituted 7, for μi, K. Mr In the last equality, we have simplified and used the fact that Tr(w Next, we need to evaluate A(i). We note that by Lemma 3.5, f is
pp ) p v p
n) A ET P,0) [n] = -V (- In D))
- ;
20
D (D- -D 2D22)n22-1 InID -InIp
(5- D 2)
D
In 2
D
1 -000- -D 22)
1
T
In 2
P2
2
1 D 2) F₂ - 1
-
, ;
Before we obtain the partial derivative with respect to 71 need the following lemma.
Lemma A.5. Let x E X and A E Raxe, Then Be
0 In IA xx. = 22(A xx T)-xx. -x.
18

---

DRO with Bayesian Ambiguity Sets
Proof.
① dr
Tr -(A xx." xxxT)-1 (axx)) T
In A xx
1
dx
(60)
(-(A - xx " (x(OOr) + (8x)) [)
(06)
Tr (-(A xxx ) 3-'x(0x) + Tr (-(A (1xx-x''] 1 (8xxxx²)
((v)
Tr (-(A - xx ) x(0x) 1) + Tr (-(A xx
))
2 ((A xx. T) xx(( ))
( 22 - xx T) ) x.
where (i) follows from the Jacobi's formula (see e.g. Petersen & Pedersen, 2008, Equation 43) which says that Oln [X|
Tr(X-10X) (ii) follows from the chain rule, (iii) follows from the property of trace that says Tr B) Tr(A)+ +TrA))Trr(B)) (r(B), (iv) follows from the fact that =(0x)" is symmetric and finally (v) follows from the property Tr(Aww) = v Au.
We now ready derive the partial derivative of In Z(F, D) with respect to 7, using the above Lemma:
1 G DD---2)0-
2) In F2 911 )
1
1 D 2) (D-
1 -
071 2
D
where we used the fact that 0 |X| Tr(X -xx)x.)xx). Using the fact that for a square, non-singular matrix X: ax X-T (Petersen & Pedersen, 2008, Equation 49), the partial derivative with respect to 72 is:
1ax).
In XX
0 1 1 D
1 a 2) F2 | = 10 - - 2) (Tx
TTT
072
1/
Hence,
1 D 2) + 10 D 2) (f )
(k D 2) -1
P
==
-T
-T
+R D 2)
From the definition of A(r) for the multivariate normal distribution, we have
1 T
A(7)
In
22pz2
11 1/2" im
4"
T TT
In
(R - D 2)
(- - D D 2) T
v
p
2
4
In
(R D - 2)
+ (R D 2) v pt
1
2
D
1
D 2)μ -
In(k
D 2)
In
V
+
(R
μ.
2
2
2
19

---

DRO with Bayesian Ambiguity Sets
We have all the ingredients for caleulating D):
G(F.D)
_
a
(- In
Z
2)
A(î)
02
D
R D 2
D
1
+ R
2
T
-1
In2
D
+
h
V
p
2
2
2
2h
2
2
+
D
In(k- - D 2)
+
In
VV
-1
(R
D -
-22)
T
v
-1
p
2
2
D
D
D
D
In 2-
UD
+
In(K- D D-2 2)
2
2
2k
2
where we used the fact that |V -1 normal likelihood from the parameter r). Using equation (26), we have
[1 -1 Finally, for our implementation, we need to get the parameters i. E of the
=
Σ = (-2f12)
v
D -2
(29)
A = Ein v..
Tolerance level e In the well-specified case, where we assume that 1 - P2 for some 8* E O, it is easy to obtain the required size of the ambiguity set exactly. Let 4" & (4", E*) E RD x sh and Io & N(k", E*). Further let j and as in (29). Using Corollary A.4 we obtain:
(n) E
[dKL(P' NNEE H, Ej]]
= dKL
P
I|N
(a, )
D
In
2DD
1
R-D-2
+
D
+
D
In
D -2
2
2
2
2
2
1
15T5-11 5p= - -ii + Tr (T-((5-15) (5-155)) }
log
D + (n- - (1)
2
Σ
D
1
R
D 2
D D
1n2
+
In D
2). 2)
2
2
2h. 2
Posterior predictive distribution For completeness, we remind the reader of the posterior predictive distribution, which needed for the implementation DRO-BASpp. Under the setting of Corollary A.4, the posterior predictive distribution the following multivariate Student-t distribution (see e.g. Murphy, 2023, p. 96):
@ + 1)
P.. = T R-D+1
R(R D +1)
B. Proofs of Theoretical Results
B.1. Proof of Proposition 3.2
Before proving the required upper bound, we recall the definition of the KI divergence and its convex conjugate. Definition B.1 (KL-divergencee) Let μ, e P(E) and assume fa is absolutely continuous with respect to v (μ < v). The
KL divergence of μ with respect to v is defined as:
(1|l1) In (u(EE)
u(dE).
v(dE)
20

---

DRO with Bayesian Ambiguity Sets
Lemma B.2 (Conjugate of the KL-divergence), Let P(=) be non-negative and finite. The convex conjugate d&a (-ll)) of dxx(-Ilv) is
dz (x = In ( ((<p((t)dd))
Proof.
See Proposition 28 and Example 7 in Agrawal & Horel (2021).
We are now ready prove Proposition 3.2.
Proof. We introduce Lagrangian variable 2 0 for the KL constraint on the left-hand side of (6) as follows:
sup (ulF((
Ee(Ja]
EQ
(
inf
sup (EP(Z)
Ee[sx] 7< - -dxl
720
(4))
inf
7e + (7dKL (-I1Pn))) (fz)
220
fz
inf ne + y In Ep
exp
720
Equality (i) holds by strong duality since P,, is a strictly feasible point to the primal constraint. Equality (ii) holds by the definition of the conjugate function. Finally, equality (iii) holds by Lemma B.2 and the fact that for y 2 and function e (v) = (y/7).
B.2. Proof of Proposition 3.3
Proof. The result follows from a standard Lagrangian duality argument and an application of Jensen's 's inequality. More specifically, we introduce a Lagrangian variable y 2 for the expected ball constraint on the left-hand side of (13) as follows:
(3) S inf
sup Eqlfa
sup QEP(E)
Bolfal + ye - yEn (Q[[P)]
Q:Eo-n(da (Q(Po))
720
(9)
inf
7E + sup DEP(E)
Eolfal En ( [P)]
720
inf
y + E [t (P))) (fx)
720
(K0)
inf
ne + En [[TdxL (P(l))))) (fe)
720
( inf
fn
7e + En IIn
exp
20
Inequality (i) holds by weak duality. Equality (ii) holds by linearity of expectation and simple rearrangement. Equality (ii) holds by the definition of the conjugate function. Inequality (iv) holds by Jensen's inequality (LE(-))* < E[(-)]
because the conjugate is convex function. Equality (v) holds by Lemma B.2 and the fact that for y 2 0 and function ø (77) (y) = 70" (1/7).
B.3. Proof of Lemma 3.5
We first give closed-form expression for the expected log-likelihood under the posterior. We start by deriving an expression for the posterior mean, i.e. ń E (I)
[v). We follow a similar technique as the one presented in Endres et al. (2022).
We start by using the fact that the posterior density integrates and take partial derivatives with respect to T. Note that
21

---

DRO with Bayesian Ambiguity Sets
by the differentiability assumptions and the fact that the posterior p.d.f. is Lebeeggee-ittegrable, we can interchange the differentiability and integral operations:
i 1- (-DA(m)
exp
dn =1
D(
exp (7 --D((n)) dn
=0
2(F,D)
2(7.D)+ + 1
2(F / |oop
dn 0 n (n-BAm)
{ T
Z(F
7,
-V;
2(F,D)
?(,,D)
1
-の= = -
Z(F,D)
Z(F,5)
+ n = - v7 (-Im2(7.0))
(30)
We now have an expression for i) and we can hence write:
E
p- In p(5| pmp
n)] = E (-p()1..D) (mn((E) + + s({) - A(r)]
= =lnh(E) + +E p [n]] s(5) - E p [A(n]
= =Inh(g) + +fi (()) E /nnntn/7,0) [
=In/(g) + (1) s(E) A(î) + A(() E
r [A(n)]
(A()]
In n)) A(f) E p, (A(m)
(31)
where in the last equality we used the exponential family form of the likelihood (see Definition 3.4). To obtain an expression for the expected log-partition function we follow similar argument: we start from the posterior p.d.f. and differentiate with respect & under the differentiability and integrability assumptions:
F (+'n=
TT (n-BA()) dn =1
exp
0
1
-"n (-DA()) dn) =0
exp
00
n D(F
0
/ exp
T -RA(n)
/ exp
dn - 0 7 {n 11 - (2(m)) ?n-bA(m)
dn +
02
2(F,D) Sa dv
2(7.D) + 1
/
A())exp (7'n- n (7n--BA()) dn =0
7(F,D)
7(7,D) 8
/, [A(n)] (A()))))))((()1-))))))))))))))) = a
1
E
Z(F,D)
07
Z(F.D)
p. [A(n)] [A(m) 0
(-In(Z(FDD)
(32)
07
By substituting Equation (32) in Equation (31) we obtain closed-form expectation for the expected likelikel as
22

---

DRO with Bayesian Ambiguity Sets
follows:
E
In m)] = In p(5 I ()) + A(f) - E (mp((FF,)(A(m))
= 1 | ()) 0
(-1m(((7.D) - A(())
00
- In p(E | n) - G)F
(33)
is straightforward to show that G(F, ?) is non-negative:
(- (-In(Z(F.D)) A(())
G(F D)
d2
E 1- (A())] A (E (p
0
where the equality follows from Equation (32) and the inequality follows from Jensen's inequality since the log-partition function of an exponential family likelihood is convex function (Brown, 1986, Theorem 1.13). Finally, we can proceed with the main result. Starting from the left-hand side, we have
E1 [dKL (Q(lP)) (#)
V 9()) a(E) (p( | 1))
(_2(t)
de
[/- 9(E) In (q(E)) ((5) In (p(E m)) de
((E) In (9(E) - ((() E () [In (p(E n))] ((E) In (a(E)) - -(()) (Inp(E) n) G(F,
(i0)
- 2) dg
a(LE) q(E)
P(E 7) dE + / ((5) -GF de
(Q)) + G(F,D).
where (i) is by the definition of the KL-divergence; (ii) follows by log properties; (iii) holds by linearity of expectation; (iv)
holds by (33); (v) holds by rearrangement and properties of log and (vi) holds by the definition of the KL-divergence.
B.4. Proof of Theorem 3.6
We begin by restating the Lagrangian dual from the proof of Equation (13) for the exponential family case, but with the added claim that strong duality holds between the primal and dual problems:
sup Qu (Q).))
Ee[fa]
inf sup QEP(E)
Ea[ss] + - - 7ln (Q
(34)
720
Proof. The conditions under which our claim of strong duality holds will be proved later. Next, we substitute the right-hand side of equation (vi) above into the dual problem in (34):
sup
Be~e(fZ(E)
T (2 P,)]ss
inf ne sup / fa(E)((E) -y (der (Q ll P(E ()) + G(F, D) 720 (EP(E)
= inf
7E - GG D) + (7 dkL (: p(E ())) (f)(
720
inf 7(E G(7, 2) 7 In Eppp(eh) exp (z(()
720
23

---

DRO with Bayesian Ambiguity Sets
where the second and third equality holds by the definition of the conjugate of the KL-divergence and by Lemma B.2. Finally, it remains to argue that strong duality holds. First, note that the primal problem is concave optimisation problem with respect to distribution Q. Second, when e > G(T, D),, then distribution + = p(5 f) is strictly feasible point to the primal constraint because
(dki Po)] 0 G(F,D).
C. Bayesian DRO Reformulation
We begin by recalling the two-stage stochastic optimisation proposed by Shapiro et al. (2023) for Bayesian DRO. Let X' be the set of feasible decisions. The decision x E X' is the called the here-and-now decision: the decision is made before the realisation of the uncertainty 0. The first stage objective function is (xx 0). The first-stage decision solves:
min RA-11 [H (2, 0)]
(35)
ZEX
The Lagrangian decision the second-stage decision: the decision is made after the realisation the uncertainty 0. The second stage problem is:
fc(
}}
H 0) = inf AS0
de A Aln Bg-p(E10)
exp
(36)
A
We emphasise the two-stage nature of this program: the variable a function of and can only optimised after the realisation of 0.
Shapiro (2023) solve the two-stage stochastic program with a nested SAA. The outer SAA samples 0 ( Ev) where i No. The inner SAA samples Ej p(E10t) where j I,...,Ng: For given 0, and given decision E X, the authors solve:
Σ exp ((x,,5s)
Hxx G(r inf ASO
de AIn Aln
(37)
Ne E, 3p((()))
The drawback of this method is the necessity to iterate over each decision in the set of feasible set X. For example, in the one mie-dimensional Newsvendor Problem, the authors set X' {or E R : 0 s I 50) and then iterate over X' with grid search:
1. For each x = 0, 1,...,50,, 50, evaluate (37). Let be the minimum attained.
2. For each æ from x= x= + increments of 0.01, evaluate (37) and return the minimum attained.
When the set X large, this strategy is clearly not scalable. Instead, we jointly minimise the decision variable and Lagrangian variables using an out-of-the wo-ooff--bbooot box commercial solver. Our algorithm relies upon the observation that the samples 0g ~ NN form a discrete, empirical distribution. Let X; be the Lagrangian variable that depends upon sample 0.. From Section 2.3.1 of Shapiro et al. (2021), we can switch the expectation (35) with the minimisation in (37) to obtain:
No
N₂
{0) X
inf S.A....AAN
Aie + A In
Σ
exp
AN,, > 0, E
(38)
No
N i=1
where we have denoted the jth sample from 0(E by We now have a joint minimisation over the decision r and Lagrangian variables X; where i = 1.... Ng. However, the second term in (38) cannot yet given directly to a solver. We make two transformations to (38) to get the minimisation problem into standard form that can be recognised by a solver. The first transformation uses the log identity in a In The second transformation introduces No N
24

---

DRO with Bayesian Ambiguity Sets
epigraphical variables biy and constraints f(x ( S big for I,..., No and j I..... Ng. Re-writing (38) with these two transformations, we have:
No
Nr
1
Lij
inf x ANo
Are Aln
Ne + A
Σ
exp
No
Xi
1=1
==1
(39)
f(r, (1)) A1,..., ANo > 0, x X
taj
i= 1, No. 1, .....,N
To conclude, observe that the third term is perspective function of the form X; LSB(4:/X)) where 4 (4a,... , (ine) ) and LSE is the Log- Sum- function. As A - O, we must take care in our implementation to avoid numerical instability. Fortunately, we can make use of Lemma C.1 in our optimisation algorithm, which says as X; -> 0, then
A TLS((/AA)) max(ta, (4Ng)
Lemma C.I. Let LSE(Y). .B))
) be the Log-Sum-Eay Exp function defined by
LSE Vn) 2 In exp(y)).
The perspective of the Log-Sum-Exy function converges to the max function, that is:
lim LSE
1/1
yn )
max(y),........... Mn).
Proof. Let y" max.y)..... yv) and C the cardinality of the set maximum elements:
C = Kyv = = 1,....nll. nt|.
The Log-Sum- Seg--xxppppp function has the following property:
LSE (y1.... (JI,...,yn))
Un) = y" + LSE(1) y",..., ..... y²).
Using this property, have
lim LSE 1/1 7
Yn
1/1 y
3²
lim In 700
exp
3* + lim +
In = y*
To see why the second equality holds, notice that - 3" < 0 for all i = 1,...,
n. Thus, we have
lim exp ("i (wーv² 0,
if ys 3¹ < <0.
7-+0
1, if y: - y = O,
where we assume 0/0 for the case y: 3"
Linear objective and Normal likelihood. Recall from Section 3.5 that when the objective function fz({) = 5 and the likelihood is a Normal distribution (p, E). then we can use the MGF solve the optimisation problem over À in (37) to obtain
H H
V Σr.
Assuming the posterior distribution Normal-inverse- weaiireeeee----rrrtttt. (NIW) distribution, the BDRO dual from (35) can be re-written as
min E,,E~NIW (x 0)] min EA~NIW [A x] + 2e EX-NIW x
(40)
=EX
The expectation over μ above can be computed in closed form, whilst the expectation over can be approximated by Monte Carlo sampling.
25

---

DRO with Bayesian Ambiguity Sets
D. Further Details the Connection Between DRO-BAS Formulations
In this section, we provide the proof of Corollary 3.7 and make some additional comments on the relationship between the two formulations. We begin with Corollary 3.7.
Proof of Corollary 3.7. By (17), BASpF is equivalent to a KL-based discrepancy set with nominal distribution P., and posterior-adjuster constant G(F,D) D). To see this, consider some and Q E P(E) such that Q < P. for any 1 E Q Then by (17):
Expa
dky
dkL (( c (Q, S - G
This means that the DRO-BASPE problem in (12), restricted to the exponential family case, is equivalent to the following optimisation problem:
min
sup (g
QEB 7,,,0 (P4)
This observation is specific the exponential family and should not expect this hold for general model families and posteriors.
Another interesting connection between the two formulations regards cases where one constitutes the subset of the other. One such case is the following:
Lemma D.1. For general model family Pe and fixed 0, BASpE is subset of BASpp, i.e. A, S (PP.).
Hence, for fixed e, if the DGP is contained in both ambiguity sets, then the DRO-BASpe risk upper bounded by the DRO-BASpp risk resulting in DRO-BASpp decisions being overly conservative.
Proof of Lemma D.1. This result follows from Jensen's 's inequality by noting that the KL is convex on the second argument and hence for any Q E A,(II) we have
dRL. (Q) dkl. (Q] EQl[ S Egwn (QI $
and hence Q E (P.).
In the exponential family case, this holds for all <ig. with epg defined in (20). In particular, for all <pe it follows that:
Eg-e- [fz(E) I
sup Q. (dea se
Eg-c(fz(E)] &
sup Codaa (Q) En (r,)s
Ee-olfa(EE]
where the first inequality follows from (21) and the second inequality follows from Lemma D. However, worth noting that this not guaranteed for values smaller than epg so no definite conclusions can be made for the performance the methods for different values of e or for any S <pp- Exploring this further would be an important aspect of future work. We further discuss computational differences of the objectives in the next section and empirically investigate both methods in Section 4.
E. Supplementary Experiments & Details
In this section, we provide additional details about the setup of experiments, the metrics used to evaluate the algorithms, and some supplementary experiments.
26

---

DRO with Bayesian Ambiguity Sets
M- with Normal DGP
M=100 with Normal DGP
-1900 with Normal DGP
46
DRO-BASPE DRO-BASPP
Out-of-summle mett, ml()
44-
BDRO
42-
1.0
1.0
40-
0.00
20.5
0.0 0.6
38
0.1
10001
0:00+
0.00
750
800
850
900
750
800
850
900
750
800
850
900
Out-of-sumple variance, v(e)
Out-of-sumple variance, v(c)
Out-of-sumple variance, v(c)
M 25 with Contaminated Exp DGP
==100 with Contaminated Exp DGP M M=900 with Contaminated Exp DGP
DRO-BASPE DRO-BASPP
0.2
Out-of-samplo meun, (n(()
160
..---. BDRO
0/2
0.2
140
0.1
0.1
0.000
0.03
0000
0.00
3500
4000
4500
5000
5500
6000
3500
4000
4500
5000
5500
6000
3500
4000
4500
5000
5500
6000
Out-of-sample variance, v(()
Out-of-sample variancr, r(())
Out-of variance, n(()
Figure The out-of-sample mean-vare variance tradeoff normal DGP (top row) and Contaminated Exponential DGP (bottom row). We vary for DRO-BASPE, DRO-BASer, and BDRO when the total number of samples from the model is 25 (left), 100 (middle), and 900 (right) and plot the markers bold. For illustration purposes, blurred markers/lines show the other smaller values M; for example, the right column shows M 900 in bold and M 25, 100 in blurred. Each marker corresponds to single value of some of which are labelled (c.g. e = 0.1, 3.5,3.0). 3.0), Filler markers indicate the given e lies on the Pareto fronticr of the model-based algorithms. If the marker is unfilled, this indicates the given is Pareto dominated by at least one other point.
Implementation. We implemented the dual problems for DRO-BAS (Theorem 3.6) and BDRO (Shapiro et al., 2023) in Python 3.11 using CVXPY version 1.5.2 (Diamond & Boyd, 2016; Agrawal al., 2018) and the MOSEK solver version 10. 1.28. Three machines were used to run experiments, each with a Dual Intel Xeon E5-2643 v3 @ 3.4 Ghz (12 cores/24 threads total) with 128GB RAM (5 GB per thread/ 10GB per core). The SLURRM workload manager was used to schedule jobs. Each job was allocated single core and 10GB of RAM such that upto 12 jobs can be ran in parallel on a single machine.
Out-of-sample mean and variance. For a given c, we calculate the total out-of-sample mean m(c) and variance u(c) of the cost as:
m(c) =
f(x0)..n4))) Entc),
JT
J
T
((oll))stdd (00)-m(() (nte)) m(e))²
v(c) = JT -
where a!! is the obtained solution on training dataset D( The variance formula across all I' observations and all J test datasets. For random seed j = n....,5000 we sample 20 training observations
above is simply the total variance
D(()) from the DGP, then sample T 50 test points. This contrasts with the formula proposed in Gotoh et al. (2021) for
27

---

DRO with Bayesian Ambiguity Sets
Solve time (seconds)
Total sample time (seconds x 10 x10-1)
DGP
M
PRO-BASPE
DRO-BASPP
BDRO
DRO-BASPE
DRO-BASPP
BDRO
Exp
25
0.024 (0.003)
0.024 (0.003)
0.068 (0.012)
0.097 (0.007)
0.117 (0.009)
0.360 (0.018)
100
0.036 (0.003)
0.037 (0.003)
0.148 (0.020)
0.099 (0.007)
0.122 (0.015)
0.614 (0.045)
900
0.422 (0.030)
0.428 (0.032)
0.724 (0.040)
0.114 (0.010)
0.155 (0.013)
1.611 (0.117)
Tr. N
25
0.024 (0.003)
0.024 (0.003)
0.067 (0.012)
0.070 (0.006)
0.138 (0.008)
0.121 (0.009)
100
0.035 (0.003)
0.035 (0.003)
0.145 (0.020)
0.072 (0.006)
0.142 (0.009)
0.156 (0.013)
900
0.398 (0.020)
0.406 (0.021)
0.683 (0.033)
0.090 (0.008)
0.185 (0.012)
0.292 (0.025)
5D A
25
0.031 (0.004)
0.030 (0.004)
0.104 (0.012)
0.423 (0.034)
0.400 (0.023)
4.279 (0.243)
100
0.080 (0.009)
0.080 (0.010)
0.210 (0.022)
0.437 (0.033)
0.422 (0.025)
7.852 (0.342)
900
0.919 (0.097)
0.948 (0.100)
1.145 (0.072)
0.525 (0.039)
0.592 (0.040)
22.150 (0.892)
Table Newsvendor Problem: Average solve (in seconds) and sample time (in seconds X x10-4) with the associated standard deviation in brackets. The Exponential, ID Truncated Normal (T. .V) and 5D Normal DGPs correspond to the Newsvendor problem setting in Section 4.1. The time of the algorithm with the fastest solve is in bold; if two algorithms have the same solve time, then the sample time in bold. The solve and sample times for the 1D well-specified Normal and contaminated Exponential DGPs in Figure 4 are very similar to Tr. N and SD N respectively, and are thus omitted.
bootstrapping application which was also used by Shapiro et al. (2023) in their BDRO experiments.
E.1. Newsvendor Problem- Additional Details
This subsection is dedicated to additional details and experiments that complement Section 4.1 on the Newsvendor Problem.
Data-generating processes. In the main text, we evaluated the methods on two well-specified settings: an exponential distribution with rate parameter À = 20 and a 5DD multivariate normal distribution with mean μ4+ = (10, 20, 30, 35, 22) and full covariance E.. In this section, we also provide supplementary experiments for univariate Normal distribution DGP with mean pa 25 and standard deviation a. 10; the likelihood is also Normal, so the model is well-specified.
In Section 4.1, we also evaluated the algorithms when the likelihood is misspecified. The DGP is Truncated Normal distribution with mean μa, = 10, standard deviation a, = 10, and truncation & E [0, oo); the likelihood is univariate Normal distribution. In this section, we provide another misspecified example. The DGP is a Contaminated Exponential distribution whilst the likelihood is an Exponential distribution with conjugate Gamma posterior/prior. 80% of the observations from the Contaminated Exponential DGP are sampled from an Exponential distribution with rate parameter À 20 - the other 20% of observations are sampled from a Normal distribution with mean 100 and standard deviation 0.5.
Prior hy byprpaarameters. When the likelihood is a univariate Normal distribution (i.e. when the DGP is Normal or Truncated Normal), the prior and posterior are Normal-Gamma distributions; we set the prior hyperparameters to be jí = 0
and K, a, B = When the likelihood is an Exponential distribution, the prior and posterior are Gamma distributions with
prior hyperparameters a; 3 = 1. Finally, when the likelihood is a multivariate Normal distribution with dimension D = 5, the prior and posterior are Normal- -ishrse- distributions with prior hyperparameters jí = -0 R., v D ++1 1,
= 7 + D 2, and Ip, where In the D-dimensional identity matrix.
Supplementary experiment: Normal DGP. The top row of Figure 4 shows that the conclusions of Section 4.1 hold for the well-specified case of a Normal DGP/lkcelhhod,, that is, for M = 25, 100, DRO-BAS Pareto dominates BDRO in the OOS mean-variance tradeoff, whilst performance is similar for M = 900.
Supplementary experiment: Contaminated Exponential DGP. The bottom row of Figure 4 shows misspecification context where the DGP is a Contaminated Exponential and the likelihood is an Exponential distribution. As increases, the OOS mean of DRO-BAS and BDRO increases fast but with little reduction in the OOS variance: a decision maker is unlikely to desire this property. This implies that we should use a very small value of because a larger does not give
28

---

DRO with Bayesian Ambiguity Sets
5 with Exponential DGP
== 20 with Exponential DGP
N= ==100 with Exponential DGP
140-
Out-of-sumple muaat, m(c)
0.5
120
DRO-BASPE
0.5
DRO-BASE ORO-BASPP
BDRO
0.3g
100
0.5%
0.5
00000
80-
0703
4000
6000
8000
10000
4000
6000
8000
10000
4000
6000
8000
10000
Out sample variance, o(e)
Out-of sample variance, 1:(e)
Out sample variance, an(e)
Figure 5. The oft- sample mean variance tradeoff for Exponential DGP when the total number of observations n is 5 (left), 20 (middle), and 100 (right). For illustration purposes, blurred markers/lines show the previous two values of πı; for example, the right column shows = 100 in bold and = 5, 20 in blurred. Each marker corresponds to single value of some of which are labelled (e.g. e = 0.1, 3.0). If the marker is filled, this indicates the given lies on the Pareto frontier. If the marker is unfilled, this indicates the given is Pareto dominated by at least one other point.
much more robustness. Contrast this to the other DGPs we have analysed where the mean- veaanvariance tradeoff creates nice Pareto frontier curve on a subset of the e values whereby an increase in the OOS mean is traded for a decrease in the OOS variance. In particular, contrast this to the well-specified Exponential example from the top row of Figure 2. All of this provides motivation for future work on Bayesian ambiguity that specifically target model misspecification.
Solve and sample times. Table shows the solve times on the Newsvendor Problem for algorithms on different DGPs. As noted in Section 4.1, the solve and sample times are broadly comparable for DRO-BAS -AS and BDRO, although both DRO-BAS algorithms are faster across all DGPs and all sample sizes M.
Results for varying number of observations. We now examine the results from the Newsvendor problem for an increasing number of observations. We fix the number of MC samples to M = 100 and run the experiment for dataset size n = 5, 20, 100. Results for n = 5, 20 were also reported in Shapiro et al. (2023) for the well-specified Normal and Exponential DGPs/likelihoods. Figure 5 shows that expected performance of both methods improves as the number of observations increases. Moreover, DRO-BAS continues to Pareto dominate match BDRO in all cases with the difference between the two methods being bigger for smaller number of observations.
Why does the OOS variance increase as increases? The behaviour of the OOS variance v(e) in the Newsvendor experiments is due to the behaviour the variance of the solution (denoted by (e)). In turn, the variance of the solution is driven by the Newsvendor asymmetric cost function and its interplay with the worst-case distribution for each e. Figure 6 shows the behaviour of the solution for the Newsvendor problem under a Normal DGP corresponding to the top plot in Figure 4. The Newsvendor cost-function is piece-wise linear with 2 pieces, and hence, the true risk (expected cost under the DGP) has two different gradient slopes. The optimal solution lies at the intersection of the two pieces of the objective function, which coincides with the point with respect to where the true risk is zero The piece corresponding to larger solutions has a significantly smaller slope (see 4th plot of Figure 6), leading to a smaller true risk.
Let's consider a fixed value of M = 25 in Figure 6. For small values of e, the variance of the solution (e) is smaller because fewer distributions are included in the ambiguity set; hence, the obtained solution is fairly stable over replications (see first plot of Figure 6). However, because BAS likely does not include the DGP, the decision is prone to risk; thus, the OOS variance of the cost function u(e) is large (see the Pareto curve of the first plot of Figure 4). In this regime, the OOS variance (of the cost) reduces as we increase epsilon and better capture the DGP.
As c increases further, the ambiguity sets begin to include more distributions, leading to the mean solution moving to values larger than the optimal solution where the slope of the true risk is smaller (see 4th plot of Figure 6). This makes sense because, as we increase e, we become more conservative. However, as increases to very large values (in this case > 0.5),
29

---

DRO with Bayesian Ambiguity Sets
M= with Normal DGP
M=100 with Normal DGP
M=900 with Normal DGP
Expected cost under normal DGP
Optimal solution
Expected cost under the DGP Optimal
BDRO Bayes
60
BAS
DRD-BASy Bayes
50
Salutonx
40
Solution X
Solution
Solution
30
10
10
20-3
20
10
10-2
10-1
10a
10-1
10-2
10-3
10a
50
100
150
200
Figure 6. The first three plots showcase the mean solution (decision variable x)) with the associated variance (error bars) obtained by the methods applied to the Newsvendor problem with Normal DGP for M 25, 100,900. The last plot shows the true risk (expected cost under the DGP) for each value of the solution x. Red dotted lines show the optimal solution under the DGP.
the ambiguity set contains lot of arbitrary distributions, significantly increasing the out-of-sample cost. The methods then push the solution towards smaller values of æ as can be observed by the big variance Uz (e) on the first plot for larger values of
This behaviour is common for all values M, however, there an important distinction: for smaller values of M (25, 100), the bars extend way below and above the optimal solution. If we associate this with the form of the true risk on the expect very high variance v(e) of the out-of- soffsa cost. On the other hand, the variance of
last plot in Figure 6, the solution (c) for large M (M 900) does not extend below the optimal solution by large degree. This means that as the optimisation becomes more exact, the methods suggest staying on values bigger than the optimal solution, which corresponds to the smaller slope of the true risk. This is why for M 900 in the top row of Figure 4, the out-of-sample variance v(e) does not increase by a lot for large values of E.
E.2. Portfolio Problem: Additional Details
The dataset from Bruni et al. (2016) contains two baselines which we include for comparison in Figure 7: a Markowitz meaa-variame risk model (Markowitz, 1952) and the DowJones index. In Figure 7, we show the cumulative returns on the test datasets. We note that all three DRO methods have larger cumulative return than the Markowitz and DowJones index baselines.
Sliding time window construction (Bruni et al., 2016). The first train D and test DJ dataset contain the first 52 weeks of returns and following 12 weeks of returns respectively. We construct the second train dataset (2) by excluding the first 12 weeks of returns from D and including the 12 weeks from Df' whilst the second test dataset DY2) contains the next 12 weeks of data following D We repeat this procedure for all j I....,J until the end of the dataset.
E.3. Discussion Empirical vs Model-based DRO and Additional Comparisons
In this section, further discuss the relationship between empirical and model- based DRO methods. This work focuses scenarios where the decision-maker has access to both observations from the data-generating process (DGP) and a model family that describes the underlying data distribution. Specifically, we examine situations where incorporating uncertainty quantification about the model parameters into the worst-case minimization problem crucial. This setting is particularly relevant when model is required to capture a complex relationship between covariates and outcome variables, such as in regression tasks Bayesian risk minimization, where the decision-male seeks protection against a poor posterior
30

---

DRO with Bayesian Ambiguity Sets
Figure The Portfolio cumulative return across datasets. For each DRO method, line corresponds distinct value (shown in brackets in the legend). The vertical dotted line marks the start of the first test set week 52.
BDRO (le 05)
BDRO (0.001)
BDRO (1.0)
8
DRO-BASPa (G Ie-+1e- 05)
DRO-BASpa (G 0.001)
DRO-BASpa (G 1.0)
DRO-BASpy (Ie 06)
Out-of-sample comulative inturn
DRO-BASpp (0.001)
6
DRO-BAS (1.0)
DowJones Index
Markowitz
0
200 400
1000
1200
1400
Week number
distribution arising from limited noisy data. In such instances, purely data-driven (empirical) DRO formulations may not be the most suitable approach. To better understand the practical differences between these methods, we empirically assess our proposed approach against two empirical DRO methods. Firstly, the Empirical KI DRO framework introduced by Hu Hong (2013) uses the following ambiguity set:
B. (P...) [P dkL( Sc..
Secondly, compare against Wasserstein DRO with a 2 norm (see e.g. Kuhn et al., 2019; Gao & Kleywegt, 2023).
Figure 8 shows the OOS mean-variance trade-off of Empirical KI. DRO (in gray with crosses) and Wasserstein DRO (in brown with plusses) versus the model-based DRO approaches of DRO-BAS and BDRO on the Newsvendor Problem for two DGPs. For given DGP, note that the gray and brown curves do not change for M = 25, 100, 900 because Empirical DRO does not sample. The results show that DRO-BAS and BDRO perform better than Empirical KI. and Wasserstein DRO when sufficiently large number of samples is taken from the model (M 900). However, for M 25, Empirical DRO has better the OOS performance than model-based DRO approaches because more samples need be taken better approximate the model. For M 100, Empirical DRO and the model-based methods broadly comparable, with Empirical KI. DRO having advantage on the Truncated Normal DGP, likely due to misspecification of the likelihood.
E.4. Cross-validation for choosing
In practice, the decision-maker is unlikely to be able to calculate the optimal value of because they do not know the distributional form of the DGP. Instead, the decision-maker must calibrate from the available training data. One method for achieving this is to perform cross-validation (CV) by splitting the data into training folds, then evaluating the OOS performance proposed solution on the test folds. We perform 10-fold CV on the Newsvendor for DRO-BAS with a univariate Normal DGP and n = 100 training observations. We train our model only on the training folds, then after solving for each candidate e, we calculate the CV mean and variance of the objective function on the validation folds.
31

---

DRO with Bayesian Ambiguity Sets
M- with Normal DGP
M 100 with Normal DGP
=900 with Normal DGP
46
1.0
DRO-BASpe
sul-olsaaple nenn, m(()
DRO-BASPP
41-
BDRO
Gagwee 1.0
Empirical KL.
42
10.0
10.9
TUED
Empirical Wasserstei
1.
1.0
40-
0.5
0.5
0.
0206
05
38
(X00)
0.001
0.001
02001
750
800
850
900
750
800
850
900
750
800
850
900
Out-of-sample variance, v(e)
Out-of-saplle variance, v(c)
Out-of-sample variance, v(e)
22 with Truncated Normal DGP
M=100 with Truncated Normal DGP
M-900
with Truncated Normal DGP
1.0
37
10.0
10.0
10.0
DRO BASPE
Out- of su--uuppl mean, m(()
DRO-BASpP
36
1.0
BDRO
Empirical KL
35
05
0.5
1.00
Empirical Wasserstein
1.
1.
1.0
34-
0.3g
0.0
0.05
0.5
0.5
33
0.00
0.00
0.001
00.001
0.001
32
500
350
600
650
700
500
550
600
650
700
500
550
600
650
700
Out-of-eample variance, v(())
Out-of-sample variance, vife)
Out-of-eample variance, vfe))
Figure 8. Comparison of the OOS mean-variance tradeoff the Newsvendor for Empirical KL DRO and Empirical Wasserstein DRO Bayesian model-based DRO approaches on the Newsvendor Problem with Normal DGP (top) and Truncated Normal DGP (bottom).
The results in Figure 9 show how the decision maker can choose value of e depending upon their appetite for risk. For example, if the decision-maker wishes to minimise the CV variance, then they should choose the value of e marked by the stars in Figure 9. The triangle markers in the figure show a way to choose c that compromises between the CV mean and variance by taking a average the CV mean and the CV standard deviation. By considering the CV mean-variance is now well-informed choose e with knowledge of how the chosen value could perform on
tradcoff, the out-of-sample, unseen data.
32

---

DRO with Bayesian Ambiguity Sets
M with Normal DGP
M=100 with Normal DGP
M-900 with Normal DGP
Cross-vdidation mntau (CV-meun)
DRO-BASPP
41
min CV-mean
2.7
CV-var
40-
min CV-mean-std
0.5
DRO-BASpt
39
0.6
X min CV-mean
U.N.b.
min CV-var
38
0.02
min CV mean std
Ve X
0.15
37-
0.02
aap X
0.02 --..
0.005
0,001
0.00
700
750
800
830
700
750
800
830
700
750
800
830
Crose-validstion variance (CV-vur)
Cross-validstion variance (CV-var)
Crost-vaichtion variance (CV-var)
Figure The cross-validation (CV) mean-variance tradeoff the Newsvendor Problem with Normal DGP and 100 training observations. We highlight three ways the decision maker can choose e: firstly, minimise the CV-mcan (cross markers); secondly, minimise the CV-var (star markers); and thirdly, minimise 1 (CV-mean CV-std) (triangle markers), where CV-std the standard deviation of the objective function on the validation folds.
33


Reviews:
[{"review_id": "z0TGzDZjMJ", "reviewer": "Reviewer_V1H1", "summary": "The paper provides two new ways  to define the distributional robust counterparts for the  Distributionally Robust Optimization (DRO) taking into account posterior information, called Bayesian Ambiguity Sets (BAS) . In particular, the authors address the problem of the worst-case risk optimization where the wort-case distribution is taken subject to an ambiguity set. The first set is BAS with posterior predictive (BAS-PP), that is defined by bounding the KL divergence by $\\epsilon$ from the parametric distribution defined by posterior expectation. The authors propose to use it in the cases when distributions have bounded Moment Generation Function, since the DRO has a closed form in this case. For the more general distributions, the authors propose to use the BAS based on posterior expectation (BAS-PE), which bounds the expectation of the LK divergence to the potential parametric distribution, where the expectation is taken over the posterior distribution of a parameter. The authors prove that this formulation allows DRO to have a closed form for the case of exponential family of distributions. The authors demonstrate that the DRO with their ambiguity sets allows one-stage dual problem formulation, whereas the closest benchmark Bayesian DRO by Shapiro et al. (2023) needs to define two-stage optimization and requires more sampling. They also provide an empirical comparison it terms of computational complexity of solving the DRO including the sampling time, and compare the mean-variance trade-offs of the solutions obtained by different DRO formulations.\n\n## Update after rebuttal:\n\nI thank the authors for the detailed response and I keep my original score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully considering our work and providing feedback that greatly improve the paper:\n1. **Clarification of line 248 right side**:  $\\epsilon^{\\star}\\_{\\text{PE}}(n)$ in Eq. 20 is defined as the expected KL divergence between the data-generating process $\\mathbb{P}^\\star$ and model $\\mathbb{P}\\_\\eta$, indexed by the natural parameter $\\eta$. For any $\\epsilon \\geq \\epsilon^\\star\\_{\\text{PE}}(n)$ we have that $\\mathbb{E}\\_{\\eta \\sim \\Pi}[d\\_{KL}(\\mathbb{P}^\\star || \\mathbb{P}\\_\\eta)] = \\epsilon^\\star\\_{\\text{PE}} \\leq \\epsilon$ and hence $\\mathbb{P}^\\star \\in \\mathcal{A}\\_{\\epsilon}(\\Pi)$ by definition of the BAS-PE ambiguity set $\\mathcal{A}\\_\\epsilon(\\Pi)$. Since $\\mathbb{P}^\\star$ belongs to the ambiguity set, the worst-case risk will be at least as big as the risk under $\\mathbb{P}^\\star$ (since the supremum over a set is always at least as big as the value attained by any of its elements), resulting in Eq. 22. We will add this explanation to the main text and highlight its importance: if one chooses the radius at least as large as $\\epsilon^\\star\\_{\\text{PE}}(n)$ then the worst-case risk minimised by DRO-BAS-PE will upper bound the true risk under $\\mathbb{P}^\\star$.\n2. **Explanation of variance behaviour for increasing values of $\\epsilon$ in Fig. 2**: The reason is well-understood but quite challenging to convey compactly: the behaviour of the OOS variance $v(\\epsilon)$ in the Newsvendor experiments is due to the behaviour of the variance of the solution (denoted by $v\\_{x}(\\epsilon)$) (see Fig. 10 in this anonymised link: https://github.com/ICML-anon-2025/paper-11717) across epsilons. In turn, the variance of the solution is driven by the Newsvendor asymmetric cost function and its interplay with the worst-case distribution for each $\\epsilon$. The Newsvendor objective is piece-wise linear with 2 pieces, and hence, the true risk (expected cost under the DGP, in this example a Normal) has a similar two-piece behaviour. The optimal solution lies at the intersection of the two pieces of the true risk. The piece corresponding to larger solutions has a significantly smaller slope (see 4th plot of Fig. 10), leading to a smaller true risk.\n    - Let’s consider a fixed value of $M=25$. For small values of $\\epsilon$, the variance of the solution $v\\_{x}(\\epsilon)$ is smaller because fewer distributions are included in the ambiguity set; hence, the obtained solution is fairly stable over replications (see first plot of Fig. 10). However, because BAS likely does not include the DGP, and the decision-making is risk-prone, the OOS variance of the cost function $v(\\epsilon)$ is large (see the Pareto curve of the first plot of Fig. 4).  In this regime, the OOS variance reduces as we increase epsilon and better capture the DGP. \n    - As $\\epsilon$ increases further, the ambiguity sets start including a lot more distributions, leading to the mean solution moving to values larger than the optimal solution where the slope of the true risk is smaller (see 4th plot of Fig. 10). This makes sense as, as we increase $\\epsilon$, we become more conservative. However, as $\\epsilon$ increases to very big values (in this case $> 0.5$), the ambiguity set contains a lot of arbitrary distributions, significantly increasing the OOS cost. The methods then push the solution towards smaller values of $x$ as can be observed by the big variance $v\\_{x}(\\epsilon)$ on the first plot for larger values of $\\epsilon$. \n    - This behaviour is common for all values of $M$, however, there is an important distinction: for smaller values of $M$ (25, 100), the error bars extend way below and above the optimal solution. If we associate this with the form of the true risk on the last plot, we can expect a very high variance $v(\\epsilon)$ of the OOS cost. On the other hand, the empirical variance of the solution $v\\_{x}(\\epsilon)$ for large M ($M = 900$) does not extend below the optimal solution by a high degree meaning that as the optimisation becomes more exact, the methods suggest staying on values bigger than the optimal solution which correspond to the smaller slope of the true risk. This is why in the third plot of Fig. 4 in the main text ($M = 900$), the OOS variance $v(\\epsilon)$ does not increase by a lot for big values of $\\epsilon$.\n    - We believe this plot and the above explanation give great intuition on how the specific form of the cost function can affect the behaviour of the method and will hence include it in the Appendix.\n\n3. **Other comments**: We will change the statement in line 102 to your suggestion. We will further add a reference to Definition 3.4, noting that we are following the notation of Murphy et al. 2023 for the conjugate exponential family. We accidentally omitted the definition of $h(\\xi)$, which is referred to as the scaling constant.\n\n4. **Typos and references to proofs**: We will correct the typo and add direct references to the location of each proof after each mathematical statement.", "claims_and_evidence": "Yes, as far as I can see. However, it would be good to add the direct references / hyperlinks to the proofs in the Appendix of each Lemma / Theorem directly after introducing them in the body of the paper.", "methods_and_evaluation_criteria": "Yes, it seems so.", "theoretical_claims": "I tried to follow the derivations and claims in the body of the paper, and they all seem to be correct. I did not check the appendix.", "experimental_designs_or_analyses": "The experiments seem to be quite thorough.", "relation_to_broader_scientific_literature": "The paper provides a new way to define the ambiguity sets for the distributional robust optimization that take into account the bayesian posterior, and allow simpler optimization formulations for certain distribution families than the previous formulations.", "essential_references_not_discussed": "Not that I'm aware of of", "other_strengths_and_weaknesses": "The paper is very clearly written, very clean, almost everything is properly defined. The first formulation DRO_BAS_PP was proposed before in the non-bayesian framework, but extended here to the Bayesian framework, as they discuss in the paper. The proposed second formulation DRO_BAS_PE seems to be novel. The result of the closed form formulation of the DRO in this case for exponential family of distributions also seem to be novel.", "other_comments_or_suggestions": "There just a very few typos that I noticed, and comments I would like to make. \n\nTypos:\n- Line 268, Corollary 3.7: I think it should be $\\mathcal A_{\\epsilon}$ instead of $A_{\\epsilon}$\n\nComments:\n- Line 102 left side: Let x be a decision variable that minimizes a stochastic objective function... Maybe, that should be chosen to minimize? Otherwise it sounds like it is a solution already.\n- Line 198, right side:  When defining exponential family with conjugate prior, could you please add the particular reference where it is taken from? \n- Line 203 right side: Also, the function $h(\\xi_i)$ in Definition 3.4 seems to be undefined. What $h$ stands for?\n- Please add the references to the proofs in the Appendix directly after the corresponding results in the body of the paper.", "questions_for_authors": "- Line 248 right side: It follows that ...(21) - Where does it follow from? Here the flow was not very clear to me. Please clarify. \n- Figure 2. Can you please explain why for increasing $\\epsilon$ until 1, in some of the Pareto-curves the out-of-sample variance $v(\\epsilon)$ grows again?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LTlVdk7HUW", "reviewer": "Reviewer_fJ7Y", "summary": "In this paper, the authors focus on Distributionally Robust Optimization and introduce two new ambiguity sets with the aim of informing the construction of ambiguity sets using Bayesian Statistics. More specifically, they use the posterior distribution to construct these ambiguity sets. In the first set, we consider all distributions whose distance from the posterior-predictive distribution is less than a parameter \\(\\epsilon\\). In the second case, we take into account all distributions whose expected KL-divergence (w.r.t. the posterior distribution) from the nominal distribution is bounded by \\(\\epsilon\\). For these two cases, the authors provide appropriate reformulations. They then demonstrate the performance of their approach on a newsvendor and a real-world portfolio optimization problem. They show that their approaches peform as good as the BDRO method or better while being easier to solve and taking less time.\n\n## Update after Rebuttal\nThe authors have strengthened the paper with better numerical experiments. However, I decided to maintain my score after receiving the response of the authors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully considering our work and providing feedback and suggestions that greatly improve the paper:\n\n1. **Additional comparisons to other DRO ambiguity sets**: Firstly, the suggested empirical-based ambiguity sets, based on the KL-divergence and Wasserstein distance, are fundamentally different to model-based approaches (e.g. DRO-BAS, BDRO) as they are fully data-driven and are not appropriate in cases where the decision-maker relies on a model to make a decision. There is already a discussion of the model-based vs empirical KL DRO setting in Appendix E.3, however, we will add a short statement in the introduction pointing the reader to this discussion. \nTo this end, we have already included a comparison to the standard, empirical KL divergence ambiguity set that you suggested in Appendix E.3 and Figure 7. To further strengthen our empirical comparisons, and following your suggestion, we have also now conducted a comparison to the Wasserstein-based ambiguity set centred on the empirical measure for a well-specified (Normal DGP) and a misspecified (Truncated Normal DGP) Newsvendor example (see Figure 8 of this anonymous link: https://github.com/ICML-anon-2025/paper-11717). We observe that, for M sufficiently large, DRO-BAS outperforms all empirical methods. We will include this new comparison in Sec. 4 and in the Appendix. \n\n2. **How do the worst case distributions identified by BAS and BDRO differ?**: Crucially, for BDRO, a single worst-case distribution does not exist because BDRO considers an expected worst-case approach (see Eq. 2, Figure 1 and the discussion on lines 67-79) rather than a worst-case approach, as advocated by DRO methods. In contrast, since both DRO-BAS formulations correspond to a worst-case risk minimisation objective, we obtain the worst-case distributions in Sec. 3.6. If we sample $\\theta_j \\sim \\Pi$ from the posterior for BDRO, then one can obtain a worst-case distribution for each $\\mathbb{P}\\_{\\theta_j}$ via the argument in Hu & Hong (2013), eq. (8). However, notice that in general, the minimiser of an expected objective is not the same as the average of the minimisers of the individual objectives as $\\min\\{f(x) + g(x)\\} \\geq \\min\\{f(x)\\} + \\min\\{g(x)\\}$ for any functions $f$ and $g$. Hence, even looking at the posterior mean or mode of the worst-case minimisers of the inner worst-case objectives in Eq. 2 would not necessarily give us a single distribution $p$ that yields a worst-case risk of the form $\\mathbb{E}_{\\xi \\sim p} [f(x,\\xi)]$ corresponding to the risk minimised by BDRO. In the limiting case of infinite data observations ($n \\rightarrow \\infty$) all of BDRO, BAS-PE and BAS-PP concentrate to a KL ambiguity set based on the data-generating process $\\mathbb{P}^\\star$ hence the worst case distribution is asymptotically the same for all methods but it is different (or non-existent for BDRO) for finite sample sizes.\nThis further highlights a fundamental difference between BDRO, which opts for an expected worst-case objective, and DRO-BAS, which follows the common DRO route of defining a worst-case objective. DRO-BAS provides the decision-maker with an exact and interpretable formulation of the worst-case risk being minimised. We will expand further on these points in lines 60-63 where we discuss the main difference of our method to BDRO and also include the above points in Sec. 3.6, which deals with the DRO-BAS worst-case distribution. \n\n3. **Extension to general phi-divergence metrics?**: This is something we are actively working towards and believe would be very valuable future work.\n    - The analysis for DRO-BAS-PP would involve the derivation of the dual formulation of the problem based on convex conjugate results for $\\phi$-divergences. Results about the radius and the worst-case distribution might be more challenging and would rely on properties of the posterior predictive and $\\phi$-divergences.\n    - The analysis for DRO-BAS-PE appears even more challenging: one would need to derive, in closed-form, an expression for the expected $\\phi$-divergence under the posterior (similarly to Lemma 3.5). This would allow us to derive the convex conjugate form of the expected $\\phi$-divergence, which is otherwise challenging to obtain. The convex conjugate expression of the expected divergence is essential in order to derive the strong duality result in Theorem 3.6, but the closed-form of the expected KL divergence further allowed us to obtain the results about the tolerance level selection (Sec. 3.4), a closed-form objective in the Gaussian case with linear cost function (Sec. 3.5) and the closed form of the worst-case distribution (Sec. 3.6).\n    - In the discussion, we will explain the challenges associated with extending BAS to other $\\phi$-divergences and suggest this as important future work. We believe this framework opens the door to alternative BAS formulations grounded in expected distances or divergences, such as the $\\phi$-divergence you mentioned.", "claims_and_evidence": "The theoretical results provided in the paper appear sound. The numerical experiments demonstrating the benefit of this approach over the BDRO method appear sound, and both real and synthetic data are used well. The key limitation, I believe, is the limited comparisons to alternative methods. While BDRO is indeed the most comparable method, it would be interesting to see evaluations against most common DRO sets such as standard KL divergence sets and sets with other metrics to evaluate the advantages and disadvantages of the Bayesian approach.", "methods_and_evaluation_criteria": "The authors demonstrate the performance of their method using a pareto frontier with the out-of-sample mean and variance of the solution. They use both real and synthetic data to demonstrate their approach. All this appears reasonable.", "theoretical_claims": "I verified the key results associated with duality theory specifically Proposition 3.2, Proposition 3.3 and Theorem 3.6.", "experimental_designs_or_analyses": "The experimental analysis presented in the paper is sound and is similar to existing work in this domain.", "relation_to_broader_scientific_literature": "The paper extends existing literature on the use of Bayesian Ambiguity Sets for distributionally robust optimization by introducing new types of ambiguity sets which combine together KL-divergence and posterior information from data. This allows us to simultaneously use data and prior beliefs.", "essential_references_not_discussed": "None that I know of.", "other_strengths_and_weaknesses": "Strengths:\n1. Novel ambiguity sets which incorporate both data, prior beliefs and distance metrics\n2. Tractable reformulations for these ambiguity sets\n3. Discussion of how to choose tolerance parameters.\n4. Numerical experiments with both synthetic and real data. \n\nWeaknesses\n1. While the comparison with other Bayesian DRO approaches is present, it is difficult to evaluate how the Baysian Ambiguity Sets perform when compared against other DRO approaches such as Wasserstein ambiguity sets etc.", "other_comments_or_suggestions": "1. Have the authors also considered developing these sets for general phi-divergence metrics instead of just KL-divergence.", "questions_for_authors": "1. I suggest authors compare their approach against other standard ambiguity sets so we can better understand how useful the Bayesian Ambiguity Sets are in practice. \n2. Can the authors discuss a bit about how the worst case distribution that is being identified by the two BAS and BDRO differ amongst each other.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "ZxUkPUTueQ", "reviewer": "Reviewer_oPX6", "summary": "This paper introduces KL-based DRO formulations with two kinds of Bayesian ambiguity sets, the posterior expectation and the posterior predictive. The authors show that both formulations can be recast into a direct minimization problem, with more efficient closed form worst-case risk solution for exponential family distributions. Empirical study also shows the effective of the proposed estimators.\n\n## update after rebuttal\n\nI hope the authors would add comparisons to BDRO, standard ambiguity set, $\\phi$-divergence in revision, as promised by the authors in their rebuttal. But in the current form, I still lean towards accept weakly.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully considering our work and providing feedback that greatly improves the paper:\n\n1. **Upper bound in Eq. 13**: There is potentially a misunderstanding here: our work does not rely on this bound and goes beyond this inequality to achieve the strong duality result (with equality) in Theorem 3.6. In detail: \n    - *\"What's the point of the upper bound in Eq. 13?\"*: It provides a weak duality result for general parametric Bayesian models without closed-form posteriors, e.g. Bayesian neural networks. The discussion after Proposition 3.3 explains the challenge associated with BAS-PE leading to the upper bound in Eq. 13. Exactly for this reason we study the widely used exponential family models in Sec. 3.3, allowing us to move from an upper bound (weak duality) to an equality (strong duality) in Theorem 3.6. \n    - *\"It resembles Eq. 8, but is not quantitative since it depends on $f$ and $\\theta$ which makes the generalization property unclear\"*: The goal of the duality result is to reformulate the maximisation in Eq. 11 as a minimisation so that the objective can be jointly minimised in the Lagrange variable $\\gamma$ and decision variable $x$. As a duality result, it necessarily relies on $f$ and $\\theta$ as these are part of the worst-case risk (the same dependence is in Proposition 3.2, Theorem 3.6, and in the duality results of BDRO and other methods).\n    - *\"Is it possible to provide a more quantitative bound, [...] Otherwise, the theoretical analysis in the paper, except for the conjugate, looks like trivial corollaries of results in Hu \\& Hong, 2013.\"* The theoretical analysis in Sec. 3.3 is far from trivial and cannot be seen as corollaries of Hu et al. 2013 as their analysis is for divergences and not for expected divergences. The goal of Sec. 3.3 is to go beyond the weak duality bound in Eq. 13 and provide stronger results. In particular, it provides an important formulation of the expected KL divergence in Lemma 3.5, which is crucial to overcoming the difficulty of obtaining the convex conjugate of the expected KL. This leads to a strong duality result in Theorem 3.6 along with the results in Corollary 3.7, Eq. 20 and Sec. 3.6, which have not been derived before for an ambiguity set based on an expected divergence (rather than just a divergence). To avoid similar misunderstandings in the future, we will include the clarifications above before Proposition 3.3.\n\n2. **Optimal radius selection**: Such a probabilistic guarantee would be very interesting but is challenging future work that we add to the discussion. It would require finite sample concentration results about the KL divergence with respect to the posterior distribution. This is challenging as evident from the lack of such results in the DRO/KLD literature. To provide an alternative method of selecting the tolerance level, we now also perform a cross-validation radius selection analysis (see response 2 to GxWu)\n\n3. **Restriction to the exponential family**: As discussed in Sec. 5, this is an interesting topic for future work. The key property is that the family of distributions should have a closed-form expression for the expected KL divergence, such as we have in Lemma 3.5. Otherwise, one can apply Proposition 3.3 to obtain an upper bound on the worst-case risk. We will include this comment in the discussion at the beginning of Sec. 3.3. Thankfully, we have obtained such a result for the exponential family, which is widespread across probabilistic graphical models, as well as models in spatial statistics and time series analysis, which also rely heavily on the exponential family. Moreover, it plays a central role in Variational inference, where it is the de facto approximating family, making our work broadly applicable across Machine Learning domains.\n\n4. **Choice of KL divergence (KLD) & missed paper in lit. review**: Firstly, the KLD is a natural choice as the Bayesian posterior itself targets the KLD minimiser between the model family and DGP. Hence, intuitively, the expected KLD and the KLD with respect to the posterior predictive become good measures of separation between our posterior beliefs and the DGP (see main text lines 119-127 and Shapiro et al. 2023). The KLD further allows us to obtain a strong dual formulation in Theorem 3.6 for the DRO-BAS-PE setting and tractable reformulations (see Sec. 3.5). Moreover, by using the KLD, DRO-BAS admits the formulation of the optimal radius and the worst-case distribution, which are very important for interpretability and applicability of the method, overcoming shortcomings of the KL-based BDRO. We will further discuss the challenges associated with extending the framework to other distances/divergences in future work (Sec. 5) (see response 3 to fJ7Y) and include the missed paper in our literature review.\n\n5. **Typos and other comments**: We will correct the typos and properly define the function $h$, which corresponds to the scaling constant of the exponential family.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes. I have checked the proofs of Proposition 3.2 and 3.3, both of which look correct to me.", "experimental_designs_or_analyses": "Yes. I have checked the portfolio experiments which are reasonable.", "relation_to_broader_scientific_literature": "The key contributions are related to the KL divergence-based distributionally robust optimization literature by approach it from a Bayesian perspective.", "essential_references_not_discussed": "Since the method is mainly based on KL-based DRO, the authors may want to discuss a seminal work of the more general $\\chi^2$ divergence-based DRO.\n\n\n[1] Duchi, J., & Namkoong, H. (2018). Variance-based Regularization with Convex Objectives. Journal of Machine Learning Research, 19, 1–55. http://jmlr.org/papers/v19/17-750.html.", "other_strengths_and_weaknesses": "**Strengths**\n\nIt is novel to propose a Bayesian DRO formulation to tackle the worst-case risk instead of the expected worst-case risk. The examples with conjugate exponential family provide insights into applicability of the proposed methods.\n\n**Weaknesses**\n\nThe derived upper bounds in the theoretical analysis look qualitative rather than quantitative. The adopted KL divergence is restrictive, whose efficient dual formulation is well-known thus makes most of the dual analysis look mathematically trivial. See my detailed comments below.", "other_comments_or_suggestions": "- Line 035, right column, the data **is** noisy.\n- Line 290, Eq. 9,11,12, missin period.\n- Line 201, Definition 3.4, function $h$ is not defined or remarked before being used.\n\nThe authors should proofread the symbols, commas and periods used in the paper, especially in the context around math equations.", "questions_for_authors": "The paper is well-written and the idea is novel. I have a couple of minor concerns though.\n\n1. What's the point of the upper bound in Eq. 13? It resembles Eq. 8, but is not quantitative since it depends on $f$, $\\theta$, which makes  the generalization property unclear. Is it possible to provide a more quantitative bound, for example, by adopting Rademacher complexities? Otherwise, the theoretical analysis in the paper except for the conjugate looks like trivial corollaries of results in Hu\n& Hong, 2013.\n2. The optimal radius selection in Section 3.4 and its approximation with empirical data makes sense. However, there is still a gap between posterior distribution and the true distribution. The choice in line 265, right column does not take into account such gap. In other words, given a confidence level, say $\\tau$, and number of samples $M$, how can I choose the radius that yields a non-asymptotic bound for Eq. 21 to hold with probability at least $1 - \\tau$?\n3. The theoretical guarantees and computational efficiently are somewhat tightly coupled with exponential family distributions. The authors admit the limitations in Discussion as well. What properties should a family of distributions encompass to go beyond exponential family while retaining the advantages claimed in the paper?\n4. Why do the authors choose KL divergence-based DRO instead of DRO based on Wasserstein distances or MMD?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2pcPfOuTmc", "reviewer": "Reviewer_GxWu", "summary": "The authors introduce a novel formulation for distributionally robust optimization based on Bayesian posterior updates. Their model leverages KL-divergence, and they propose two ambiguity set designs with efficient sampling methods. These methods are computationally better than Bayesian DRO under mild conditions. Experimental results demonstrate their computational and statistical benefits.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing feedback that greatly improves the paper. Please see this anonymous link for Tables and Figures: https://github.com/ICML-anon-2025/paper-11717 \n\n1. **Table 1**: We have changed the table and caption to distinguish between variables and the input size of the dual (see link above). As you suggested, we will also add a brief discussion of Table 1 inside Sec. 3.5 with a more detailed explanation in the appendix. \n\n2. **Practical choice of $\\epsilon$ and cross-validation (CV)**: \nWe have now implemented a 10-fold CV procedure for choosing $\\epsilon$ from a set $E$ for BAS-PE and BAS-PP. As suggested, we took Fig. 7 for the Normal DGP Newsvendor. We train our model only on the training folds, then after solving for each $\\epsilon \\in E$, we visualise the CV mean and variance on the validation folds across all replications in Fig. 9 of the link provided above. On Fig. 9, we have marked the $\\epsilon$ that achieves 1) the smallest CV mean, 2) the smallest CV variance, 3) the smallest CV mean/standard-deviation tradeoff.\n    \n3. **Statistical benefits of DRO-BAS over BDRO**: \nOne key difference is that we have a single worst-case objective, leading to an interpretable closed-form worst-case distribution (Sec. 3.6), whereas BDRO does not (see response 2 to fJ7Y). At the limit of infinite observations, both DRO-BAS and BDRO converge to a KL-based ambiguity set based on the DGP. However, our analysis is straightforward and more intuitive compared to BDRO, which uses a much more complex argument (p.1286-1292) to prove where the expected worst-case concentrates. \nIn terms of statistical properties, we follow suggestions from the DRO literature (Shapiro et al. 2023, Gotoh et al. 2021) and focus on the mean-variance frontier, which is a type of statistical trade-off, showcasing our domination over BDRO. See also response 2 to V1H1 (on the statistical behaviour of the solution).\n\n4. **Interpretation of BAS in comparison to PDRO**:\nAlthough the dual formulations of BAS and PDRO with the KL are both model-based and look related, they are fundamentally different. Our ambiguity sets are a-posteriori informed, integrating prior beliefs and data evidence. Consequently, the resulting nominal distribution ($\\mathbb{P}_n$ for BAS-PP and $\\mathbb{P}\\_{\\hat{\\eta}}$ for BAS-PE) contains all the information from our posterior beliefs, including their uncertainty quantification, unlike the point estimator approach of Iyengar et al., 2023. This allows us to propagate uncertainty from the posterior beliefs about the parameters to the ambiguity set. We will highlight this important distinction by expanding the lines 53-63 with the points above.\n    \n5. **Scalability**: If the posterior is not available in closed-form, then the sampling time will increase (similarly to all Bayesian DRO methods) unless one performs approximate inference such as Variational or Laplace approximations. In this paper, we provide efficient formulations for a big class of models - the exponential family (which goes beyond just the Normal and Exponential distributions). See also response 3 to oPX6. Moreover, working with the exponential family enables our tractable formulations of the DRO-BAS problem and leverages the conjugacy property, which greatly enhances the scalability of our method.\n\n6. **Monte Carlo guarantees**: It would be valuable to theoretically examine the effect of the Monte Carlo sampling size to the out-of-sample cost, similar to our empirical analysis in Sec. 4. This is a non-trivial analysis as it requires establishing bounds on the worst-case KL objectives based on a nominal distribution and its empirical approximation. This is also likely the reason why Iyengar et al., 2023 only provide these types of guarantees for the Wasserstein and $\\chi^2$-divergence, and why Shapiro et al., 2023 omit them for the KL-based BDRO. Such guarantees would be a notable contribution on their own as they would likely lead to similar results for other model and KL-based DRO methods (like PDRO and BDRO). We will include this as future work in Sec. 5.\n\n7. **Choice of divergence**: Please see response 3 to fJ7Y for a discussion of other $\\phi$-divergences and response 4 to oPX6 for the benefits of KL-based BAS. The advantages of the KL have also been widely discussed in the DRO literature (Hu et al. 2013, Shapiro et al., 2023) and the closely related DR Bayesian Optimisation literature (Husain et al., 2023). We see extending BAS to other distances or divergences as an interesting future research direction.\n\n8. **Other comments**: We will correct the vector notation for $h$ in the newsvendor objective. Concerning visualisations in Fig. 2 & 3, we have kept the $\\epsilon$ markers because we do not want to give a false impression that the line is continuous and the values of $\\epsilon$ are equidistant, but we will work to make the figures more legible. \n\n[1] Husain H. et al. Distributionally robust Bayesian optimization with ϕ-divergences. 2023.", "claims_and_evidence": "I had some questions related to expressions in Table 1:\n\n1.\tThe authors do not clarify how the problem dimension $D$ influences the number of variables. It would be helpful to reference this explicitly, at least in Appendix.\n\n2.\tRegarding the column “Linear f”, it is nonclear why $BAS_{PE}$ and $BDRO$ exhibit a dependency on $D^2$. Given that the number of decision variables appears to be only $O(D)$ (and $O(M_{\\theta} D)$) after the transformation in Line 290 or Equation (40), further clarification is needed.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria appear well-justified for the application.", "theoretical_claims": "I reviewed the proofs for all the main results (except concrete examples in Appendix A) and did not find major issues that worth mentioning.", "experimental_designs_or_analyses": "I checked the validity of experimental designs and had several problems:\n\n1.\tI appreciate the visualization of Figures 2 and 3, particularly regarding the mean-variance frontier. It would be insightful to analyze which $\\epsilon$ values would be practical for real-world implementations. \n\n2.\tIn Figure 7 of Appendix, the authors present multiple $\\epsilon$ values for each method across the whole time period. It would be beneficial to select the optimal $\\epsilon$ via cross validation and compare their performance accordingly, as is common in DRO literature. Also, all different methods appear to collapse under $\\epsilon = 1$, but this is not well visualized.", "relation_to_broader_scientific_literature": "The paper contributes a novel approach by incorporating Bayesian posterior updates into the KL-DRO framework.", "essential_references_not_discussed": "I do not find the authors missing any important literature.", "other_strengths_and_weaknesses": "Overall, the paper is well-written and clearly articulates its contributions. Below, I outline some concerns and areas for improvement:\n\n1.\t**Statistical Benefits**: Besides computational advantages, the statistical benefits of DRO-BAS are not entirely clear. Can authors further discuss how DRO-BAS compares to Bayesian DRO in terms of statistical performance? Additionally, under what conditions should one prefer DRO-BAS over Bayesian DRO given the prior? Based on the paper’s description, the observed experimental benefits seem primarily due to improved computational approximation.\n\n2.\t**Novel Considerations**: While I appreciate the authors’ interpretation for these Bayesian ambiguity sets, the model design in Equation (18) and Theorem 3.6 appears to closely align with the PDRO method from Iyengar et al. (2023) when using KL-divergence cases (from the standard KL reformulation technique from Hu and Hong (2013)). Both approaches define an ambiguity set centered at an estimated parametric distribution via KL-divergence. Although the interpretations differ, a discussion clarifying the novelty of the approach would be beneficial. \n\n3.\t**Scalability for General Models**: While the proposed approach performs well for exponential and normal distributions, posterior updates for general large-scale models may still be computationally expensive. This scalability challenge is a well-known limitation of Bayesian DRO.", "other_comments_or_suggestions": "1.\tIn the multi-product newsvendor, the objective formulation should be written as $h 1^{\\top} max(0, x - \\xi)$. The current formulation is of vector output.\n\n2.\tThe texts in Figures 2 (especially) and 3 mix together across different methods. A clearer visualization strategy is to use an arrow to indicate the direction from 0.001 to 1 simply, instead of marking all the values there.", "questions_for_authors": "1.\tSince the sampling number affects the performance of DRO-BAS, are there any Monte Carlo sampling guarantees similar to those in Iyengar et al. (2023) that provides strong performance guarantee?\n\n2.\tThe choice of divergence functions appears flexible. For example, I do not see a fundamental issue when extending the design idea to $chi^2$-divergence. In such a case, Property 3.1 might only require the existence of a second moment. Could the authors elaborate on this or discuss necessary modifications? Such extensions would enhance  the practical applicability of the method, especially given concerns about KL-divergence’s conservativeness and little use in practice.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: All reviewers agree that this well-written paper presents solid contributions. The paper is clear, well-motivated, and organized well. The theoretical results are novel and significant, and they are explained well. The numerical contributions are insightful too. The reviewers did not identify any significant concerns.
Decision: Accept (spotlight poster)