Minimax Posterior Contraction Rates for Unconstrained Distribution Estimation on $[0,1]^d$ under Wasserstein Distance
Authors: Peter Matthew Jacobs
Lekha Patel
Anirban Bhattacharya
Debdeep Pati
Venue: Accepted by TMLR ()

Abstract: We obtain asymptotic minimax optimal posterior contraction rates for estimation of probability distributions on $[0,1]^d$ under the Wasserstein-$p$ metrics using Bayesian Histograms. To the best of our knowledge, our analysis is the first to provide minimax posterior contraction rates for every $p \geq 1$ and problem dimension $d \geq 1$. Our proof technique takes advantage of the conjugacy of the Bayesian Histogram.

Published in Transactions on Machine Learning Research (01/2025)
Minimax Posterior Contraction Rates for Unconstrained Distribution Estimation on [0, 1] under Wasserstein Distance
Peter Jacobs
pmjjoobb@sandia.pp
School of Computing University of Utah
Lekha Patel
Ipatel mootel@@sndia.coo
Scientific Machine Learning Deparbmend Sandio National Laboralories
Anirban Bhattacharya
anárbanb @ittat www.atttttttttttat..mmmmmmmmmmae edu
Department of Statistics Tezas ABM University
Debdeep Pati
dpali2 wawliitwwii...m edu
Department of Statistics University of Wisconsin
Reviewed on OpenReview: https: openreview. net/ forum? id UrSgGSTM2J
Abstract
We obtain asymptotic minimax optimal posterior contraction rates for estimation of proba- bility distributions on [0, 1] under the Wasserstein-p metrics using Bayesian Histograms. To the best of our knowledge, rates for every p 2 1 and problem dimension d 2 1. Our proof technique takes advantage of the conjugacy of the Bayesian Histogram.
analysis is the first to provide minimax postcrior contraction
1 Introduction
The Wasserstein metric is a popular tool for comparing two distributions y and v defined on a common metric space (E*, -1. I12) where E CCE R. For 1 1sp oo, the Wasserstein distance W, is defined as
/ |1 v1 dz(z,))
1/v
inf Wo(mi, = (FEM(U)))))
where M(ni,p) is the set of couplings of μn and (22 specifically the joint probability measures on E E with marginals and μ2 respectively. Some benefits of using the Wasserstein metric include its sensitivity to distance in the underlying space, ability to compare distributions regardless of continuity level, and its dimension equivalent. representation as the I.P distance between quantile functions, which facilitates quantile function inference (Zhang et al. 2020).
In this paper study the problem of non parametrically estimating distribution Po on Edd (where E [0, 1]) under the Wasserstein metric from independent and identically distributed (i.i.) random variables Y....,,, drawn from Pa- Our focus is on the unconstrained problem; that is, we place no additional assumptions on Po- From the viewpoint of analyzing only frequentist estimators, this is well studied problem. The frequentist convergence rates of the empirical measure under the expected Wasserstein distance are studied in (Fournier & Guillin 2015; [Singh Póczos 2018 Bobkov Ledoux]
2019 Weod Bach
2019) to varying degrees of generality. A consequence of the work of (Singh & Póezos 2018) is that on the
metric space ((0, 1]d, || -. [|2), for d N, for the class of Borel probability measures, the empirical measure

---

Published in Transactions on Machine Learning Research (01/2025)
is minimax optimal (at least up to logarithmic terms) for every p 2 1. Further, the minimax rate is lower bounded by n n-1/2p for d < 2p, and n-1/d for d >>>>> 2p.
Far less has been done in providing frequentist guarantees for Bayesian statistical procedures when the inferential goal is to estimate a non-maraetr distribution underneath a Wasserstein distance. In a non- parametric Bayesian model aimed at inferring a probability distribution on Fd, for each sample size n, a prior Hon is placed on the space of Borel probability measures on Ed. We denote this space Pa(L)). The sample size n posterior distribution, which we denote (1,Y,,,,,,..... Yr), is regular conditional distribution over Pa(E) induced from the likelihood and the prior Ho.. Given distance function d between probability measures on E (e.g Kullback-Leibler Hellinger, Wasserstein-1, Total Variation, etc.) we say the sequence of posterior distributions contracts almost surely at the rate en under Po if (P, E Pa(E)= : d(P P) 2 en)
sid converges almost surely to 0 as n - so when Y1, Y2,,.,,,,, Yn ~ Po (Ghosal& & Van der Vaart 2017
There are other useful but weaker notions of posterior contraction, such as the in probability variant which seeks only to show that for every Mn - oc, [I, E Pa(E)) : d(Po, P) 2 MaEx) + in probability under Po- We focus on the stronger almost sure version in this work because an almost sure PCR of cn holding uniformly over every Po in a finite class Pa(E) implies the existence of an estio P. derived from the posterior achieving Ep, d(Po Px) S en uniformly over Po E Pa(E). Specifically, an asymptotically minimax, almost sure PCR holding uniformly over the finite class Pa camnot decay at a faster rate than the minimax It is for this reason that if for every Po E Pa(E) the almost sure Posterior Contraction Rate (PCR) 6n is achieved, and 6n is the frequentist minimax rate over Pa(FF), we say the Bayesian method is agnostic to the prior choice in the presence of an infinite amount of data under class Pa(E). In particular, the practitioner can specify prior knowledge that may be beneficial for inference at small sample sizes when that prior knowledge is correct, but if this prior knowledge is inaccurate, the method will still be competitive (from the minimax perspective) with the best possible estimator when the sample size is sufficiently large. In our work Pa(E) is not finite, and we introduce in Definition | a slightly stronger property of contraction rate across a distribution class which cannot outperform the minimax rate regardless of the size of Pa(E); the main result of this work proves this stronger property.
rate [
Ghosal et al. 2000) provides a general three condition verification strategy for proving these PCRs. This strategy or minor variants have been the catalyst for myriad of minimax PCR results for the problem of estimating distribution via i.id samples. For example, SSricciolo (2077) uses histogram model with a prior on the bin weights and number of bins to prove minimax PCRs for the distribution class possessing a Holder smooth densities on [0, 1jd, where a E (0, 1], d 2 1 and d is the Hellinger distance. Kruijer & Van der Vaart (2008) also achieves minimax PCRs for the estimation of a Holder density class when d E (1,2) and d is the Hellinger distance using a weighted mixture of Beta density functions with a prior on the mixture weights and number of mixed densities. A notable achievement of these works is that the value a is not used in the construction of the posterior, yet minimax optimality is still achieved for the a Holder density class. That is, the methods adapt to the smoothness parameter. [Shen et al. (2013) provides minimax PCRs for the estimation of smooth distributions on Rd using a weighted mixture of Gaussian distributions with a prior on the covariance matrix of the mixed Gaussians when d is the Hellinger distance or Total Variation distance; smoothness adaptive guarantecs are provided. However, note that rather than studying the uncondtrained distribution estimation setting, these works focus on estimation of distributions known to possess a smoothness property, and in any case do not study estimation underneath Wasserstein distance.
Unfortunately the Ghosal et al. (2000) approach is more difficult to use when d is Wasserstein metric. Challenges include Wp,p 2 2 not being dominated by Total Variation or Hellinger distances, causing the need for explicit test construction. Also, the Kullback-Leible neighborhood condition, which ensures such neighborhoods of Po have sufficient prior mass, may make it more difficult to achieve the minimax rate under Wp.P p 2 because depending on the model under consideration, approximation of distributions under the Kullback-Leibles divergence may not be achievable at the square of minimax rate under W ? In light of these challenges, there have been far fewer theoretical advances in proving minimax optimal PCRs for distribution estimation under Wp, P 2 1 than under Total Variation and Hellinger distances.
1 scc appendix Lemma for proof
1 Chae et (002 (p (p33644) already encounters Kullback-TLebbe condition limitations when only estimating distributions
on
2

---

Published in Transactions on Machine Learning Research (01/2025)
Chae et al. [021 successfully derive a 2 condition verification strategy for proving PCRs under Wasserstein distance when F = R. This work appears to be the first providing set of general conditions for proving Wasserstein PCRs for the unconstrained, non-parametric distribution estimation problem, and their method is applicable for distributions with unbounded domain. Their results are restricted to the study of 1 di- mensional distributions. Also, the Ghosal et al. (2000) and Chae et. al. (202T) frameworks depend on the posterior distribution being available through Bayes formula. Camerrlengii
al. (2022) observes that this can be limiting and develops method to derive PCRs when the posterior is not available via Bayes formula. They apply their technique to derive Wasserstein PCRs for each d E N,
for the model placing Dirich- let process prior on the data generating distribution. The PCR derived for Po E Pa([O, 1]) is 2 n (aंश which via the discussion earlier in this section is slower decaying than the minimax rate by a polynomial factor for every d P. 2 1. In the Deconvolution problem, where for i . n); Y, = X; + 6 and X1...., .n sid Po is independent of Ep M4 and M40 is known and the goal is to infer Po. Wasserstein distance PCRs have been derived in Rousscau & Scricciolo (2023 Gao & van der Vaart (0066) and Scricciolo 2018). Po is called the mixing distribution. The distribution of Y1, however, is the convolution Po Mu-
1.1 Contributions
Our main contribution is Theorem ! In it we obtain PCRs for every dimension d 2 and for every distance wp.P 2 and the PCRs achieved are minimax optimal at least up to logarithmic terms. To the best of our knowledge, our result is the first to provide a minimax optimal PCR across each (d E N. 2 1) setting for estimating an unconstrained Po E Pa([O, I]). These rates are achieved using a Bayesian Histogram model that partitions [0, 1 into by equal area squares where bn := [f.) for sequence k, growing as a function of the sample size n at the appropriate rate, uses the Multinomial likelihood to weight the constant density within each square, and places a sample size dependent Dirichlet prior distribution on the weight vector with prior concentration vector as,, (of dimension 11)). This model induces sequence of posterior distributions as, over Pa([o, 1]). In Theorem [ we show that
Ta PP((,, 1J) ()) WP P) (n),,))
in a very strong sense under Po (implying almost sure and in probability convergence) provided that
1. If d S 2p then kn n Ejerttiezt Oj.b. nt, en =
2. If 2p, then lin nd, Ejertooag Oj.or S n- En -
where log terms in 6. which specified in the theorem are ignored here. In the problem of providing optimal PCRs for estimating distributions on [0, 1jd under Wasserstein-p distance, our results close the gap between the minimax rates for this problem and the Wasserstein PCRs provided by Camerlenghi et al. 2022)
The remainder of this paper is organized as follows. In Section 2 we formally introduce the Bayesian His- togram model. In Section g we introduce the strong notion of posterior contraction under which our thoorems are proved and show connections between this notion of posterior contraction with minimax lower bound rates and traditional PCR statements. In Section 4 we state the main theorem and the three fundamental lemmas upon which the main theorem depends. We then prove the main theorem. In Section 5 we provide instruction on how to use the prior distribution to express prior beliefs when using this model in practice. In Section we provide the proofs of the lemmas and in Section 7 we provide concluding remarks.
2 Bayesian Histogram
2.1
General Notations
Since we always consider probability distributions on 1)4, we drop the E notation of the introduction and denote
Pa (Borel Probability Measures on [0, 1)d)
3

---

Published in Transactions on Machine Learning Research (01/2025)
Excluding the right end points are notational convenience but extension of the arguments that follow to include the right endpoint is trivial.
For sequences of numbers an, bn defined for sufficiently large n E N, an S 6, means there exists a C > 0 and
No E N such that for 2 No, & Cb.. For b, d N, we denote [b] = (1.2..... .b) and [b] [ [+]
For R., B(B) denotes the Borel measurable subsets of B. For j E N, S refers to the (j 1) dimensional probability simplex. That is 5 = {(f1,..., ..))
E R Σi= 2t = 1, 2 0 for t E [i]] Also note that I = {i E R : a > 0} and for = E N and a E R4, the Dirichlet probability measure Dirichlet: B(S==1)- + [0, is given by
Dirichlet(Gla) ((x)) J 1=l
dx.
(2)
where B(a = (01, .2,).,,,,.))))) = I ((as)
is the z dimensional Beta function and ((x) denotes the I mi a,)
Gamma function evaluated at and G E B(S=-1), For b E N and a multi-index i = (d1, d2........................ (a) E [b] define
A = i1 Clearly, (E form a partition of [0, 1)d. For vector of weights π {mybetoy E ghu-1, the d
l2 12-1 12
id 1 id b b
x
x x
(3)
b
dimensional Histogram probability measure Histogram B((0, - [0,1] is a weighted mixture of uniform distributions on the partition sets At,, defined by
Histogram(GIr, b) = /E
Σ iE(b!
8 Th E Ar,))dy,
(4)
where G E (((0,1)))
2.2
Bayesian Histogram Model Definition
We suppose Y1, Y2,..., ; Yn sequence kn, let bn ;= 2kn, where Kn [log2(kn)], an == (Tns)iENon E Sbnd-1, For E N, the Bayesian Histogram model likelihood and prior are given by
rid ~ Po where Po E Pa- For E N, let an = fago)sem E TR4.. For an increasing
Vi..., Yn n Histogram(-IIarg bn),
n la.. Dirichlet
(5)
Also, let 2/ (- Yn) refer to the posterior probability measure over Sbad-1 derived from Equation 5 As
04,6, > 0 for every i E [b,,]d and for every N, Equation 5 induces sequence of posterior distributions over Pa- Specifically let v/a Sbd-1 + Pa be the map that takes a given {Tg)se(M) and produces its corresponding Histogram probability measure. That is
( ( Histogram b).
(6)
For a measurable set C Pas the posterior mcasure II,, is
In,k. as.. (B|Y..... Y,,) (B)
(7)
Because the conjugate prior the Multinomial distribution is the Dirichlet distribution (see for example Gelman et al. (2013). Section 3.4) and the induced likelihood over the vector of bin counts the Multinomial distribution, it straight.frward to show that.
z (-11, Yn) = Dirichlet (ab)),
where for i E [b,]]'
a Qg,b. + Σ I(Y3 E Ar...).
(8)
4

---

Published in Transactions on Machine Learning Research (01/2025)
Now allowing ab.. {I R 030d, define the sequence of estimators for Po, denoted Pa. by
Pn, Vibo {
EEsttai (Es aj dn sElom1
tbb, ...h......,........... Y.)lea(..))
(9)
where the second equality above holds if 04,6, for i [b,]]a.
We note that posterior distributions derived from improper prior distributions are not considered in this work, and therefore to consider the posterior measure sequence II, require that di,b, > 0 for i E [6,] However, allow Pn to be defined regardless of whether not the prior distribution over the simplex is proper. In particular, it is still defined in the event that some or all of the 04,b, parameters are zcro. When the prior distribution is proper, Pa has an additional interpretation: it is the Posterior Mean Histogram. In the lemmas and theorems that follow that involve analysis of the posterior distribution sequence 11,, we make clear that we require 0i,b.. > for E [b,] 1 and n N.
Pn.k,,na. (and ou,.) are indexed by the choice of kn (which determines the total number of bins) and ab which gives the prior concentrations those bins. In the subsequent subsection we establish constraints in and a that ensure Pn,k, fn,,,,00. Mon and &, are minimax statistical procedures.
3 Notions of Minimax Posterior Contraction
First introduce strong notion of posterior contraction across an entire distribution class which cannot decay faster than the minimax rate in general (regardless of the size of the space). It is for this reason we call this stronger notion of posterior contraction muniimaa--onscions PCR.
Definition 1. (minimax-conscionn PCR) Let à be distance over Pa and II some sequence of posterior distributions over Pa- The sequence en is called a minimuz-cosscious PCR for the sequence of posterior distributions II on the space (Pa,d) d) if there exists sequence zn such that zn + O and sequence 5n such that 8n S En: 2i=1 8n < oo and for some N sufficiently large, and n 2 N, for every Po E Pa, whenever
YI...., Yn Po,
sid
Po (P 2
S 2m) 21 21-5g 8n.
Now make the connection between mánémaz-ccnsccouu PCRs and minimax lower bounds. Lemma 1. If En a minimaz-conssiore PCR for sequence of posterior distributions II, on the space (Pa, d) and distance d is bounded above on Pa and mn satisfics
inf sup Ep, Ep.d(Po P) mn: mm,
P PoEPa
where the inf is taken over all function of n iid samples, then en Z mn- In particular, a minimaz--cnscioos PCR can never decay faster than the minimax rate.
Note that a minimax-conssioos PCR is a stalement about the behavior of a posterior distribution sequence across an entire distribution class. Next we make the connection between a minimax PCR and the traditional, almost sure PCR definition discussed in the introduction. Specifically, minimax-conscious PCR implies an almost sure PCR al the same rate for every distribution in the class.
Lemma If en is a minimarr consccoou PCR for the sequence of posterior distributions II on the space (Pa,d), then for every Pa Pa, whenever YI...., Yn vid Po.
IP :.::::(P..M d(P.Po) cn) + Po almost surely as n oc.
The proofs of Lemmas and are contained in Appendix 8.1 Note that Lemma 1 onlly requires that 2, < eventually and does not use that Σ/=1 8n < oc. The criteria that 2n - and Σ 8n are only used in Lemma 2
5

---

Published in Transactions on Machine Learning Research (01/2025)
4 Posterior Contraction Results
Our results utilize the following two assumptions for d E N and 2 1. Assumption 1. For n E N
12/2p d S 2p.
kn
11 n2/d
> 2p.
and Assumption 2.
1//2
d < 2p.
Σ ojinn S
nl
d 2p.
jE(b,]
Our main PCR result the following theorem.
Theorem 1. Suppose & 1 and kn satisfies Assumption and
+ log (n)
d < 2p,
en = Cold.p)
n * log
(n)
d = 2p,
(10)
12 log²
n1) d > 2p,
Now assuming that for each E N and [6m]" ag.b O, and that as, satisfies Assumption we have that for S < and d E N and Co(d,p) sufficiently large, (ddd,)) is a minimar-conscion
PCR for the
sequence of posterior distribution 1I ann on the space (Pa. Wp) where bn = 2 (ka)]
According to Singh & Póezos 2018
11 d 2p,
inf sup Er, W. (P. Pb) PoEPa
(11)
d > 2p,
where the inf is laken over all estimators P from n observations. Thus the PCRs of Theorem are up to logarithmic terms attaining the minimax rates. The assumption the prior concentrations, Assumption 2 is flcxible enough to support a vague prior. Specifically, the mcan of a Dirichlet distribution with common concentration on all categories is a discrete uniform distribution, so the practitioner wishing to encode vagueness by asserting that under the prior on average all bin probabilities are equal will want to set all prior bin concentrations to common value. When d < 2p, by Assumption [ the number of bins is x n thus Assumption
0. Likewise when d 2p, by Assumption there are bins and Assumption 2 is satisfied when all concentrations Also note that while Assumption 2 places an upper bound on the total volume of the prior concentrations to ensure the prior does not overwhelm the empirical Histogram at large sample sizes, it in general does not place any shape restrictions on the prior; in particular other prior shapes besides the uniform can be constructed.
satisfied when each concentration set to 5n +) for some
Cn-ă.
The proof of Theorem I is composed from the following three auxiliary lemmas. The first auxiliary lemma upper bounds the rate of convergence of the posterior mean histogram, Pn,k.,as, towards Po in mean W distance. The second lemma establishes an exponentially decaying upper bound the probability the WP distance between the posterior mean histogram and Po deviates from its mean by more than > 0. The third lemma establishes PCR around P Pa,kn,a, rather than Po- It is the third lemma that leverages the conjugacy of this model.
Lemma Let YI,..., Yn sid Po E Pa- Suppose kn satisfies Assumption that for and j [b,]d, ajob 2 Then
as satisfies Assumption and
1/2
d < 2p.
sup EB, (Po. P Pr.k,, ) n log(n) PoEPA
d = 2p. d > 2p.
11

---

Published in Transactions on Machine Learning Research (01/2025)
Lemma 4. Let ,I..., Yn sid Po E Pa. Suppose kn satisfies Assumption and ay.b. > for each n E N and E [bn]a. Then for S p Do and d E N and c > 0
Po (WP (Po, Pa an. Ep (Po, P P.. abn 2 S exp(-221 (odd--eeeee)
sup PoEPa
Lemma 5. Let Yi.... Yn Po Pa. Suppose kn satisfies Assumption 1 Let > 1, and let (d) be sequence satisfying
# log (n)
d < 2p:
(d(d.p) = Ciddpp)
2 log (n)
d 2p.
(12)
n à logz (n)
d d> 2p,
Then, provided that 0j.b.. > 0 for each n E N and j E [bn]d, we have that for 1 1 p < oo and N, there crists (dd p), (d. sufficiently large such that for each Po E Pd.
T alb., (P E Pa: (P,,, PR.Nn..on. 2 To (d. p)) I 2 2 (n)
almost surely under Po whenever n 2 (d. v).
The main technical challenges appear in proving the auxiliary lemmas. Given Lemmas 4 and 5 Theorem follows easily and show this now. For case in notation, through the remainder of the paper
drop the kn and or,, subscripts from the notation for the posterior, thus Un, is referred to as II, (and P PR ab..
is referred as Pa). This does not cause ambiguity in what follows because the values of kn and as, are given in Assumptions and 2
Proof of Theorem
By the triangle inequality and union bound, for each Po E Pa.
P P Pa: Wp(Po, Pa) > <n(d.p)
II, E W,(Po, P) (d.p)) s TI
2
+1, (PEPa: W (P, 2
(d(d.p)
2
= -1(
-I(MA(PO.PA>> Pa)
<x(dp)
PEEPa (,(P,Pa) (d(pp)
+ nI
(13)
2
To handle the first term on the right hand side of Equation
13 first note that by definition of en (d. (see
Equation 100 and Lemma 3. for each d E N, S 0o, we have that suprePp, EBG WT(Po. Pn) = o (d,
In particular, there is N such that for 2 N, and every Po E Pa, whenever Y ,I,...,y, Yn
Po.
(d.,p)
WP Px) EA WP (d(.p)
Po (PP(P..P))
= Po
Ep W3(P..P
2F
2
(d,,p)
< Po
WP Ph) - Ep, W (Po, P.)
20+1
< exp
(2p++ (d.p))
(14)
where in the last inequality above have applied Lemma 4 with of (d. and also that 1, we have that
(d.p)) 2
Now again using the definition
2p 2d (d. p)) S exp (-2 (2p+1)) (d, p) log(n)) S n exp (-2- nne
(15)
where the last incquality is provided Co(d, p) 2 2 2112 and thus 2 13p+ 1) - PC (d, 2 2. By Equations
and 15 we have that
sup Po (wp(Po, Pa) <n(d.p)
$ -2
(16)
PeePa
2

---

Published in Transactions on Machine Learning Research (01/2025)
for de N. 1 and Co(d,p) 2 2 2dd /2. To handle the second term in the right hand side of Equation
note that setting Co(d, p) 2 2C ( p) we have that Tn (d. I 5u(d_) for every p 2 1, d E N where Tn (d, p) is
2
as defined in Lemma 5 Using this and Lemma 5 we have that for every p 2 1, d E N, an N(d.p) such that for each Po Pa, when YI..... Yn aid Po.
s(d.p)) 2
11
S II Pa: Wa; Fm
(dd - 2 22log" "log">(0) (n)
(17)
with probability 1 under Po when 2 pd..)). By Equations
13 16 and 17 we conclude the existence
of sequence 8n x n-2 and single N(d, such that for N (dd,p) and each Po E Pd, whenever sid Y Vi...., Yn ~ Po.
Po (T,, (P (Po, P) en (d) S 2 log (
(n) > 1 on
(18)
for d E N.p 2 as long as Co(d.p) 2 max (d. 2 22 Finally note that 2 (n) + 0 since > and also د5= on < oo and on = olen (d. p)) since 8n x n The theorem statement thus follows. j
In practice, the Posterior Mean Histogram, introduced in Equation g and shown to achieve the same rates for estimation as the posterior itself in Lemma 3 (after application of Jensen's 's inequality), can be used the functional point estimator for the unknown distribution.
5 Using Prior Information
As discussed in the introduction, minimax optimal posterior contraction results are an indicator that a Bayesian method (in the worst case risk sense) robust to the selection of an inaccurate prior when the sample size is large.
The general benefit of the prior in scenarios where the practitioner has a belief about Po before data collection, is able to encode this information through the prior, and the belief ends up being correct, in which case the small sample size performance can be better than under purely frequentist estimation approach.
The type of belief that is representable through the prior in this model is hypothesis about the probability distribution of partition of [0, 1jd. Specifically, suppose the practitioner believes, but not certain, that Po satisfies that on size M partition of [0, 1] denoted { the probability in region R, is py for j [m]. Further suppose there exists kn E (1, sufficiently large such that there exists size M partition of (i]eeptoje, denoted [4
LM and for each j [M]
R2 U iEI
Ai,2
where recall A4,2 is defined in Equation 3. In words, each region R2 is expressible as unions of members of the level ko dyadic partition. Note that because R, is expressible as union of members of the partition (A hie and the partitions {A1,2 hee[294 for k 2 ko are nested in the (Ai Jigps partition, R2 is also expressible as a union of members of {A4, ]EEp29 for k 2 kp- In particular for k 2 ko, there is a size M partition of {i)septojs, denoted { and for each j E [M], Rg = Uler, A We recommend the following procedure for incorporating prior knowledge in these circunstances:
When kn of Assumption I satisfies [log2(k,)] < ko, reset kn := kp- Otherwise use knp as in Assump- tion 1 This small sample size analysis docision has no impact on the large sample (asymptotic) results of Theorem [
2. For a user specified C > chosen prior to data collection and for each j E [M] and E Iykk.. set Cp.
Og,b,, 11
8

---

Published in Transactions on Machine Learning Research (01/2025)
Following this procedure, the total prior concentration Rg amounts to Cpy. Note, by choosing C' prior to data collection, C' cannot depend on the potentially growing sample size. So for every n, we have that Eie(o.]" Gj,b, = C' and in particular Assumption 2 is satisfied. Further, by Equations and g the Posterior Mean Histogram, Pn, satisfies that for each j E [M)
EL D E Ap,b.) + Cpi
Pr (R)) Σ (A E
Ei= E(EEER)))) ( (n+c)) (n+c) tDS (xfc):
C
n+C
KETs.kn
GET,AAm
This is a convex combination of the empirical proportion and the prior proportion in region Rg for which the weights depend only on the relationship between C and n. C should be chosen larger under high confidence prior beliefs and smaller under low confidence prior beliefs. A simple question to elicit C' for the practitioner ask themselves: for a given small weight < 90 1, how many samples no would need be available for them to assign this small weight to their prior beliefs and 1 go weight to the empirical proportions? tc go and solving yields choice for C.
Setting C
6 Proofs of Auxiliary Lemmas
In this section we prove Lemmas and First we need to state couple of technical tools.
6.1 Technical Tools
The first tool the multiresolution upper bound on the Wasserstein distance. See Weed & Bach (0119
Section or Singh & Póezos (2018) Appendix A for good review. Here we use an application of this general result for the metric space ((0, 1 II 112).
Lemma 6. (Wasserstein Multiresolution Upper Bound) Let So = [0, 1)d and for k E E,,
{
11 2
1
12
12 x
id id for (11, .e.....,dd ia) E (241
S =
x
x
2
2
2
2
2
If μ. are probability measures on [0, 1)d, then for 2 1 and K any posilive integer,
((5)
Kp
(k-1)p 1)p
WF(m) 0012
Σ
2
SES.
Proof. This is straightforward application of Proposition 1 of Weed & Bach 2019
The next technical tool is upper bound on the L1 concentration of Multinomial distribution around its mean.
Lemma 7 (Multinomial concentration). (X,...... ..., X&) Multinomial n. P1y....pk) and 7 = Σ/=1 npyl, then
k
B(Z/n) S
Proof. Applying Jensen's inequality and then Cauchy-Sehwan
7 E
X Var
1
- Pj Pj)
(1 p,)
n
n
Vn ==1
Vn
n
f=1
j=1
j=1
The last tool is the concentration of the Dirichlet distribution around its mean in the La norm.

---

Pr (Aj...) By definition of Published in Transactions on Machine Learning Research (01/2025)
Lemma 8. (Dirichlet Concentration) Let lo N and (T1, "2,.... T&)) Dirichlet(a1, 02... az). .z)..., Then for
8 >>>
(a) (Q)=Av
IP
Σ " R(TT)]
S 0
where ā Ez
Proof- Basic properties of the Dirichlet distribution give that for j (1,2,...,k), J, Betala,, -oj). Also, if X Beta(a, (3 then Var(X) aB/((a + 8) + 3 1)). Using these properties, in addition to Jensen's 's inequality and Cauchy-Schward inequality, we have that
E
Σ F B(TT))]
5
Var(Ty)
aj(a aj)
a² (1+1)
$ (a)
V aā ag)
$ (a)
Eas
Σ ā za-aj aj
(a) Vālāk- ā)
< (の) -o)
(19)
By Markov the result follows.
6.2 Proof of Lemma 3
We now prove Lemma
Proof of Lemma 3 By the definition of the partitions {SkJAen in Lemma the partitions St, S2,... Sk. are nested the sense that for e [K, - and Sg, there exists a collections of sets in S such that S is exactly the union of these sets. In particular the sets S E Sş are expressible as unions of sets from Sk Also using that bn 2K,, we have for k 21,,2..... Kn} and Sg of indices Is,k. S [6,]] such that
S
U Aj.
(20)
jElSAnm
where recall Aj.a defined Equation 3 Moreover {UJETS.... Ag. Jss, partitions [0, 1)d and (Is... Ises, partitions [b,] d Using this and Lemma 6 (the Wasserstein multiresolution upper bound), we have that
KaP
(A- 1)p
EA WP P.) p S
1 2
E
Σ SESa
Pa( U
Ag...) PP(S)
(21)
JEls.um
Since the Aj sets are disjoint, for S E Sk, Po (UJETEAn Ag..n) Ejetsk.
Pr, (see Equation we thus have
ay,b + En I(Y E Ag.o.) Po(S)
PA( U Ayr..) Po(S) -o(.)) jETgA..
Σ
DiE Og
jETs.m.
10

---

Published in Transactions on Machine Learning Research (01/2025)
Ejelsk oj + Ei=1 I(Y E USSES.... Ag.bo) Po(S)
n + Zie [bn] Og,b.
2 I(Ye E 5)
EjElsun ag.bn
Po(S)
n + Le ai,ba
n
n2 n Eielon 04,bn
Σ I(Y, S)
Ei= I(Y, S) Po(S) (22)
n + EiElbn] Og,ba.
n1
n
Ejels ag.b
Eie(b.)" O4,b,
Using this and that Sy partitions [0, 1) (and therefore # Eses, 1 10,,1((,,,, E S) = 1) and Lemma (Multinomial concentration) yields
Eje(on]a Ugo0 +n VSa
Ep. SESu
Σ
Pa( U -
2 n Lie(th)) Okg.bn
jETSAnm
Using this and Equation
21 (and that |Syl = =dk and Kn 2 (ogg((in)) we have that
K.P
Eiglon as,on
(k-1)p
Epp WP,,, Pw) x
2
+
n EiElon]" Or,
②
+
2 k(p-$)
Eic(b.] Ogg
km +
n (max 2 p) [Td 2p) + Kn,T( = Kn,[d==pp) 2p))
(23)
+ Lie(b.]" Obg.bn
Applying Assumptions and now allows to conclude that Ep, P.). S n- when d < 2p, - log(n) when d 2p and S n- when d 2p. The I arguments of this proof do not depend on Po (through constants or through eventuality) and so we conclude
n 1/2 1/2
d < 2p.
sup Ep, (Po, PA) PoePa
p/d log(n) n ppd
d 2p.
d 2p.
6.3 Proof of Lemma 4
We now prove Lemma
Proof of Lemma We will use the general dual formulation of W. distance and then apply Mediarmid's inequality (McDiarmid et al. (1989); see Proposition of Combes (024) for the statement of the Standard Mediarmid inequality). Towards this end, define for ,t, v E Pa
= -d1,,,,
(24)
WT (M, v) the optimal transport cost between μ1, V under the rescaled Polish space ((0, () d 1/2 with optimal transport costs being distance raised to the thh power. Also the diameter of [o, 1)d in the rescaled space is [3] Thus letting
l1
C := (flf 1) [0, 1). continuous w.r.t d d
&eesaaling the space have diameter is technically nescessary apply the dual formulation of Wp but we da here so that only need account. for the diameter the space interest end proof
11

---

Published in Transactions on Machine Learning Research (01/2025)
and for f E Co letting f (0, + R be the function satisfying
(v) = sup =E(0,1)
812))
(25)
the conditions of Weed Bach 2019 proposition 19 are satisfied and we have for μ4 v E Pa.
WF((, v) = sup E, - Ey SEC,
(26)
We now prove the bounded difference inequality nescessary to apply Mediarmid's inequality. So let nn F2, Fn E [0, 1) and x = (21, (2.... In In,1, (0,1)) Additionally we use the notation
WP(Po, (x, indicate that P. constructed from [0, 1),, and let js [b,]" and jæ [6,] satisfy that In E bja,b, and zn As and let Cj.b, Ei= I(x; Ag,b,) for j [b, Via equation 26 and expressing expectations sums over the partition members Aj.., we thus have that
W, (Pn()n), W, (Pa(="))
aj.br + -Cgbi
I f(z)d
Σ by jE[bn]
-Ex~Ro((X E Ason))((()
sup sEC
n Lie(on)" Ohi
Ag.on
sup MeC,
Bdl n Eielonie Okeiiin
Og.bn + -Cgsnn
Jayn f'(z)dx
Exari E Aj. (27)
ajaba + +jeebn - IIj + n n Lie(h..' O4
bd
f(xx)d Ex~poIXX E Af
Assión
by Ojrr,bn + -Caarin + T( F fmc)
n Lie(bu," Oii.b..
We write the above equality more succinctly as
W (Pn) Po) Wr (Pa( sup (Ji(f)+ sup (Ji(") + Js(f)), SEC,
(28)
r'E,
where Ji C1 R, J2 Co R, J3: : Co + are defined
Ji(f) = Σ byd Og..b. + Cj.b
Ese(on.) Oti,iin / f(x)dx - -Ex~Po1(X E (x),
Ag
and
J2(f) Σ by
n + DiE(bn] Ori,bn J.Ag.n on /
aj.b. + Cj.bb
f(x)do)
-Ex (f
sE(Dedart
and
aja.h. + CCj.hh T(j # $zx)
Jo(f) = by
f(x)dx - Ex-rol(X E (f
n n +IEE(on,1 Ohicbin
(29)
Ajarbn CCj.r dn + I + goc)
JAp Asseion f(x) - EXXPOTTX Af
n + Eiep..) O4i.bn
Now observe that by adding and subtracting (f(
sup J1()) + J2(f) < sup J1(f) Jz(f) + sup Jz(f) Jz(f).
(30)
SEC,
SeC.
SEC,
12

---

Published in Transactions on Machine Learning Research (01/2025)
Using this and Equation 28
have that
W, (P,(z)), W (Pr(e,,), Po) I sup Jz(() Js(f).
(31)
SEC.
In the difference J2(f) Jz()), the expectations involving f cancel and thus
w (Pn()n), Po) W/ (z')), Po)
/
Og + Cy.ds
Ogr.bn Crr.bn - 1(5= # fo'
$ sup sECs
6n
f(z)dz
Yi + Lie(t..) Oii.bn.
n + EiElo Oi
Assien
/
Oj...bn + Cjaria n1 + Eielonia Oiibi J.
ajar bn + Cjrr,bn + +IJ 1 fzx)
Un
(ax)dd
(32)
+ Dielbnia Ose,bi
Agarion
b # fw
sup
f(z)dxx
f(z)dx
SEC, n + Eielb.. (b.] Ors
Agxiin
Aggiinn
1
n
where note in the last inequality we have used that C. consists of only non-negative functions bounded above by 1, and that for each j E [b,]%, Say.n. dx 1 W (an)),), Po) < 1/2 and so conclude that
By an identical argument, W (Pa()))) Po)
MV, (Pa(2)))P)) - W, (Px()))) (...)..)) Po) S
1 n
Since s/ (0,1) was arbitrary the bounded difference condition in the last coordinate is satisfied with bound 4. The argument for the other 1 coordinates proceeds identically and so conclude by Mediarmid's inequality that for e > 0
Po WP Po) En, W (Pa, Po) 2 2c) S eexp(22ne"
(33)
Finally recall for all p.v& E Pa, Wp(pev) d'/2w, (1, Using this and Equation 33 and that Wp is symmetrie the lemma follows.
6.4 Proof of Lemma 5
We now prove Lemma
5
Proof of Lemma 5 The first inequality we present below upper bounds the WP distance between histograms using the multi-resolution upper bound. So again using Lemma and the sets Is from Equation 20 we have for n E N. N1, E Sb.d-1
K.p
(k-1)p
WVo. (TT), 1h.., (Tz)) 2p/2
|/bb,, (II) 1/o., (Tz)
2
1=1
SESa
Kwp
K
(k 1)p
/
Σ
babb (TT)(
U
Ag...))
Vo. (T22
U
2
k
SES,
jETg.km
jETs.
Krp
K,
(k- 1)p
/2
Σ
Mi
T
k=1
SESa
jEls,
jEls.u.
(34)
13

---

Published in Transactions on Machine Learning Research (01/2025)
Also, for W1 Shad-1 and & E [Kn], define
(T), =
Σ
Σ
E Jels,a.
E2g (Tg]Vi,..... Yn)
(35)
SES, jefs,...
Using Equation 34 the definition in Equation 35 the preimage form of I. (Equation 7 the definition of P.
[Equation 9), the definition of z (the posterior measure over the simplex Stud-1), and that by Assumption
and the definition of Ty (d.dp)) p) in Equation 12 2 K,P ( p)) (as + oc) we have that almost surely
under Po and eventually in and for each d E N
[P, E Pa: Ww) 2 Tr (d.
( E Shud-1,
:WP (T 1), Do, (Ez; (T MTTYY,,-.. Yn))) 2 (d,
S 2/
T1 Shad-1
+
2(k (T 2 d d=
2kaP
(36)
K
S zm
#1 E Shad-1
1
Vn (T( 2 2 // 7 (d,
2( 1)p
k=l
Kn
2n E Shud-1
Vn (T k)) 2 28 p/2 p [d
2
We will now derive an upper bound on 6S d-1 |a (T. 1 k) which, if it holds, ensures the event inside the probability of the last line of Equation 36 must be false. We will use this upper bound and the union bound to control from above the probability in the last line of Equation 36 bound on sup ES
Regarding the upper
maxje(K.l Vn (n k), note that
Ka 2 2-kp
log" (n)2
log" (ng)(n)
(n)
Kn
kp
2 k(p 5) $ <d..d)
(37)
Vn+ Eiep,,) ag.b.
Vn+ Eis(o 04,bn.
k=1
To see the $ in Equation 37 observe that by Assumption 2 the total prior concentration is dominated by and therefore the term in front of the summand LHS of S is x log Thus by definition of (d. (Equation 12 this is sufficient to conclude LHS I (f p) in the d << 2p case. In the = 2p case the sum contributes a factor log(n) to LHS and so again LHS S (d), In the 2p case, the sum contributes an asymptotic factor 2K.(4 x An nt to LHS so that LHS (n)m and so again we C₁ (d,,,)) sufficiently large so that eventually
/n
LHS $ p). By Equation 37 for each N. p in
log 2og7(n) (n)2
Σ 2 2-hp
2D-1 d 1/2 (d.
(38)
Vn+ Eielo.) Oiihh,
The upper bound on sup suPr,Esend-1 maxxe/K, (T k) we are seeking is thus
log" (n)2
n + EtE Citb
Thus with (id, p) sufficiently large so that Equation surely under Po
38 holds, we have that eventually in n and almost
Kn
2/ ı E Shnnd11
Vi(TII k)) 2
20 d 2--1d- 2/2 p (d.
2kp
14

---

Published in Transactions on Machine Learning Research (01/2025)
2
kn #1 E Shndd1 (1, s.t (T(T1, k) > log" (m)
n + 2 [ Oj
K
2dk
Shnd-1, (x( > Ing
(39)
Ej aj.b.
where in the last line we have used the union bound. Now recall (Ig,,a partitions [b,]] for k E (1,2,..,,n}} Kn). In particular, since zn = Dirichlet(- birj, Isee),)) under zno (Ejetsmn Tyl Dirichlet (1E Finally note that by definition of Sk, [Sx] 2dk So for and k E (1,2,..., Kn applying Dirichlet con- with 8 == log ((n) we have that for Ci(d,p) sufficiently large, eventually
1ses,) Moreover, Eses, Ejels.... Jj = Ejep, aÿ.02 n *n ag.b..
centration of measure Lemma | in and almost surely under Po
Kn on
2dk
m Shudd11 (2 k) log" (m)
V n Σ [je( Oj.bn
I K, [ogg((())
(40)
Finally note Kn [lg(kk)) I log(kn) + < 2 log(n) and the asymptolic arguments of this proof do not depend on Po either through constants or eventuality. Thus by Equations 36 each d E N and p 2 and C1 p) sufficiently large, and an (d, (depending only on d, but not on Po), and n 2 N
39 and 40 we have that for
TT Pa: (P,,PP) (.) s 2log'
(41)
almost surely under Po-
7 Conclusions
In this work we obtained minimax optimal PCRs for unconstrained distribution estimation on [0, 1] under- neath the Wasserstein-p distances for every data dimension d. To the best of our knowledge these are the first PCRs achieving minimaxity for every problem dimension under Wp,P 2 distance. Our proof technique avoids verifying a Kullback-Liebler prior support condition by using conjugacy and direct analysis of the posterior distribution.
These results may be useful to practitioners needing to estimate distribution underneath Wasserstein distance when they have some knowledge prior to data collection about the shape of the distribution they are estimating, intend to encode this through a prior distribution to potentially achieve increased accuracy at low sample sizes, and yet simultaneously require a guarantee of precision at large sample sizes that is robust to inaccurate prior selection.
An important area for future work to determine whether for high dimensional data, Bayesian models can adaptively achieve minimax optimal PCRs underneath Wasserstein-p distances in constrained distribution estimation settings where it is safe to assume that the distribution to be estimated is of low entropy has smooth density.
8 Appendix
8.1 Proofs of Lemmas Regarding Minimax- Conscious PCRs
Mininax-Conssious
In this section we prove the lemmas that relate PCRs to minimax rates. The first proof is inspired by Ghosal
et al. (2000) Theorem 2.5.
Proof of Lemma 1 Define Hn Pa + [0. as
H. TT, E Pa d(P,)) (())
15

---

Published in Transactions on Machine Learning Research (01/2025)
and
7n == sup (Q) QEPa
and define the n estimator Pn to satisfy (P,)) > Zn 1. recall n is a function of the sample ,,,..., Yn and therefore so is Pn. By the lemma assumption, and since 2n < 3 eventually, there exists an N such that for n 2 such that for every Po Pa
Po (Po) 2/3) 2 6n
(42)
And since for Po Pa. the event Hn > implies Z, Y/2 and in particular H,((a) > 2. we also have that for 2 N and every Po E Pa
!
n (Po) > 2 2 6n
2
Po (P,) > 2
(43)
3
Now note that by definition of H, and since 1I, is probability measure, for every Po.
2 (Pn) _ n (n((P)) > 3
[ E Pa: max(d(Po, Q), d( Q)) en
Using this and Equation 43 and triangle incquality, for n 2 N and each Po E Pd.
Po(d(Po. Pn) < 2c,)) 2 8n Using this and that d is bounded above on Pa and that 8n S en by assumption, we have that for n 2 N
(44)
S en + on S en sup Ep. Pw)
(45)
PoEPa
Thus the lemina statement now follows by the definition of the minimax rate.
proof of Lemma Let Po Pa. By the lemma assumption, we have that
80
Po (In : 2 en) 2n)
8n < 00
(46)
n=1
By the Borel-Cantelly Lemma conclude that with probability 1 under Po, it is eventually true that
I. Pa å(P, Po) 2 Gn) 2n
Since 2n + 0, we conclude that almost surely under Po
II, E Pa: d(P, Po) 2 +
+
In the introduction we claim that an almost sure PCR holding uniformily over a finite distribution class cannot decay faster than the minimax rate for that class. While we do not actually use this claim our work, for completeness, proof of this claim is provided below. This lemma is also inspired by Ghosal et al. (2000) Lemma 2.5.
Lemma 9. Let d), be finite metric space of probability distributions, and II sequence of posterior idd
distributions over C. Further suppose for every Po C, whenever Y.....,,, Yn.... Po.
T å(Po, P) 2 (.) 0 Po almost surely. Then there erists a sequence of estimators P., E C, such that
Ep, d(o. Pr) 2en
PbEC
16

---

Published in Transactions on Machine Learning Research (01/2025)
for sufficiently large In particular if 2n satisfies
inf max Ep P) 2) an P.ec
where the inf is taken over estimators P based on observations, then en kn.
Proof. Define
H - - [0,1]
(Q) = TT d( < (n)
and
Zn max (Q)
QEC
=ar and let Pa = arg mac Hn (Q). For Po E C, since
T. C: d(Po. P) 2 6n) + Po almost surely,
we have with probability under Po there is an N(P) such that for n 2 N(P))
(a 1
In particular (Px))) as well. Since 1I, is a probability measure and with probability under Po, when 2 N(Po), min(H, (P,), H., (Po)) 1, have that
(PEC: d(P,
< { #
with probability under Po for n 2 N(P)). Letting Q be any point common to both these sets, we have that
ä(Po, P.) < d(Po. Q) a(Q, 2En
with probability 1 under Po for n n N(P)). In particular, Epo d(Po. Pa) s 26, for n 2 N(Po). This argument holds for all Po E C, so letting max N(P), we conclude that for n 2
max Er d(o. Pa) 26n
PoEC
9 Acknowledgements
This work was supported by contracts DOE DE-SC0021015, NSF CCF-2115677, and the Laboratory Directed Research and Development program at Sandia National Laboratories, multimission laboratory managed and operated by National Technology and Engincering Solutions of Sandia, LLC, a wholly-owned subsidiary of Weleywell International, Inc., for both the U.S. Department of Energy's 's National Nuclear Security Ad- ministration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressod in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. The authors further acknowledge and thank both Jeff Phillips and the reviewers for helpful conversations.
17

---

Published in Transactions on Machine Learning Research (01/2025)
References
Sergey Bobkov and Michel Ledoux. One-dimensiona empérical measures, order stotistics, and Kantorovich transport distances, volume 261. American Mathematical Society, 2019.
Federico Camerlenghi, Emanuele Dolera, Stefano Favaro, and Edoardo Mainini. Wasserstein posterior con- traction rates in non-dominated bayesian nonparametric models. arXiv preprint orXio:2201.12220 2022.
Minwoo Chac, Pierpaolo De Blasi, and Stephen G Walker Posterior asymptotics in wasserstein metrics on the real line. Electronic Journal of Slatislics, 15(2): 33635 -333--3777, 2021.
Richard Combes. An extension of mediarmid's inequality. In 2024 IEEE International Symposium on Information Theory (ISIT), pp. 79- 79-84. IEEE, 2024.
Nicolas Fournier and Arnaud Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3-4):707, 2015.
Fengnan Gao and Aad van der Vaart. Posterior contraction rates for deconvolution of dirichlet-laplace mixtures. 2016.
Andrew Gelman, John B Carlin, Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin. Bayesian Data Analysis. CRC Press, 2013.
Subhashis Ghosal and Aad Van der Vaart. Fundamentals of nonparometric Bayesian inference, volume 44. Cambridge University Press, 2017.
Subhashis Ghosal, Jayanta K Ghosh, and Aad W Van Der Vaart. Convergence rates of posterior distributions. Annals of Stalistics, pp. 500 531, 2000.
Willem Kruijer and Aad Van der Vaart. Posterior convergence rates for dirichlet mixtures of beta densities. Journal of Statistical Planning and Inference, 138(7):1981-1992, 2008.
Colin McDiarmid et al. On the method of bounded differences. Surveys in combinatorics, 141(1):148 188, 1989.
Judith Rousseau and Catia Scricciolo. Wasserstein convergence in bayesian and frequentist deconvolution models. arXiv preprint ar arXiw:2309.155: 15300, 2023.
Catia Scricciolo. On rates of convergence for bayesian density estimation. Scandinavian Journal of Statistics, 34(3):626 642, 2007.
Catia Scricciolo. Bayes and maximum likelihood for 1-wassorstein deconvolution of laplace mixtures. Statistical Methods & Applications, 27(2))333 362, 2018.
Weining Shen, Surya T Tokdar, and Subhashis Ghosal. Adaptive bayesian multivariate density estimation with dirichlet mixtures. Biometrika, 100(3):623 640, 2013.
Shashank Singh and Barnabás Póczos. Minimax distribution estimation in wasserstein distance. arXio preprint arXio:1802.088: 08855 2018.
Jonathan Weed and Francis Bach. Sharp asymptotic and finite-sample rates of convergence of empirical measures in wasserstein distance. Bernoulli, 25(4A):2620-2644 2019.
Rui Zhang, Christian Walder, Edwin V Bonilla, Marian-Andrei Rizoiu, and Lexing Xie. Quantile propagation for wassersteinn-apprxiinatt gaussian processes. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21566-21578.
Curran Associates, Inc., 2020.
18


Reviews:
[{"review_id": "pB8wZEcty2", "reviewer": "Reviewer_feWi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths \n\n-  Theorem 1 does not put constraints on $P_0$  or strong assumptions on the shape of the prior distribution, and I find Assumption 2 reasonable.\n- I think readers would find the both the main result (Theorem 1) and the analysis in the two supporting lemmas (Lemma 1 and Lemma 2) interesting. \n- The authors communicate a clear strategy for proving the result through the text. \n\n\n### Weaknesses \n- Perhaps because of page limitations, some of the upper bound derivations in the proofs of Lemma 1 and Lemma 2 carry around a lot of symbols and indexes, which makes it a bit hard to follow the steps taken. I think it’s important to revise those a bit to make it easier for the readers, as those two lemmas reveal a lot about Theorem 1.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors derive minimax optimal posterior contraction rates (PCRs) under quite general conditions and for any number of dimensions or Wasserstein-$u$ metric (that is Theorem 1, which is the main contribution of the paper). The authors consider the unconstrained estimation case where we wish to approximate some $P_0$ probability measure on $[0,1]^d$ given a set of samples. No other assumptions are placed on $P_0$. To set up a posterior for study, the authors use a Bayesian Histogram model that partitions $[0,1]^d$ into equal area squares, where a sequence $k_n$ determines the number of bins and n denotes the number of data points. A multinomial likelihood and Dirichlet prior combination results in a Dirichlet posterior (due to conjugacy). Then the authors induce concentration assumptions on the prior and specific rates of increase for $k_n$ and with those (and the conjugacy) show that the PCRs of the proposed model are, up to log terms, the minimax rates from Singh and Póczos (2018).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "# Clarifications for acceptance \n\nI would appreciate clarification of these points by the authors, either in discussion and/or in edits of the paper. \n\nWhile I could follow most of the proofs, the final proof of Lemma 2 takes some effort. I suggest:\n\n1. When constructing the upper bounds in Equation (23), most terms don't change in the long expression. Consider describing before Equation (23) what is going to happen or otherwise refactoring the presentation of the bounds by assigning terms that don't change to some shorthand (see next note).\n\n2. Similarly, around Equation (26), it is difficult to follow the steps leading to the final upper bound. One idea would be to use a temporary expression, e.g., \n\n$$V(\\ldots):=\\sum_{S\\in S_k}\\left | \\sum_{j\\in I_{S,k,n}}\\ldots\\right |,$$\n\nto clean up the wider expression (as that part does not change during the derivation of the bounds) and focus the reader on what actually changes. The authors have the space to even attach a number to this expression to make it easy to refer to. This expression could then be reused to simplify Equation (27) as well.\n\nGenerally, I suggest rewording a bit Lemma 1. and Lemma 2. to make them easier to parse. \n\n# A few suggestions that would help with presentation, but are not essential for acceptance\n\n1. Clarify whether the expectation under Equation (12) should be $E_{P_0}$ or $E_{p_0}$ (the difference is the capitalization of $P_0$). Same for all following expectations in Proof of Theorem 1. This appears to be a typo.\n2. Capitalize all proper nouns in text, e.g., \"equation 1\" → \"Equation 1\", \"proof of lemma 1\" → \"proof of Lemma 1\", etc.\n3. Define what $a_n\\lesssim b_n$ means in Section 2.1 to help readers not immediately familiar with the notation.\n4. In Equation (14), consider whether cancelling the terms on the right of the $\\lesssim$ symbol, e.g., $n^{-1/2u}/(n^{-1/2u}\\log^{\\gamma/u}(n))=1/(\\log^{\\gamma/u}(n))$, etc., would help the reader more quickly see the limit behavior.\n5. Under Equation (18), refer to the equation where $A_{j,b_n}$ are defined, e.g., \"where $A_{j,b_n}$ as defined in Equation (3).\"\n6. The multi-resolution lemma (Lemma 3) does not define the K constant; a statement like \"Let K be a positive integer\" would be sufficient.\n7. In Section 4.2, after \"Proof of Lemma 1\", the authors state \"Due to the nesting of the dyadic models\". I appreciate that this is a standard term for mathematicians, but it probably would be helpful for the reader if the authors explicitly stated which are those models.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4KBSx5A0pR", "reviewer": "Reviewer_9k4x", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- The rate derived under no assumption on $P_0$ seems new and optimal\n- The Bayesian histogram technique seems to be original\n\n**Weaknesses:**\n\n- The paper is generally not very well written and hard to follow.\n- It is also very technical. While it might interest some people in Optimal Transport, it is a paper which might interest more a statistic audience.\n- There is no experiment showing the rate.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper derives a minimax posterior constraction rate for the estimation of a distribution $P_0$ on $[0,1]^d$ under the $p$-Wasserstein distance for $p\\ge 1$. This rate is derived using Bayesian histograms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To be honest, I am not an expert in statistics, and even less in Posterior Contraction Rates, so I will try to give some hints on what could be improved. But I probably did not understand a lot of things.\n\nWhile the content of the paper is technical, it might be possible to make it more friendly to read for a broader audience, e.g. by adding more background on the previous related results and what have been done. For instance, the beginning of the second page describes related works, but is very hard to follow. \n\nIt would also help the reader to have a concrete example of an estimator. I personally did not understand well how to estimate in practice the distribution P using this method.\n\nPlease improve the abstract. The two first sentences seem basically to be the sames. \n\nAlso using the Wasserstein-$v$ metric does not seem very standard, as usually $p$ is used.\n\nTypo:\n- \"practicioners\" in Conclusion.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "64phHSm99t", "reviewer": "Reviewer_iaVK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- the paper is mathematically rigorous, concise and straight to the point. Its contributions are clearly stated and its proof well organized.\n- I do not know about the PCR formalism, and I could understand it from this paper alone; I learnt interesting facts and tools by reading this manuscript.\n- The proposed estimator is quite natural (a Bayesian Histogram with adaptive bin size)  and it is potentially useful for the literature to have an analysis of this estimator for the Wasserstein distance, in arbitrary dimension. It is also interesting to know that it is minimax (and the conditions on the prior parameters that are optimal).\n\nWeaknesses:\n- It is not clear to me what is the advantage of the proposed estimator compared to the \"empirical distribution\" estimator which already achieves the minimax rates (up to log-terms). From the point of view of the Wasserstein metric, there does not appear to be any advantage of this estimator. (But their must be other, stronger, metrics on the space of distributions that favor this more regular estimator).\n- I had difficulties understanding what was precisely done in prior works from the literature review in Section 1. This section seems more concerned about explaining what was what *not* done in prior work (each mentioned paper is quickly followed by \"... but they do not ...\"); this matters, but a more \"positive\" overview of the literature would be useful to non-expert readers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the nonparametric estimation of a distribution on $[0,1]^d$ ($d\\geq 1$)  in the Wasserstein distance $W_p$ $(p\\geq 1)$ given $n$ iid samples. The authors use a Bayesian framework: the proposed estimator is the posterior under a Bayesian Histogram model, with bin weights drawn from a Dirichlet distribution ; whose parameters should be specified by the practitioner, for each $n$. This paper shows that this estimator achieves the minimax rates, in the sense that the upper bounds on the asymptotic estimation rate in $n$ shown in the main result (Theorem 1) match the known lower bounds up to logarithmic terms (due to Singh & Póczos (2018)). The upper bound are formulated in the form of PCR (Posterior contraction rates) and the proof technique relies on the multiresolution upper-bound technique  (due to Weed & Bach).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Some of the vocabulary and facts from Bayesian statistics would deserve a bit more pedagogy to the benefit of the general TMLR reader. For instance: p3. “Due to conjugacy, it is straightforward to show that” (include reference, explain which distributions are conjugate to each other). Also more details on the motivation behind the PCR formalism and its exact definition would be welcome (e.g. the \"slowly increasing\" sequence $M_n$ in p1 was quite mysterious to me).\n- The minimax rate in expectation (Eq.(11)) and the PCR rates of Thm. 1 are not stated in the same formalism. Can you explain why they are indeed directly comparable? In particular, does Eq. (11) indeed give a bound on the fastest PCR rate achievable? This sounds reasonable but deserves justification, in particular because the minimax claim is in the title of the manuscript.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have adjusted the manuscript to the reviewers' satisfaction during a productive and collaborative feedback period. All three reviewers are happy with the latest version of the manuscript.
Decision: Accept (poster)