Title: Optimality in Mean Estimation: Beyond Worst-Case, Beyond Sub-Gaussian, and Beyond $1+\alpha$ Moments
Venue: NeurIPS 2023
Accepted: yes
Keywords: mean estimation

instance optimality
Abstract: There is growing interest in improving our algorithmic understanding of fundamental statistical problems such as mean estimation, driven by the goal of understanding the fundamental limits of what we can extract from limited and valuable data.
The state of the art results for mean estimation in $\mathbb{R}$ are 1) the optimal sub-Gaussian mean estimator by [Lee and Valiant, 2022], attaining the optimal sub-Gaussian error constant for all distributions with finite but unknown variance, and 2) the analysis of the median-of-means algorithm by [Bubeck, Cesa-Bianchi and Lugosi, 2013] and a matching lower bound by [Devroye, Lerasle, Lugosi, and Oliveira, 2016], characterizing the big-O optimal errors for distributions that have tails heavy enough that only a $1+\alpha$ moment exists for some $\alpha \in (0,1)$.
Both of these results, however, are optimal only in the worst case.
Motivated by the recent effort in the community to go "beyond the worst-case analysis" of algorithms, we initiate the fine-grained study of the mean estimation problem:
Is it possible for algorithms to leverage *beneficial* features/quirks of their input distribution to *beat* the sub-Gaussian rate, without explicit knowledge of these features?

We resolve this question, finding an unexpectedly nuanced answer: "Yes in limited regimes, but in general no".
Given a distribution $p$, assuming *only* that it has a finite mean and absent any additional assumptions,
we show how to construct a distribution $q_{n,\delta}$ such that the means of $p$ and $q$ are well-separated, yet $p$ and $q$ are impossible to distinguish with $n$ samples with probability $1-\delta$, and $q$ further preserves the finiteness of moments of $p$.
Moreover, the variance of $q$ is at most twice the variance of $p$ if it exists.
The main consequence of our result is that, no reasonable estimator can asymptotically achieve better than the sub-Gaussian error rate for any distribution, up to constant factors, which matches the worst-case result of [Lee and Valiant, 2022].
More generally, we introduce a new definitional framework to analyze the fine-grained optimality of algorithms, which we call "neighborhood optimality", interpolating between the unattainably strong "instance optimality" and the trivially weak admissibility/Pareto optimality definitions.
As an application of the new framework, we show that the median-of-means algorithm is neighborhood optimal, up to constant factors.
It is an open question to find a neighborhood-optimal estimator *without* constant factor slackness.

Optimality in Mean Estimation: Beyond Worst-Case, Beyond Sub-Gaussian, and Beyond 1 + α Moments

Abstract

There is growing interest in improving our algorithmic understanding of fundamental statistical problems such as mean estimation, driven by the goal of understanding the fundamental limits of what we can extract from limited and valuable data. The state of the art results for mean estimation in R are 1) the optimal sub-Gaussian mean estimator by [Lee and Valiant, 2022], attaining the optimal sub-Gaussian error constant for all distributions with finite but unknown variance, and 2) the analysis of the median-of-means algorithm by [Bubeck, Cesa-Bianchi and Lugosi, 2013] and a matching lower bound by [Devroye, Lerasle, Lugosi, and Oliveira, 2016], characterizing the big-O optimal errors for distributions that have tails heavy enough that only a 1 + α moment exists for some α ∈ (0 , 1) . Both of these results, however, are optimal only in the worst case. Motivated by the recent effort in the community to go 'beyond the worst-case analysis' of algorithms, we initiate the fine-grained study of the mean estimation problem: Is it possible for algorithms to leverage beneficial features/quirks of their input distribution to beat the sub-Gaussian rate, without explicit knowledge of these features?

We resolve this question, finding an unexpectedly nuanced answer: 'Yes in limited regimes, but in general no'. Given a distribution p , assuming only that it has a finite mean and absent any additional assumptions, we show how to construct a distribution q n,δ such that the means of p and q are well-separated, yet p and q are impossible to distinguish with n samples with probability 1 -δ , and q further preserves the finiteness of moments of p . Moreover, the variance of q is at most twice the variance of p if it exists. The main consequence of our result is that, no reasonable estimator can asymptotically achieve better than the sub-Gaussian error rate for any distribution, up to constant factors, which matches the worst-case result of [Lee and Valiant, 2022]. More generally, we introduce a new definitional framework to analyze the fine-grained optimality of algorithms, which we call 'neighborhood optimality', interpolating between the unattainably strong 'instance optimality' and the trivially weak admissibility/Pareto optimality definitions. As an application of the new framework, we show that the median-of-means algorithm is neighborhood optimal, up to constant factors. It is an open question to find a neighborhood-optimal estimator without constant factor slackness.

1 Introduction

Mean estimation over R is one of the most fundamental problems in statistics: given n i.i.d. samples from some unknown distribution p , how do we most accurately estimate the mean of p , with probability ≥ 1 -δ , from the n samples? The conventional approach is to take the sample mean, the empirical average of the samples, as the estimate; this is justified by the law of large numbers, which says that that in the limit of having infinitely many samples, the sample mean will converge to the true mean. However, it has been long known that while the sample mean is optimal for estimating the mean of well-behaved distributions such as Gaussians, it is sensitive to the presence of outliers in samples drawn from heavy-tailed distributions, for which the sample mean estimator can have abysmal performance.

The classic median-of-means estimator [e.g., NY83; JVV86; AMS99], independently invented several times in the literature, was proposed to mitigate the sensitivity of the sample mean. Its accuracy on any distribution p with finite variance is-up to constant factors-as good as the accuracy of the sample mean on a Gaussian with the same mean and variance as p . In other words, the error is of sub-Gaussian rate . The past decade has seen renewed interest across computer science and statistics in understanding the limits of the mean estimation problem. In one dimension, the current state of the art is 1) for distributions p with finite variance σ 2 p , the recent mean estimator proposed by Lee and

Valiant [LV22] has sub-Gaussian rate σ p · ( √ 2 + o (1)) √ log 1 δ /n , which is tight even in its constants, up to a 1 + o (1) factor, and 2) the work of Bubeck, Cesa-Bianchi and Lugosi [BCL13] showing upper bounds matching the lower bounds of Devroye, Lerasle, Lugosi and Oliveira [DLLO16], which show that for heavy-tailed distributions having only a 1 + α th moment for some α ∈ (0 , 1) , the median-of-means algorithm in fact still achieves the optimal accuracy up to a constant factor. Both of these results, however, are optimal only in the worst case, meaning that [LV22; DLLO16] have optimal performance with respect to the class of distributions with the same 2 nd or 1 + α th moments respectively; but estimators may do better than these bounds on particular input distributions.

The natural and immediate next question is, even though Gaussian distributions are the hardest case in mean estimation, and thus sub-Gaussian performance is worst-case optimal: can one do better, at least for some 'easier' distributions? Can we develop 'instance-dependent' algorithms and analysis techniques? Is it possible for algorithms to leverage beneficial features/quirks of their input distribution to beat the sub-Gaussian rate, without explicit knowledge of these features and without losing robustness to heavy-tails?

We resolve this question and show an unexpectedly nuanced answer: 'Yes in limited regimes, but in general no'. For some distributions, even median-of-means can beat the sub-Gaussian bound, but only for a limited parameter regime per distribution-namely, if the number of samples is not too large (Proposition 14). In general however, we show a strong and comprehensive negative result. Our main technical result (Theorem 2) is a fine-grained indistinguishability construction: given a distribution p , assuming only that p has a finite mean and absent any further assumptions, we show how to construct a distribution q n,δ -in terms of a sample complexity n and a failure probability δ -such that p and q are impossible to distinguish with n samples, with probability ≥ 1 -δ , and yet the means of p and q are well-separated by some function ϵ n,δ ( p ) (stated formally in Definition 1). This in particular implies that no n -sample estimator with failure rate δ can simultaneously have error less than 1 2 ϵ n,δ ( p ) on both p and q . The function ϵ n,δ ( p ) is such that, for p with a finite variance, if we take log 1 δ /n → 0 , we have ϵ n,δ ( p ) → Ω( σ p √ log 1 δ /n ) , showing that no estimator can asymptotically outperform the sub-Gaussian rate for both p and q n,δ simultaneously. Additionally, as shown in Section 2.1, the construction of q n,δ is conservative, such that d q d p ≤ 2 at all points, meaning that q has a finite 1 + α th moment whenever p does, and furthermore, σ 2 q ≤ 2 σ 2 p whenever σ 2 p exists. Thus, the same indistinguishability result still applies even if we further require the existence of higher moments for both p and q .

The key message of our paper is that such lower bounds are to be circumvented , through identifying additional favorable distribution structure for the mean estimation problem. 1 This observation has already led to further work in the area, guiding the design of a new algorithm that outperforms

the sub-Gaussian rate via a symmetry assumption. Gupta et al. [GLP23] show that, assuming the distribution is symmetric about its mean, one can achieve finite-sample and instance-dependent Fisher information rates for mean estimation, which can be significantly better than sub-Gaussian rates. We view our paper and the [GLP23] result together as a call to arms to explore other structural assumptions that can sidestep our lower bound construction.

Beyond the asymptotic implications in the finite variance setting, our results fully characterize mean estimation in the regimes of 1) finite variance and finite samples, and 2) infinite variance, and indeed, even when no 1 + α th moment exists for any constant α > 0 . In particular, we give a simple, yet very general re-analysis of the median-of-means estimator on distributions only assumed to have a finite mean. Its estimation error on distribution p , with probability 1 -δ over n samples, is O ( ϵ n,δ ( p )) (Proposition 14), matching our main indistinguishability result up to constants (Theorem 2).

1.1 Our Results and Techniques

Our main result is an indistinguishability result, for every distribution p , which serves as a mean estimation lower bound. Ideally, given the motivation earlier in the introduction, we want to show that the sub-Gaussian rate is a lower bound, but such a bound cannot hold in finite samples. To see this, consider a distribution which has ≪ 1 /n mass that is extremely far away from the mean, and which contributes the majority of the variance, yet only a minuscule portion of the mean. Given only n samples, with high probability we will not see any samples from this mass, and so mean estimation can in fact be 'effectively' performed on the conditional distribution without this outlier mass. The conditional distribution has essentially the same mean as the original, yet has far smaller variance, thus allowing us to beat the (variance-dependent) sub-Gaussian rate. We thus start this section by defining an error function ϵ n,δ ( p ) for each distribution p (Definition 1), capturing the estimation error we expect for p , taking into account that we intuitively expect both algorithms and lower bounds to ignore outlier mass. Our main result will then construct, for every distribution p , a new distribution q that is indistinguishable from p using n samples, yet has mean difference at least ϵ n,δ ( p ) from p .

Definition 1 Given a (continuous) distribution p with mean µ p and a real number t ∈ [0 , 1] , define the t -trimming operation on p as follows: select a radius r such that the probability mass in [ µ p -r, µ p + r ] equals 1 -t ; then, return the distribution p conditioned on lying in [ µ p -r, µ p + r ] .

Theorem 2 shows that, given any distribution p with a finite mean, it is possible to construct a distribution q n,δ such that 1) p and q are indistinguishable under n samples with probability 1 -δ , yet 2) the means of p and q are separated by Ω( ϵ n,δ ( p )) . The construction of q n,δ is also such that d q/ d p ≤ 2 , showing that, in many senses, ' q does not have more extreme tails than p '.

̸

Theorem 2 Let n be the sample complexity and δ be the failure probability, and recall the definition of the error function ϵ n,δ from Definition 1. Assume that there is a sufficiently small constant which upper bounds both log 1 δ n and δ . Then for any distribution p with a finite mean µ p , there exists a distribution q = p with mean µ q such that:

| µ q -µ p | ≥ 1 32 ϵ n,δ ( p )

log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ

d q d p ≤ 2 .

In particular, by a standard fact (Fact 1) on the squared Hellinger distance, this implies that p and q are indistinguishable using n samples, with probability 1 -δ . Furthermore, since d q d p ≤ 2 , we have σ 2 q ≤ E q [( X -µ q ) 2 ] ≤ E q [( X -µ p ) 2 ] ≤ 2 E p [( X -µ p ) 2 ] = 2 σ 2 p .

Using a standard testing-to-estimation reduction, it follows from the main theorem that there can be no estimator that can achieve error at most 1 64 ϵ n,δ ( p ) simultaneously on p and q n,δ .

Corollary 3 Let n be the sample complexity and δ be the failure probability, and recall the definition of the error function ϵ n,δ from Definition 1. Given a distribution p with finite mean, consider the construction of q in Theorem 2. Then, there is no estimator that achieves error less than 1 64 ϵ n,δ ( p ) over n samples with probability 1 -δ , for both p and q .

We contrast this lower bound with more standard impossibility results that have the flavor: 'one cannot estimate the mean to within σ · ( √ 2 + o (1)) √ log 1 δ /n , since there are a pair of Gaussian distributions of variance σ , separated by twice this distance, that are indistinguishable in n samples up to probability 1 -δ '. As opposed to showing the indistinguishability of 'nice' distributions like Gaussians, we instead show that for any distribution p of interest (with finite mean), we exhibit a generic construction of a hard-to-distinguish 'partner' distribution q , of rather different mean, yet all of whose tails are comparable to those of p .

For reference, the definition of Hellinger distance and the standard fact we reference above are:

Fact 1 Consider the squared Hellinger distance between two distributions p and q , defined as

If the two distributions p and q are such that log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ , then there is no test that distinguishes p and q with probability 1 -δ using n samples.

Complementing the specific construction of q and analysis of its properties, we also introduce a new and general definition framework, speaking to the challenge of capturing 'beyond worst case analysis' in this nuanced setting. In Section 3 we motivate and introduce this notion. Broadly, we want to capture the intuition of 'instance-optimal' algorithms, namely, an algorithm that performs as well on the given distribution p as any algorithm customized towards p ; however, this definition is unattainably strong in our setting. By contrast, weaker notions such as 'admissibility' or Pareto-efficiency are too weak to rule out trivial and effectively useless estimators. We introduce a new notion which we call 'neighborhood optimality' that subtly blends between these notions. See Section 3 for details. Appendix A.3 also gives an in-depth discussion comparing neighborhood optimality with the more commonly-used notion of local minimax [AD20b; AD20a; HLY21]. While the two definitions look superficially similar, we show in the appendix-using a general proposition and a concrete example-that our new notion is a stronger and also more robust definition in the context of mean estimation.

As an application of this new framework, we show in Section 4 that the standard median-of-means estimator is neighborhood optimal up to constant factors. It is an open question to design a neighborhoodoptimal estimator which does not have such constant factor slackness.

1.2 Open Questions

We briefly discuss a few open questions and future research directions raised by our results.

Optimal constants for neighborhood optimality estimators-in theory and in practice While median-of-means enjoys neighborhood optimality as we show, the hidden multiplicative constants constants in our analysis are not tight. The median-of-means estimator is also not recommended in practice due to its large asymptotic variance [Min19]. The goal thus is to show that more modern estimators, for example [LV22], also enjoy neighborhood optimality. Ideally, such analysis would yield optimal constants.

Other distributional structures for avoiding asymptotic sub-Gaussian lower bound As mentioned in the introduction, we view our paper as a call to arms to investigate distributional structures and assumptions that allow mean estimators to go beyond the sub-Gaussian error rate. The paper of Gupta et al. [GLP23] is the first work aiming to circumvent our lower bounds-showing the benefit of symmetric distributions for mean estimation. The hope, and challenge, is to find other realistic settings that enable mean estimation algorithms beyond the sub-Gaussian benchmark.

High-Dimensional Neighborhood Optimality Generalizing the framework and results of this paper to the high-dimensional setting is a compelling direction for future work. In high dimensions,

for a distribution with covariance Σ , the corresponding sub-Gaussian rate for mean estimation under the ℓ 2 norm is Θ( 1 √ n √ tr(Σ) + ∥ Σ ∥ log 1 δ ) . We conjecture that, like in the 1-dimensional case, mean estimators cannot outperform the sub-Gaussian rate in the asymptotic regime as n → ∞ . More specifically, we conjecture that whenever a distribution p has a finite covariance, a version of Theorem 2 holds for some ϵ n,δ ( p ) that approaches the high-dimensional sub-Gaussian error as n →∞ , up to constants. Further, for distributions without a finite covariance matrix, it is open to fully characterize the instance-by-instance error rates in high dimensions, and finding an analog of ϵ n,δ ( p ) that characterizes both the upper and lower bounds.

1.3 Related Work

The mean estimation problem has been extensively studied, even in one dimension. In the classic setting where the underlying distribution is assumed to have finite but unknown variance, the medianof-means algorithm, independently discovered by different authors [e.g., JVV86; AMS99; NY83], was the first to achieve sub-Gaussian rate to within a constant factor in the high probability regime. The seminal work of Catoni [Cat12] reinvigorated the study of mean estimation, by proposing the first estimator which attains sub-Gaussian rate tight to within a 1 + o (1) factor, but his estimator requires a-priori knowledge of the variance, or a bounded 4 th moment assumption that allows accurate estimation of the variance. This work further showed that the sub-Gaussian rate is a lower bound on the optimal estimation error. Subsequent work by Devroye et al. [DLLO16] proposed a different estimator, also attaining 1 + o (1) -tight sub-Gaussian rate under the bounded 4 th moment assumption, which has additional structural properties. Recent work by Lee and Valiant [LV22] constructs an estimator achieving sub-Gaussian rate to within a 1 + o (1) factor for any distribution with finite but unknown variance, absent any knowledge assumption or bounded 4 th moment assumption.

In the more extreme setting where the underlying distribution may have infinite variance, but is guaranteed to have finite (but unknown) 1 + α th moment for some α ∈ (0 , 1) , Bubeck et al. [BCL13] proved an upper bound on the error achieved the median-of-means estimator, and Devroye et al. [DLLO16] showed a matching lower bound up to a constant factor.

See the in-depth survey by Lugosi and Mendelson [LM19] on sub-Gaussian mean estimation and regression results prior to 2019.

'Beyond worst-case' analysis is a theme of much recent work and attention in the computer science literature. See [Rou21] for examples of 'beyond worst-case' analyses in various contexts in computer science and statistics including, for example, combinatorial algorithmic problems, auction design, and hypothesis testing. Both of the above tightness results in mean estimation are in the worst-case, and in this work, we present (to our knowledge) the first 'beyond worst-case analysis' results for the mean estimation problem. We emphasize also that our results are applicable even to distributions with a finite mean, but without any finite 1 + α th moment for any α > 0 .

The notion of admissibility in statistics and the analogous concept of Pareto efficiency serve as a main motivation for our definition of neighborhood optimality, introduced in Section 3. They are wellstudied notions in their respective fields, to the extent that they are standard topics in undergraduate courses. See for example the textbooks by Keener [Kee10] and Starr [Sta97] for expositions.

The other main definitional motivation is the notion of instance optimality, which falls under the umbrella of 'beyond worst-case analysis' in the computer science literature. We highlight some of the uses of instance optimality in statistical contexts. Valiant and Valiant [VV17] gave the first instance optimal algorithm for the identity testing problem (in total variation distance) for discrete distributions. In later work [see VV16], the same authors showed how to instance-optimally learn discrete distributions (in total variation distance).

A different line of work studies 'instance optimality' in the context of mean estimation with differential privacy (DP) [e.g., AD20b; AD20a; HLY21]. Specifically, these works address the problem of differentially-privately estimating the mean of a data set, where the data set itself (or equivalently, the uniform distribution over the data set) is the instance. In the DP setting, instance optimality is also not satisfiable by any estimator for the same reason as the non-DP setting, since the hardcoded estimator is always differentially private. They instead use a local minimax (or locally worst-case) notion of optimality (although in these works this notion is sometimes also called 'instance optimality'; we take care to distinguish the two definition styles in this paper), where the particular locality/neighborhood

structure they use is restricted to data sets that have Hamming distance 1 from the instance, since this is a key component in the definition of DP. The local minimax notion of optimality is closely related to our notion of neighborhood optimality. We compare and contrast the two notions in Appendix A.3, and explain why our definition is stronger and more appropriate for our context.

2 Proof of main results

This section gives the construction and proof of our main result, Theorem 2.

Our construction of q from p will have 2 cases, depending on which of the 2 terms in the definition of the error function ϵ n,δ dominates. Recall that the error ϵ n,δ ( p ) is the sum of two terms involving the 'trimmed' distribution p ∗ n : (ignoring constants) | µ p -µ p ∗ n | and σ p ∗ n √ log 1 δ /n ; intuitively, the first term measures to what degree p has an 'asymmetric tail', and the second term measures the variance of p in its central region.

First, consider the case when the first term is larger, namely, | µ p -µ p ∗ n | > σ p ∗ n √ 4 . 5log 1 δ /n . Our goal in constructing q is to maximize | µ p -µ q | subject to q being indistinguishable from p . Given that 1) the mean of p ∗ n is already far from the mean of p by assumption in the case analysis, and 2) p and p ∗ n are by construction hard to distinguish since only a small amount of probability mass was trimmed from p to make p ∗ n , we simply need to construct q as a carefully chosen convex combination of p and p ∗ n , and show that this q indeed satisfies all the properties in the definition of N n,δ ( p ) .

Next, for the remaining case when the variance term σ p ∗ n √ 4 . 5log 1 δ /n is larger than the remaining term | µ p -µ p ∗ n | in ϵ n,δ ( p ) , we now want to construct q such that the mean shift | µ p -µ q | is large compared to the variance term σ p ∗ n √ 4 . 5log 1 δ /n , while ensuring that q is indistinguishable from p . To achieve this, we create q that is a 'skewed' version of p , scaling the probability density by a linear function 1 + ax , where larger a means more mean shift between q and p , but also means that it is easier to distinguish q from p . Technically, we truncate the linear scaling factor 1 + ax to lie between 0 and 2, so that q will satisfy Condition 3 of the requirements for q presented in our main theorem, Theorem 2; the probability mass might not be 1 after this skewing, so we might need to normalize; also, we point out that for the purposes of Theorem 2 we do not care whether the mean of q is shifted to the left or right of p (corresponding to choosing a positive or negative parameter a ), and the construction makes use of this choice.

Definition 4 (Construction of indistinguishable pair) Given a distribution p , we construct a distribution q in a shift-and-scale invariant manner as follows.

Case 1: | µ p -µ p ∗ n | > σ p ∗ n √ 4 . 5 log 1 δ n . Define λ = 3 4 and construct q to be the weighted average q = λp +(1 -λ ) p ∗ n .

Case 2: | µ p -µ p ∗ n | ≤ σ p ∗ n √ 4 . 5 log 1 δ n . Without loss of generality, assume that µ p = 0 . Let parameter a be the solution in the interval ( 0 , 1 σ p ∗ n √ log 1 δ n ] to the below equation characterizing the mean shift, as guaranteed by Lemma 23 in Appendix B:

We construct two non-unit measures q + , q -, defined as scaled versions of p , as d q ± d p ( x ) = 1 + min(1 , max( -1 , ± ax )) . By symmetry the masses of q + and q -sum to 2; thus one of q + , q -has mass at least 1. Construct q by choosing that one of q + , q -, and downscaling it by a factor b ∈ [ 1 2 , 1] such that the total probability mass is indeed 1. ◁

In the rest of the subsections, we will prove all the properties of the construction needed by Theorem 2. We observe that the non-unit measure q -is 'symmetric' to q + , in the sense that its first moment shift

from p is identical but of the opposite sign. Therefore, most of the analysis below will assume the q + case without loss of generality.

2.1 Checking that d q d p ≤ 2

It is straightforward to check that d q d p ≤ 2 by construction, for both cases of Definition 4.

Lemma 5 Suppose there is a sufficiently small absolute constant that upper bounds log 1 δ n . Given a distribution p , if we construct q as in Case 1 (the large | µ p ∗ n -µ p | case) in Definition 4, then d q d p ≤ 2 .

Proof. Let the support of p ∗ n be [ x left , x right ] . At x / ∈ [ x left , x right ] , we have d q d p = λ d p d p = 3 / 4 d p d p ≤ 2 . Otherwise, at x ∈ [ x left , x right ] , we have

where the last line follows from the fact that λ is a constant in (0 , 1) and log 1 δ n is assumed to be bounded by some sufficiently small absolute constant. □

Lemma 6 Suppose there is a sufficiently small absolute constant that upper bounds log 1 δ n . Given a distribution p , if we construct q as in Case 2 (the small | µ p ∗ n -µ p | case) in Definition 4, then d q d p ≤ 2 .

Proof. For q + , we have d q + d p ( x ) = 1 + min(1 , max( -1 , a + x )) for some value of a + , and the right hand side is always between 0 and 2 ; proven similarly, we have 0 ≤ d q -d p ≤ 2 . Noting that q is constructed by scaling down one of q -and q + , we also have d q d p ≤ 2 . □

2.2 Bounding the squared Hellinger distance

We consider each case of the construction in Definition 4 separately. We first bound the Hellinger distance between p and the q constructed in Case 1 of Definition 4 (when | µ p -µ p ∗ n | is large).

Lemma 7 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 1 (the large | µ p ∗ n -µ p | case) in Definition 4, then log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ .

Since Case 1 of the construction of q in Definition 4 linearly interpolates between p and-a very slightly trimmed version of p , namelyp ∗ n , the resulting distribution q remains close to p ; the calculation is in Appendix B.1. The next lemma bounds the squared Hellinger distance of p and q in Case 2 of Definition 4 (when | µ p -µ p ∗ n | is small).

Lemma 8 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 2 (the small | µ p ∗ n -µ p | case) in Definition 4, then log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ .

The proof (see Appendix B.1) uses a technical lemma to relate d H ( p, q ) to d H ( p, q + ) or d H ( p, q -) , and then uses a linearization of the definition of Hellinger distance to relate it to the mean shift between p and q + or q -, which is bounded by Definition 4.

2.3 Lower bounding | µ q -µ p |

We show the lower bound separately for the two cases in the construction of q in Definition 4. The small | µ p ∗ n -µ p | case is a direct corollary of Lemma 23, which bounds the mean shift in this case.

Lemma 9 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 1 (the large | µ p ∗ n -µ p | case) in Definition 4, then | µ q -µ p | ≥ 1 8 ϵ n,δ ( p ) .

Proof. Without loss of generality, assume that µ p = 0 . Recall that q = λp + (1 -λ ) p ∗ n and λ = 3 4 from Case 1 of Definition 4. Thus, | µ q -µ p | = 1 4 | µ p ∗ n -µ p | . Furthermore, we have ϵ n,δ ( p ) = | µ p ∗ n -µ p | + σ p ∗ n √ 4 . 5 log 1 δ n from the definition of ϵ n,δ ( p ) and | µ p ∗ n -µ p | > σ p ∗ n √ 4 . 5 log 1 δ n from the lemma assumption and Case 1 of Definition 4, which then imply that ϵ n,δ ( p ) ≤ 2 | µ p ∗ n -µ p | . Combining the two inequalities yields | µ q -µ p | ≥ 1 8 ϵ n,δ ( p ) as desired. □

Lemma 10 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 2 (the small | µ p ∗ n -µ p | case) in Definition 4, then | µ q -µ p | ≥ 1 32 ϵ n,δ ( p ) .

+ +

Proof. Without loss of generality, let q be the non-unit measure used to construct q , that is q = bq for some b ∈ [ 1 2 , 1] . By the bound on b as well as the construction of q + in Definition 4, we have | µ q -µ p | = b | ∫ x d q + -∫ x d p | ≥ 1 16 σ p ∗ n √ log 1 δ n . Additionally, we both have ϵ n,δ ( p ) = | µ p ∗ n -µ p | + σ p ∗ n √ 4 . 5 log 1 δ n from the definition of ϵ n,δ ( p ) and | µ p ∗ n -µ p | ≤ σ p ∗ n √ 4 . 5 log 1 δ n from the lemma assumption and Case 2 of Definition 4, both of which imply ϵ n,δ ( p ) ≤ 2 σ p ∗ n √ 4 . 5 log 1 δ n . Combining the two inequalities above, we have that | µ q -µ p | ≥ 1 32 ϵ n,δ ( p ) . □

3 Neighborhood Optimality: A New Definition Framework

Our main result is a specific and technical indistinguishability result. This section aims to clarify, through a new definition framework, the optimality notion that our technical result implies.

3.1 Neighborhood Optimality

Usual notions of 'beyond worst-case optimality' include 'instance optimality' which is unattainably strong, and 'admissibility'/'Pareto efficiency' from the statistics and economics literature, which is too weak. In particular, the latter notion is too weak in the sense that a trivial estimator that always outputs the same hardcoded mean estimate is actually admissible, despite being algorithmically 'vacuous'. We define and explain these notions formally in Appendix D.

Given that neither of the usual definitions are suitable for mean estimation, in this work we give a new optimality definition, which we call neighborhood optimality . We state its definition in this section, and explore its basic properties and intuition in Appendix A, including how the definition smoothly interpolates between instance optimality and admissibility. Our definition is also related to the notion of local minimax optimality, which we compare with in Appendix A.3. The differences are subtle, yet, as we show in Appendix A.3, local minimax is too weak a notion and, when instantiated inappropriately, allows for absurd bounds to be proven. We thus advocate for this new optimality definition, which correctly rejects such absurd bounds. As an application of our new framework, we prove in Section 4 that the median-of-means estimator is neighborhood optimal up to constant factors. It is an open question to find a neighborhood optimal estimator without the constant factor slackness.

Let P 1 be the entire set of all distributions with a finite first moment over R . We say that N is a neighborhood function (defined over P 1 ) if N maps a distribution p ∈ P 1 to a set of distributions N ( p ) ⊆ P 1 . For the purposes of the rest of the definitions, it will not matter whether p ∈ N ( p ) . Similarly, an error function ϵ maps distributions to non-negative numbers, like ϵ n,δ in our main result, Theorem 2. In the later definitions, we use the notations N n,δ and ϵ n,δ to denote their dependence on the sample complexity n and failure probability δ .

Given these two notions, we can now define neighborhood Pareto bounds with respect to N n,δ , which imposes admissibility structure within the local neighborhood N n,δ ( p ) of every distribution p ∈ P 1 .

Definition 11 (Neighborhood Pareto bounds with respect to N n,δ ) Let n be the number of samples and δ be the failure probability. Given a neighborhood function N n,δ : P 1 → 2 P 1 , we say that the error function ϵ n,δ ( p ) : P 1 → R + 0 is a neighborhood Pareto bound for P 1 with respect to N n,δ if for all distributions p ∈ P 1 , no estimator ˆ µ taking n i.i.d. samples can simultaneously achieve the following two conditions:

For all q ∈ N n,δ ( p ) , with probability 1 -δ over the n i.i.d. samples from q , | ˆ µ -µ q | ≤ ϵ n,δ ( q ) .

With probability 1 -δ over the n i.i.d. samples from p , | ˆ µ -µ p | < ϵ n,δ ( p ) .

Note the strict inequality in the second bullet: namely, it is impossible to 'beat' the error function over an entire neighborhood, where 'beating' is defined as attaining the error function over the neighborhood, and performing strictly better than the error function for p . The above two bullet points essentially capture admissibility within the local neighborhood N n,δ ( p ) ∪ { p } -compare with Definition 28-and the definition requires admissibility within every such local neighborhood, over every possible p .

The neighborhood N n,δ ( p ) in a neighborhood Pareto bound can be interpreted as the set of distributions 'near p ' which, if an estimator performs well on distribution p , then we should reasonably expect or want it to perform well also on all the distributions in the local neighborhood N n,δ ( p ) .

Using the notion of neighborhood Pareto bounds, we can now define κ -neighborhood optimal estimators, which are estimators whose performances are matched by neighborhood Pareto bounds.

Definition 12 ( ( κ, τ ) -Neighborhood optimal estimators) Let κ > 1 be a multiplicative loss factor in estimation error, and τ > 1 be a multiplicative loss factor in sample complexity. Given the parameters κ, τ > 1 , sample complexity n , failure probability δ and neighborhood function N n,δ , a mean estimator ˆ µ is ( κ, τ ) -neighborhood optimal with respect to N n,δ if there exists an error function ϵ n,δ ( p ) such that min( ϵ n/τ,δ ( p ) , ϵ n,δ ( p )) is a neighborhood Pareto bound 2 , and ˆ µ gives estimation error at most κ · ϵ n,δ ( p ) with probability at least 1 -δ when taking n i.i.d. samples from any distribution p ∈ P 1 .

As a basic example and sanity check, in Appendix E, we show that any trivial estimator that outputs a hardcoded mean estimate cannot be κ -neighborhood optimal with respect to our chosen neighborhood function (Definition 15 in Section 4) for any κ .

3.2 Indistinguishability implies a neighborhood Pareto bound

Even though it might not look obvious how we can prove a neighborhood Pareto bound from its definition, we show that our main indistinguishability result essentially implies such a bound. The proof essentially follows the straightforward estimation-to-testing reduction intuition, and we give it formally in Appendix A.2.

̸

Proposition 13 ('Local' indistinguishability bounds imply neighborhood Pareto bounds) The error function ϵ n,δ is a neighborhood Pareto bound with respect to the neighborhood function N n,δ if for every distribution p ∈ P 1 , there exists a distribution q ∈ N n,δ ( p ) , with q = p , such that | µ p -µ q | ≥ ϵ n,δ ( p ) + ϵ n,δ ( q ) and it is information-theoretically impossible to distinguish p and q with probability 1 -δ using n samples.

4 Median-of-Means is Neighborhood Optimal

To apply our new definitional framework, we choose a reasonable neighborhood function N n,δ and show that the median-of-means algorithm is neighborhood optimal with respect to this choice.

In Appendix C, we give the following (straightforward) re-analysis of median-of-means, which will form the upper bound part for neighborhood optimality.

Proposition 14 Consider a distribution p with mean µ p , a sample size n , and a median-of-means group count 4 . 5log 1 δ . Let p ∗ n be the 0 . 45 n log 1 δ -trimmed distribution from Definition 1, and µ p ∗ n and σ p ∗ n be the mean and standard deviation of p ∗ n respectively. Then, the median-of-means estimator has

We can now discuss the neighborhood choice for the corresponding neighborhood Pareto bound. Recall that, intuitively, neighborhood optimality is asking 'how well can our algorithm do on p given that we also want our algorithm to do similarly well on a neighborhood of p '; and thus, the smaller we choose the neighborhood, the stronger the resulting theorem. We thus define the neighborhood of p to consist of distributions that are similar or similarly nice to p in 4 different ways:

Definition 15 (Choice of neighborhood function N n,δ in Theorem 16) Define N n,δ ( p ) to be the set of distributions q ∈ P 1 such that

ϵ n/ 3 ,δ ( q ) ≤ 100 ϵ n,δ ( p )

log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ

| µ q -µ p | ≤ ϵ n,δ ( p )

For all x ∈ R , d q d p ( x ) ≤ 2 .

As a basic sanity check, we show in Appendix E that a trivial, hardcoded estimator cannot be neighborhood optimal-this is mostly a consequence of Property 3 above. See Appendix E for a formal statement and proof. We then show:

Theorem 16 Let n be the number of samples and δ be the failure probability. Assume that there is a sufficiently small constant which upper bounds both log 1 δ n and δ .

Consider the neighborhood function N n,δ of Definition 15. Recall the error function defined in Definition 1 as ϵ n,δ ( p ) = | µ p -µ p ∗ n | + σ p ∗ n √ 4 . 5log 1 δ n . Then, for some sufficiently large constant κ , the error function 1 κ min( ϵ n/ 3 ,δ , ϵ n,δ ) is a neighborhood Pareto bound with respect to N n,δ .

Combined with Proposition 14 stating that the median-of-means estimator has error function O ( ϵ n,δ ) , this implies the median-of-means estimator is ( κ, 3) -neighborhood optimal with respect to N n,δ .

The main component of the proof is our construction of q , and the accompanying analysis of Theorem 2 showing that q is well behaved in several senses. We specifically show Lemma 17, a slight extension of Theorem 2:

̸

Lemma 17 Let n be the sample complexity and δ be the failure probability, and recall the definition of ϵ n,δ from Definition 1. Assume that there is a sufficiently small constant which upper bounds both log 1 δ n and δ . Then for any distribution p , there exists a distribution q = p such that the mean of q is 1 32 ϵ n,δ ( p ) different from the mean of p , and log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ , and q ∈ N n,δ ( p ) .

See Appendix F for the proof of Lemma 17. We will now use Lemma 17 to prove Theorem 16.

̸

Proof of Theorem 16. By Proposition 13, it suffices to show that, for every distribution p ∈ P 1 , there exists a distribution q ∈ N n,δ ( p ) with q = p such that | µ p -µ q | ≥ 1 κ (min( ϵ n/ 3 ,δ , ϵ n,δ )( p ) + min( ϵ n/ 3 ,δ , ϵ n,δ )( q )) for some large constant κ , and no tester can distinguish p and q with probability 1 -δ using n samples.

Given a distribution p ∈ P 1 , consider the distribution q ∈ N n,δ ( p ) guaranteed by Lemma 17. Since q satisfies log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ , by Fact 1 we know that p and q are indistinguishable with probability 1 -δ using n samples.

It remains to check that | µ p -µ q | ≥ 1 κ (min( ϵ n/ 3 ,δ , ϵ n,δ )( p ) + min( ϵ n/ 3 ,δ , ϵ n,δ )( q )) for some sufficiently large constant κ . By Lemma 17, we have

which completes the proof of Theorem 16.

□

References

A Interpreting Neighborhood Optimality

In this section, we derive basic properties and intuitions about the definitions of neighborhood Pareto bounds and neighborhood optimality.

In particular, we show that neighborhood optimality is a notion that smoothly interpolates between instance optimality and admissibility, depending on what neighborhood function is used to instantiate the definition. We also show a sufficient condition for proving that an error function ϵ is a neighborhood Pareto bound (Proposition 13). Lastly, we discuss and compare neighborhood optimality and the notion of local minimax optimality that has appeared in the literature.

A.1 As an interpolation between instance optimality and admissibility

Recall that Definitions 11 and 12 both require specifying the neighborhood function. Our first observation is the monotonicity in the definition of neighborhood Pareto bounds, in the sense of the following straightforward proposition which we state without proof.

Proposition 18 Suppose the neighborhood functions N n,δ and N ′ n,δ are such that N n,δ ( p ) ⊆ N ′ n,δ ( p ) for all p . Then, if a given error function ϵ n,δ is a neighborhood Pareto bound with respect to N n,δ , then ϵ n,δ is also a neighborhood Pareto bound with respect to N ′ n,δ .

We can extend this monotonicity observation from neighborhood Pareto bounds to neighborhood optimal estimators.

Proposition 19 Suppose the neighborhood functions N n,δ and N ′ n,δ are such that N n,δ ( p ) ⊆ N ′ n,δ ( p ) for all p . Then, if ˆ µ is a κ -neighborhood optimal estimator with respect to N n,δ , then ˆ µ is also κ -neighborhood optimal with respect to N ′ n,δ .

To understand how the set of neighborhood optimal estimators vary as we change the neighborhood function, we examine the two extreme examples as special cases, where N n,δ ≡ ∅ and N n,δ ≡ P 1 .

Let us first consider the case when the neighborhoods are all empty. In this case, Definition 11 simplifies to requiring a neighborhood Pareto bound ϵ n,δ to be such that, for every distribution p ∈ P 1 , no estimator ˆ µ (specialized to p ) can have error strictly less than ϵ n,δ ( p ) with probability 1 -δ over n samples from p . It is straightforward to check that any estimator that is κ -neighborhood optimal with respect to empty neighborhoods is equivalent to being instance optimal (Definition 27 in Appendix D) up to a κ factor, which as explained before, is impossible to achieve. The only possible neighborhood Pareto bound with respect to empty neighborhoods is ϵ n,δ ≡ 0 , since every distribution has a trivial estimator that outputs its mean hardcoded.

̸

At the other extreme, consider the case where all the neighborhoods contain all the distributions in P 1 . Here, Definition 11 simplifies (after taking a contrapositive) to requiring a neighborhood Pareto bound ϵ to be such that, for every distribution p ∈ P 1 , if an estimator has error at most ϵ n,δ ( q ) with probability 1 -δ for all distributions q = p ∈ P 1 , then it must have error at least ϵ n,δ ( p ) with probability 1 -δ for distribution p . It is again straightforward to check that, by definition, an estimator is 1 -neighborhood optimal with respect to the constantP 1 neighborhood function if and only if the estimator is admissible in P 1 (Definition 28 in Appendix D). As explained before, admissibility is a somewhat weak notion on estimators, and includes trivial hardcoded estimators.

Combining these two extreme cases and the monotonicity propositions, we have shown that neighborhood optimality is a notion which interpolates between instance optimality and admissibility. More technically, neighborhood optimality can be viewed as a homomorphism mapping the partial ordering of neighborhood functions (the ordering induced by set inclusion) to the partial ordering of sets of estimators (also ordered by set inclusion).

We remark that, while it may be tempting to view neighborhood Pareto bounds as lower bounds , and neighborhood optimality as exhibiting an estimator whose error function matches a lower bound up to a constant factor, such an interpretation is actually not valid. The reason is that, depending on the choice of the neighborhood function, given an estimator with error function ϵ upper n,δ and a neighborhood Pareto bound ϵ pareto n,δ , it can be the case that on some distributions p ∈ P 1 , we have ϵ pareto n,δ ( p ) > ϵ upper n,δ ( p ) ; namely, these Pareto bounds should not be viewed as lower bounds. A simple

example is the previous case where the neighborhood function is P 1 , in which case it is straightforward to check that for any (hardcoded) constant ˆ µ ∈ R , the error function ϵ pareto n,δ ( p ) = | ˆ µ -µ p | is a neighborhood Pareto bound. For any distribution p whose mean is extremely far from the constant ˆ µ , the median-of-means algorithm will have accuracy better than | ˆ µ -µ p | . Because of such examples, we take care not to refer to our bounds as lower bounds, but instead call them Pareto bounds in this paper.

A.2 Indistinguishability implies a neighborhood Pareto bound

We restate Proposition 13 from Section 3.2 here for clarity, and give the proof of it below.

̸

Proposition 13 ('Local' indistinguishability bounds imply neighborhood Pareto bounds) The error function ϵ n,δ is a neighborhood Pareto bound with respect to the neighborhood function N n,δ if for every distribution p ∈ P 1 , there exists a distribution q ∈ N n,δ ( p ) , with q = p , such that | µ p -µ q | ≥ ϵ n,δ ( p ) + ϵ n,δ ( q ) and it is information-theoretically impossible to distinguish p and q with probability 1 -δ using n samples.

Proof. Suppose the 'if' condition in the proposition is true, yet, for the sake of contradiction, that ϵ n,δ is not a neighborhood Pareto bound. Then, there exists a distribution p ∈ P 1 and an estimator ˆ µ taking n i.i.d. samples such that

For all distributions q ∈ N n,δ ( p ) , with probability 1 -δ over the n i.i.d. samples from q , | ˆ µ -µ q | ≤ ϵ n,δ ( q ) .

With probability 1 -δ over the n i.i.d. samples from p , | ˆ µ -µ p | < ϵ n,δ ( p ) .

By the proposition condition, there must exist some distribution q ∈ N n,δ ( p ) such that | µ p -µ q | ≥ ϵ n,δ ( p ) + ϵ n,δ ( q ) and it is information-theoretically impossible to distinguish p and q with probability 1 -δ using n samples. However, we can construct the following distinguisher: compute a mean estimate ˆ µ , return p if ˆ µ is within ϵ n,δ ( p ) of µ p and return q otherwise. By our assumption on ˆ µ , this distinguisher will succeed with probability at least 1 -δ , thus contradicting the proposition statement. □

Weremark that the proof of Proposition 13 actually establishes a stronger result, that the error function ϵ n,δ is a neighborhood Pareto bound for the singleton neighborhood N ∗ n,δ ( p ) = { q ( p ) } where q ( p ) is the q constructed from p according to Proposition 13. Recall that the monotonicity property of Proposition 18 says that neighborhood Pareto bounds are stronger for smaller neighborhoods; thus this bound for singleton neighborhoods implies the corresponding bound for any larger neighboorhoods N n,δ ⊃ N ∗ n,δ .

A.3 Comparing with local minimax optimality

Here, we compare our definition of neighborhood optimality with the notion of local minimax optimality from prior literature. At a high level, neighborhood optimality imposes admissibility within each local neighborhood, whereas local minimax optimality imposes minimax optimality within each local neighborhood. We argue that local minimax is too sensitive to the choice of neighborhood structure, although the two definitions are different in a rather subtle way and perhaps difficult to see at first glance. We concretely illustrate their differences via 1) a proposition showing that for practical purposes, local minimax bounds are a weaker notion than neighborhood Pareto bounds and 2) a simple example choice of an 'inappropriate' neighborhood structure, such that an intuitively absurd local minimax bound holds for this neighborhood structure, but the same bound fails to satisfy the definition of a neighborhood Pareto bound. Together, these results show that neighborhood optimality is a stronger and more robust notion than local minimax optimality.

For ease of comparison, we first phrase the local minimax definition in the same form as our definition of neighborhood optimality.

Definition 20 (Local minimax bounds with respect to N n,δ ) Let n be the number of samples and δ be the failure probability. Given a neighborhood function N n,δ : P 1 → 2 P 1 , we say that the error function ϵ n,δ ( p ) : P 1 → R + 0 is a local minimax bound for P 1 with respect to N n,δ if for all

distributions p ∈ P 1 , there is no estimator ˆ µ such that for all distributions q ∈ N n,δ ( p ) ∪ { p } , when given n i.i.d. samples from q , the estimator ˆ µ achieves | ˆ µ -µ q | < ϵ n,δ ( p ) with probability 1 -δ .

Definition 21 ( κ -locally minimax estimators) For a parameter κ > 1 , sample complexity n , failure probability δ and neighborhood function N n,δ , a mean estimator ˆ µ is κ -locally minimax with respect to N n,δ if there exists an error function ϵ n,δ ( p ) such that ϵ n,δ ( p ) is a local minimax bound, and ˆ µ gives estimation error at most κ · ϵ n,δ ( p ) with probability at least 1 -δ when taking n i.i.d. samples from any distribution p ∈ P .

The notions of local minimax bounds and locally minimax estimators also have the same monotonicity properties as neighborhood Pareto bounds and neighborhood optimal estimators, analogous to Propositions 18 and 19. When the neighborhood function N n,δ is constantly equal to the empty set, local minimax optimality again coincides with instance optimality. Furthermore, local minimax bounds can be proved using indistinguishability arguments, via a reduction analogous to Proposition 13 up to minor changes in parameters.

However, one key difference between the two styles of definition arises when we consider proving neighborhood Pareto bounds or local minimax bounds using such indistinguishability arguments. Recall from Section 3.2 that, by relying on Proposition 13, the main technical result of this paper is to show that for every distribution p , there exists a 'neighbor' q ( p ) whose mean is far from p , with q ( p ) satisfying suitable structural properties (so that q ( p ) ∈ N n,δ ( p ) in our eventual choice of N n,δ in Definition 15), such that p and q cannot be distinguished with probability 1 -δ using n samples. As explained in Section 3.2, this proof strategy effectively proves a neighborhood Pareto bound over the singleton neighborhood function N ∗ n,δ ( p ) = { q ( p ) } . Thus, it is meaningful to compare neighborhood Pareto bounds and local minimax bounds when the neighborhoods are singletons. The following proposition shows that, up to a constant factor of 2 in the error, a neighborhood Pareto bound on singleton neighborhoods implies a potentially much larger local minimax bound.

̸

Proposition 22 (For singleton neighborhoods, local minimax bounds are weaker than neighborhood Pareto bounds) Let n be the number of samples and δ be the failure probability. Consider a neighborhood function N ∗ n,δ such that for all p ∈ P 1 , N ∗ n,δ ( p ) is a singleton set containing a distribution q ( p ) = p . Suppose ϵ n,δ is a neighborhood Pareto bound with respect to N ∗ n,δ , and that ϵ n,δ ( p ) > 0 for all p ∈ P 1 . Then, the function 1 2 ( ϵ n,δ ( p ) + ϵ n,δ ( q ( p )) is a local minimax bound with respect to N ∗ n,δ . Note that the above function is lower bounded by Ω(max( ϵ n,δ ( p ) , ϵ n,δ ( q ( p ))) and can be much larger than ϵ n,δ ( p ) .

Proof. Suppose for the sake of contradiction that 1 2 ( ϵ n,δ ( p )+ ϵ n,δ ( q ( p )) is not a local minimax bound with respect to N ∗ n,δ but ϵ n,δ is a neighborhood Pareto bound.

We observe that, since ϵ n,δ is a neighborhood Pareto bound, it must be the case that for every distribution p ∈ P 1 , we have | µ p -µ q | ≥ ϵ n,δ ( p ) + ϵ n,δ ( q ( p )) . Otherwise, for any distribution p not satisfying the above, there is a trivial hardcoded estimator that outputs a number ˆ µ such that | ˆ µ -µ q | < ϵ n,δ ( q ( p )) and | ˆ µ -µ p | < ϵ n,δ ( p ) .

Since 1 2 ( ϵ n,δ ( p ) + ϵ n,δ ( q ( p )) is not a local minimax bound, there exists some distribution p and some estimator ˆ µ such that 1) with probability at least 1 -δ over n samples from p , | ˆ µ -µ p | < 1 2 ( ϵ n,δ ( p ) + ϵ n,δ ( q ( p ))) and 2) the same for q . However, we already know that | µ p -µ q | ≥ ϵ n,δ ( p ) + ϵ n,δ ( q ( p )) , which implies that we can use the mean estimator ˆ µ to distinguish p and q with probability 1 -δ over n samples. Using this distinguisher, we construct a new estimator ˆ µ ′ which outputs µ p if the distinguisher thinks the distribution is p , and µ q otherwise. This new estimator ˆ µ ′ has 0 error with probability 1 -δ over n samples, which contradicts the assumption that ϵ n,δ is a neighborhood Pareto bound and that ϵ n,δ ( p ) > 0 . □

The notion of local minimax bounds is therefore (potentially much) weaker than the notion of a neighborhood Pareto bound we introduce in this work. We now show that this can actually happen, if we do not choose the neighborhood structure carefully. We give a concrete example of a neighborhood structure in which an absurdly large local minimax bounds holds, but this bad bound is (rightfully) rejected by the definition of neighborhood Pareto bounds.

As a representative simple example, consider p = N (0 , 1) and define its neighborhood as a singleton set N ( p ) = { q = N ( η, 1) } where η ≪ 1 is small enough that p and q are indistinguishable with

n samples. Now define the neighborhood of q to also be a singleton set N ( q ) = { q ′ } , where q ′ is constructed by moving a tiny bit of mass of q such that µ q ′ = η + 10 6 , but q and q ′ are indistinguishable. Consider an absurd error function ϵ ( p ) = η/ 2 , ϵ ( q ) = 10 6 / 2 , which is far too large for q since we expect O ( √ log 1 δ /n ) ≪ 1 estimation error for q (e.g. by using a standard sub-Gaussian mean estimator). Yet, ϵ is a local minimax bound (c.f. Definition 21) under the neighborhood function N , since, given two indistinguishable distributions p and q , no estimator can get error less | µ p -µ q | / 2 . On the other hand, we can also check that Definition 12 rejects this absurd ϵ error function from being a neighborhood Pareto bound. To see this, consider the neighborhood of p , consisting only of q . Consider the hardcoded estimator ˆ µ always outputting 0: ˆ µ violates the condition of neighborhood Pareto bounds since its error for p is | ˆ µ -µ p | = 0 < η/ 2 and for q is | ˆ µ -µ q | = η ≪ 10 6 / 2 .

This example, together with Proposition 22, show that neighborhood optimality is a more robust notion than local minimax optimality when being applied to inappropriately chosen neighborhood structures. While we believe our paper uses an 'appropriate' neighborhood structure, we still emphasize the importance of introducing definitions that are properly resilient to absurd instantiations. For this reason, we have chosen to present our results in this paper as a neighborhood Pareto bound and optimality.

B Remaining proofs for Theorem 2

In Definition 4, we claimed that there exists a parameter a satisfying certain conditions in Case 2 of the construction. We formally show that this parameter exists in the following lemma.

Lemma 23 Let n be the number of samples and and δ be the failure probability, and assume that log 1 δ n is bounded by some sufficiently small absolute constant. Let p be any distribution such that | µ p -µ p ∗ n | ≤ σ p ∗ n √ c log 1 δ n , and we assume that µ p = 0 without loss of generality. Then, the equation

always has solution a ∈ ( 0 , 1 σ p ∗ n √ log 1 δ n ] .

Proof. We first point out that, for the (potentially) non-unit measure q + defined by d q + d p ( x ) = 1 + min(1 , max( -1 , ax )) , the left hand side of the equation equals the difference between the first moments of q + and p . Namely,

Our goal is to find a ∈ ( 0 , 1 σ p ∗ n √ log 1 δ n

] such that this first moment shift equals 1 8 σ p ∗ n √ log 1 δ n .

Because min(1 , max( -1 , ax )) is increasing in a and continuous in a , the first moment shift is also increasing and continuous in a . When a = 0 then q + = p , and the shift clearly equals 0. Thus it suffices for us to show that the first moment shift is at least 1 8 σ p ∗ n √ log 1 δ n when a = 1 σ p ∗ n √ log 1 δ n . The lemma then follows from the intermediate value theorem.

Consider the construction of p ∗ n in Definition 1, and let r be the trimming radius of p . We do a case analysis, either 1 σ p ∗ n √ log 1 δ n ≤ 1 /r or 1 σ p ∗ n √ log 1 δ n ≥ 1 /r .

Summarizing, in either case, at a = 1 σ p ∗ n √ log 1 δ n , we have that the first moment shift between q + and p is

yielding the lemma.

B.1 Bounding the squared Hellinger distance

We restate Lemmas 7 and 8 from Section 2.2 for clarity, then give the proofs of them below.

Lemma 7 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 1 (the large | µ p ∗ n -µ p | case) in Definition 4, then log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ .

Proof of Lemma 7. Since Case 1 in the construction of q in Definition 4 linearly interpolates between p and p ∗ n , with interpolation constant λ = 3 4 , thus the Hellinger distance between p and q can be exactly computed as a function of 1) 0 . 45log 1 δ n , which determines the masses of p, q inside and outside of p 's trimming interval, along with 2) the ratio d q d p inside and outside of p 's trimming interval, which also depends only on 0 . 45 n and λ . We then bound this (essentially) univariate expression.

Without loss of generality, assume that µ p = 0 , and let the support of p ∗ n be [ -r, r ] . Recall the construction of q in Case 1 of Definition 4: q = λp +(1 -λ ) p ∗ n . At x / ∈ [ -r, r ] , we have d q d p ( x ) = λ . Otherwise, at x ∈ [ -r, r ] , we have d q d p ( x ) = d d p ( λp + 1 -λ 1 -0 . 45log 1 δ n p ) ( x ) by the definition of p ∗ n , which is in turn equal to (1 -λ · 0 . 45log 1 δ n ) / (1 -0 . 45log 1 δ n ) . Given the above equalities, we can explicitly calculate 1 -d 2 ( p, q ) as follows.

H

We now show a technical lemma to lower bound the quantity in the last line.

Lemma 24 For any λ ∈ [ 3 , 1] and β ∈ [0 , 1] , we have

Proof. Take the second derivative of the left hand side with respect to β , giving -( λ -1) 2 4(1 -β ) 3 / 2 (1 -λβ ) 3 / 2 , which is negative. On the other hand, the right hand side e ( λ -1) β is an exponential in β and hence convex in β . Therefore, left hand side minus right hand side is concave, meaning that the difference is minimized at either β = 0 or β = 1 . At β = 0 , both sides are equal to 1. At β = 1 , the left hand side is √ λ whereas the right hand side is e λ -1 . The inequality √ λ ≥ e λ -1 is true for any λ ∈ [ 3 4 , 1] . □

Using this lemma, we have shown that

□

Lemma 8 Suppose there is a sufficiently small constant that upper bounds both log 1 δ n and δ . Given a distribution p , if we construct q as in Case 2 (the small | µ p ∗ n -µ p | case) in Definition 4, then log(1 -d 2 H ( p, q )) ≥ 1 2 n log 4 δ .

Proof of Lemma 8. Recall that given a distribution p , in Case 2 of Definition 4, we construct q by picking one of the two non-unit measures q + and q -which has mass 1 b ≥ 1 . Without loss of generality (via an appropriate reflection of p with respect to µ p ), let q + be this non-unit measure,

then use q = bq + . We use a to denote the corresponding solution to the equation in Definition 4 (note that a > 0 ).

To relate d 2 H ( p, q ) to d 2 H ( p, q + ) , we will need to use the following lemma concerning the generalization of squared Hellinger distance between a distribution and a non-negative measure with mass bigger than 1.

Lemma 25 Given a distribution p , and a non-negative measure q with 1 b ≥ 1 probability mass, define the (extended) squared Hellinger distance as d 2 H ( p, q ) = 1 2 ∫ ( √ d p - √ d q ) 2 . Then, we have

Proof. For any b ′ ≥ 0 , we have

The derivative in b ′ is therefore

The derivative is greater than 0 if and only if

The right hand side is the Bhattacharya coefficient between two distributions, p and bq , and hence is upper bounded by 1 as a standard fact. The left hand side on the other hand is at least 1 for all b ′ ∈ [ b, 1] . Therefore, d 2 H ( p, b ′ q ) is an increasing function in b ′ for the range b ′ ∈ [ b, 1] , meaning that d 2 H ( p, q ) ≥ d 2 H ( p, bq ) , as desired. □

We can now upper bound the squared Hellinger distance as follows.

Observe that for sufficiently small z > 0 , we have log(1 -z ) ≥ -2 z . Since δ and log 1 δ n are assumed to be sufficiently small, we have

C Error analysis for median-of-means

In this section, we present the matching upper bound result of the performance of the median-of-means estimator. We restate the estimator and the proposition for the sake of clarity.

Algorithm 1 Standard Median-of-Means Estimator

Inputs: n independent samples { x i } from an unknown distribution p ; and confidence parameter δ

Divide the samples into 4 . 5log 1 δ groups with equal size.

Compute the mean of each group.

Return the median of these 4 . 5log 1 δ means.

Proposition 14 Consider a distribution p with mean µ p , a sample size n , and a median-of-means group count 4 . 5log 1 δ . Let p ∗ n be the 0 . 45 n log 1 δ -trimmed distribution from Definition 1, and µ p ∗ n and σ p ∗ n be the mean and standard deviation of p ∗ n respectively. Then, the median-of-means estimator has

To prove this proposition, we use the following lemma:

Lemma 26 Defining p ∗ n to be the 0 . 45 n log 1 δ -trimmed version of p , then the empirical mean of n ′ = n 4 . 5log 1 δ samples from p is within 3 σ p ∗ n √ n ′ of µ p ∗ n , except with probability at most 1 5 .

Proof. Recall p ∗ n is p but with 0 . 45 n log 1 δ probability mass trimmed (and then scaled up to have total mass 1). Thus the probability that any of the n ′ = n 4 . 5log 1 δ samples from p are not in the support of p ∗ n is at most 1 10 by the union bound.

Conditioned on the event stated above not happening, we can view the sampling process as drawing n ′ samples from p ∗ n , with mean µ p ∗ n and standard deviation σ p ∗ n . The standard deviation of the mean of these n ′ samples is thus σ p ∗ n √ n ′ . By the Chebyshev inequality, the probability of the sample mean being more than 3 times its standard deviation from the true mean is at most 1 9 . Thus the empirical mean is within 3 σ p ∗ n √ n ′ of µ p ∗ n , except with probability at most 1 9 .

Combining these two case, where the event happens and where it does not, the overall probability of the empirical mean of n ′ samples being more than 3 σ p ∗ n √ n away from the true mean µ p ∗ n is at most 1 10 + 9 10 · 1 9 = 1 5 . □

Proof of Proposition 14. Substituting in n ′ = n 4 . 5log 1 δ to Lemma 26 for each group of the estimator, and with the fact that the mean of p is µ p , we arrive at the conclusion that median-of-means will have error | µ p -µ p ∗ n | +3 σ p ∗ n √ 4 . 5log 1 δ n except when at least half of the 4 . 5log 1 δ groups have error > 3 σ p ∗ n √ 4 . 5log 1 δ n . Since the probability of each mean having a big error is at most 1 5 by Lemma 26, the overall failure probability is thus at most the probability that 4 . 5log 1 δ coins each of bias 1 5 will yield majority heads. Letting k = 4 . 5log 1 δ , we bound this probability with a Chernoff bound, which

for our choice of t ≥ 0 , yields ( 4 5 e 0 + 1 5 e t ) k e -tk 2 . The upper bound is minimized at t = log 4 , attaining its minimum of ( 4 5 ) k . Since k = 4 . 5log 1 δ , this probability is less than δ . □

D Definition of instance optimality and admissibility

This paper needs a suitable notion of optimality beyond the worst case. One natural definition to consider is the notion of instance optimality (see Definition 27 in Appendix D) from the computer science literature [FLN01], which intuitively states that 'our algorithm performs at least as well on any instance p as any algorithm customized to p '. However, it is immediate that no algorithm A can satisfy such a definition in our setting-for every distribution p , there is a trivial estimator that is hardcoded to output the mean µ p without looking at any data; and this hardcoded estimator beats any other estimator A . On the other hand, the statistics literature commonly uses a different natural notion called admissibility (see Definition 28 in Appendix D, also analogous to the economics notion of Pareto efficiency ), which states that 'no algorithm can perform at least as well as our algorithm, and strictly outperforms our algorithm on some instance.' While instance optimality is impossible to satisfy, admissibility has the dual problem of being somewhat too weak and too easy to satisfy: a trivial estimator which outputs a hardcoded mean estimate-ignoring any samples-is admissible.

In this short appendix, we give the formal definitions of instance optimality and admissibility for mean estimation over R .

Recall the notation P 1 for the set of distributions over R with a finite mean.

Definition 27 ( κ -Instance Optimality in Mean Estimation) For a parameter κ > 1 , sample complexity n and failure probability δ , a mean estimator ˆ µ whose error function is ϵ n,δ is κ -instance optimal if, for any distribution p ∈ P 1 , every estimator ˆ µ ′ has error at least 1 κ ϵ n,δ ( p ) with probability 1 -δ over n i.i.d. samples from p .

As we remarked in the introduction, there is no instance optimal mean estimator for any κ > 0 , since for any estimator ˆ µ , we can find a distribution p on which it has some nonzero error, and thus ˆ µ cannot be instance optimal, because it performs infinitely worse on p in comparison with the trivial 'hardcoded' estimator that always outputs µ p without looking at any samples.

Definition 28 (Admissibility in Mean Estimation) For sample complexity n and failure probability δ , a mean estimator ˆ µ whose error as a function of the distribution p is ϵ n,δ ( p ) , is called 'admissible' if there is no estimator ˆ µ ′ with error function ϵ ′ n,δ such that

For every distribution p ∈ P 1 , ϵ ′ n,δ ( p ) ≤ ϵ n,δ ( p )

There exists a distribution p ∗ ∈ P 1 such that ϵ ′ n,δ ( p ∗ ) < ϵ n,δ ( p ∗ )

E Hardcoded estimators are not neighborhood optimal

We perform the basic 'sanity check' for the neighborhood structure of Definition 15, and formally show that under this neighborhood definition, no trivial hardcoded estimator is κ -neighborhood optimal for any κ > 1 .

Proposition 29 Consider a mean estimator ˆ µ β which ignores any of its inputs and always outputs the value β for some β ∈ R . For any parameter κ > 1 , ˆ µ β cannot be κ -neighborhood optimal with respect to N n,δ defined in Definition 15.

Proof. For the sake of contradiction, suppose ˆ µ β is κ -neighborhood optimal. The error function ϵ β for ˆ µ β is simply ϵ β ( p ) = | µ p -β | . Since ˆ µ β is κ -neighborhood optimal, there must exist some error function ϵ pareto ≥ 1 κ ϵ β such that ϵ pareto is a neighborhood Pareto bound with respect to N n,δ . We will reach a contradiction by showing that ϵ pareto cannot be a neighborhood Pareto bound.

Pick an arbitrary distribution p whose mean µ p is β +(1 + κ 2 ) · ϵ n,δ ( p ) . This is always possible since ϵ n,δ ( p ) is translation-invariant in p . By Property 3 of Definition 15 and the reverse triangle inequality, any q ∈ N n,δ ( p ) has mean µ q such that ϵ β ( q ) = | µ q -β | ≥ κ 2 ϵ n,δ ( p ) . Since ϵ pareto

is a neighborhood Pareto bound, there must be no estimator ˜ µ such that for every distribution q ∈ N n,δ ( p ) ∪ { p } , | ˜ µ -µ q | < κϵ n,δ ( p ) with probability 1 -δ over n samples from q ; otherwise, we would have | ˜ µ -µ q | < κϵ n,δ ( p ) ≤ 1 κ | µ q -β | = 1 κ ϵ β ( q ) ≤ ϵ pareto ( q ) , which contradicts ϵ pareto being a neighborhood Pareto bound. However, we can simply pick ˜ µ = ˆ µ µ p , the trivial estimator that outputs µ p always. Again by Property 3 of Definition 15, we have for every q ∈ N n,δ ( p ) ∪{ p } that | µ q -˜ µ | = | µ q -µ p | ≤ ϵ n,δ ( p ) < κϵ n,δ ( p ) for κ > 1 . We have thus reached the desired contradiction. □

F Remaining proofs for Lemma 17

In Section 2, we have already shown that the construction of q as in Definition 4 satisfies properties 2 and 4 of Definition 15. To complete the proof of Lemma 17, we show that q satisfies property 1 in Appendix F.1, and property 3 in Appendix F.2.

F.1 Upper bounding | µ q -µ p |

We show in this section that, for both cases in Definition 4, the construction of q is such that | µ q -µ p | is appropriately upper bounded. We will show that | µ q -µ p | ≤ ϵ n,δ ( p ) , as part of the proof showing that q ∈ N n,δ ( p ) . We furthermore show other upper bounds of | µ q -µ p | , for example, that | µ q -µ p | ≤ r 4 if r is the trimming radius for the construction of p ∗ n from p , which will be useful for the later sections.

Lemma 30 Assuming p has mean 0, and is trimmed to the interval [ -r, r ] when constructing p ∗ n (as in Definition 1), then Case 1 (the large | µ p ∗ n -µ p | case) of Definition 4 outputs a distribution q such that | µ q -µ p | ≤ r 4 and | µ q -µ p | ≤ 1 4 ϵ n,δ ( p ) .

Proof. The distribution q is constructed to be a convex combination of p and p ∗ n , with interpolation parameter 1 -λ equal to 1 4 by definition. Thus | µ q -µ p | ≤ 1 4 | µ p ∗ n -µ p | , where this last quantity is at most r 4 since p ∗ n is supported on [ -r, r ] and thus can have mean at most distance r away from the origin. Furthermore, by the definition of ϵ n,δ ( p ) , we have | µ q -µ p | ≤ 1 4 | µ p ∗ n -µ p | ≤ 1 4 ϵ n,δ ( p ) . □

Lemma 31 Assuming p has mean 0, and is trimmed to the interval [ -r, r ] when constructing p ∗ n (as in Definition 1), then Case 2 (the small | µ p ∗ n -µ p | case) of Definition 4 outputs a distribution q such that | µ q -µ p | ≤ 1 σ p ∗ √ log 1 δ . As a corollary, this is further upper bounded by ϵ n,δ ( p ) .

Additionally, if log 1 δ n is upper bounded by some sufficiently small absolute constant, then 1 8 σ p ∗ n √ log 1 δ n can also be bounded by r 4 .

Proof. Without loss of generality, suppose q + is the non-unit measure constructed by Case 2 of Definition 4 which will be downscaled to create q . We know from Lemma 23 that ∣ ∣ ∣ ∫ ∞ -∞ x d q + ∣ ∣ ∣ =

For the other corollary, observe that since p ∗ n 's support is in [ -r, r ] , σ p ∗ n is then upper bounded by 2 r . Therefore, under the assumption that log 1 δ n is bounded by a sufficiently small constant, we have 1 8 σ p ∗ n √ log 1 δ n ≤ r 4 . □

F.2 Showing that ϵ n/ 3 ,δ ( q ) ≤ O ( ϵ n,δ ( p ))

For the second item, the proof of Lemma 36 is relatively self-contained, though requiring some minute calculations. On the other hand, the analysis of | µ q -µ q ∗ n/ 3 | is non-trivial. To upper bound this term, we consider an intermediate (non-unit) measure ˜ q resulting from trimming q to the same interval as p ∗ n . We then show Lemma 32 which bounds the difference between first moments of q ∗ n/ 3 and ˜ q , and Lemmas 33 and 34 which bound the difference between the first moments of q and ˜ q in the two cases of the construction of q . The combination of these three lemmas gives Lemma 35, which bounds | µ q -µ q ∗ n/ 3 | as desired.

In much of the analysis in this section, we will assume without loss of generality that p has mean 0, is trimmed to the unit interval [ -1 , 1] when constructing p ∗ n .

Lemma 32 Let n be the number of samples and δ be the failure probability, and suppose there is a sufficiently small absolute constant which upper bounds log 1 δ n . Given a distribution p with µ p = 0 and whose trimming radius for constructing p ∗ n is equal to 1, suppose the distribution q is such that d q d p ≤ 2 and | µ q | ≤ 1 4 . Letting q ∗ n/ 3 be the ( 1 . 35 n log 1 δ ) -trimmed version of distribution q , trimming to some interval [ µ q -r q , µ q + r q ] , then:

When [ µ q -r q , µ q + r q ] is a subset of [ -1 , 1] then

When [ µ q -r q , µ q + r q ] extends outside [ -1 , 1] then

Proof of the first bullet. Define S = [ -1 , 1] \ [ µ q -r q , µ q + r q ] . Firstly, due to triangle inequality, we have

The Cauchy-Schwarz inequality then says that

Note that the second integral in the right hand side is bounded by the amount of trimmed probability mass, which is at most 1 . 35log 1 δ n . The first integral, since d q d p ≤ 2 , is bounded by twice the variance of p in [ -1 , 1] , namely 2 σ p ∗ n 2 . Thus the product of the square roots of the two integrals on the right hand side is bounded by σ p ∗ n √ 2 1 . 35log 1 δ n , yielding the desired bound. □

Proof of the second bullet. Without loss of generality, we assume µ q ≥ µ p = 0 . Thus by the assumption of this case, that [ µ q -r q , µ q + r q ] extends outside [ -1 , 1] , we have that µ q + r q > 1 , because of the asymmetry that µ q ≥ 0 . Since µ q ≤ 1 4 by the lemma assumption, we also have r q ≥ 3 4 .

̸

Recall that p has a total of 0 . 45log 1 δ n mass outside [ -1 , 1] by the lemma assumption, and that d q d p ≤ 2 . This implies that q has at most 0 . 9log 1 δ n mass outside [ -1 , 1] . As 1 . 35log 1 δ n mass is trimmed from q to construct q ∗ n/ 3 , there is thus at least 0 . 45log 1 δ n mass trimmed from q inside the interval [ -1 , 1] . Furthermore, since d q d p ≤ 2 , we conclude that there is thus at least 0 . 225log 1 δ n mass trimmed from p inside the interval [ -1 , 1] . Now denote the set on which q is trimmed, restricted to [ -1 , 1] by S = [ -1 , 1] \ [ µ q -r q , µ q + r q ] . Since p has at least 0 . 225log 1 δ n mass in S , meaning that S = ∅ , and µ q + r q > 1 from earlier, it must thus be the case that µ q -r q > -1 and therefore r q < 5 4 .

Since min x i ∈ S | x i | > 1 2 , we conclude that ( σ p ∗ n ) 2 + ( µ p ∗ n ) 2 = ∫ 1 -1 x 2 d p ∗ n ≥ ∫ 1 -1 x 2 d p ≥ ∫ S x 2 d p ≥ c 2 2 · 20 log 1 δ n . Using the standard inequality that the ℓ 1 norm is at least the ℓ 2 norm, we have that σ p ∗ n + µ p ∗ n ≥ √ ( σ p ∗ n ) 2 +( µ p ∗ n ) 2 ≥ √ c 2 2 · 20 log 1 δ n . Thus, since √ 4 . 5log 1 δ n ≤ 1 3 by the lemma assumption, we have that ϵ n,δ ( p ) = σ p ∗ n √ 4 . 5log 1 δ n + µ p ∗ n ≥ √ 9 80 4 . 5log 1 δ n . This yields the second inequality in the second bullet point.

We now prove the first inequality in the second bullet, which bounds the difference between two integrals; we bound this by bounding the probability mass in each integral and multiplying this by a bound on the integrand of each integral. Explicitly, the amount of probability mass of q that is in [ -1 , 1] but outside of [ µ q -r q , µ q + r q ] is at most 1 . 35log 1 δ n , and furthermore, for every x ∈ [ -1 , 1] \ [ µ q -r q , µ q + r q ] , we have | x -µ q | ≤ 5 4 due to | µ q | ≤ 1 4 . In parallel, the amount of probability mass of p that is outside of [ -1 , 1] but in [ µ q -r q , µ q + r q ] is at most 0 . 45log 1 δ n , and since d q d p ≤ 2 , we conclude that q has at most 0 . 9log 1 δ n mass outside of [ -1 , 1] but in [ µ q -r q , µ q + r q ] . Also, for every x ∈ [ µ q -r q , µ q + r q ] \ [ -1 , 1] , we have | x -µ q | ≤ 5 4 because r q ≤ 5 4 . Thus, the left hand side of the second bullet is at most (2 + 3) 5 4 0 . 45 log 1 δ n = 2 . 8125 log 1 δ n , as claimed. □

Lemma 33 Given a distribution p with µ p = 0 and whose trimming radius for constructing p ∗ n is equal to 1, let q be the distribution constructed from p as in Case 2 (the small | µ p ∗ n -µ p | case) of

Proof. Notice that

and since | µ q | ≤ 1 8 σ p ∗ n √ log 1 δ n by Lemma 31, we only have to bound the first term by some constant multiple of σ p ∗ n √ log 1 δ n .

Recall the construction of q in Definition 4, as a downscaled version of either q + or q -. Without loss of generality (by reflecting p and q as appropriate), we assume we use q + . Specifically, q + is the non-unit measure such that d q + d p ( x ) = 1 + min(1 , max( -1 , ax )) for some value a specified in Definition 4.

Since q is a downscaled version of q + , we will bound ∣ ∣ ∣ ∫ R \ [ -1 , 1] x d q ∣ ∣ ∣ ≤ ∣ ∣ ∣ ∫ R \ [ -1 , 1] x d q + ∣ ∣ ∣ by a constant multiple of σ p ∗ n √ log 1 δ n .

Observe that for any a > 0 , we have that x and min(1 , max( -1 , ax )) have the same sign everywhere, which means

Additionally, because log 1 δ n is upper bounded by a sufficiently small absolute constant and | µ p ∗ n | ≤ σ p ∗ n √ 4 . 5 n log 1 δ from the lemma assumption, we get | µ p ∗ n 1 -0 . 45 n log 1 δ | ≤ 2 σ p ∗ n √ 0 . 45 n log 1 δ ≤ 4 . 5 σ p ∗ n √ log 1 δ n .

We can now upper bound ∣ ∣ ∣ ∫ R \ [ -1 , 1] x d q + ∣ ∣ ∣ as follows.

□

Lemma 34 Given a distribution p with µ p = 0 and whose trimming radius for constructing p ∗ n is equal to 1, let q be the distribution constructed from p as in Case 1 (the large | µ p ∗ n -µ p | case) of Definition 4. Then ∣ ∣ ∫ R \ [ -1 , 1] ( x -µ q ) d q ∣ ∣ ≤ 5 | µ p ∗ n -µ p |

Proof. From the construction, q outside [ -1 , 1] just consists of the corresponding portion of p multiplied by λ ∈ (0 , 1) . Thus

Since q is an interpolation between p and p ∗ n , the mean of q is in between the mean of p and the mean of p ∗ n , and thus | µ p -µ q | ≤ | µ p ∗ n -µ p | . Furthermore, since 1 n log 1 δ is upper bounded by some sufficiently small absolute constant, the entire bound above can be finally bounded by 5 | µ p ∗ n -µ p | , as desired. □

Lemma 35 Given a distribution p with µ p = 0 and whose trimming radius for constructing p ∗ n is equal to 1, consider constructing q according to Definition 4. Recall also the notation for the distribution q ∗ n/ 3 which is the ( 1 . 35 n log 1 δ ) -trimmed version of q as in Definition 1, and suppose q ∗ n/ 3

is formed by trimming q to some interval [ µ q -r q , µ q + r q ] . Assuming that log 1 δ n is upper bounded by some sufficiently small absolute constant, then | µ q -µ q ∗ n/ 3 | ≤ 50 ϵ n,δ ( p ) .

Proof. By the definition of q ∗ n/ 3 ,

Further noting that ∫ R \ [ µ q -r q ,µ q + r q ] µ q d q = 1 . 35log 1 δ n µ q , since we trim exactly 1 . 35log 1 δ n probability mass outside of [ µ q -r q , µ q + r q ] , we can rearrange the above to get

As a result, to bound | µ q -µ q ∗ n/ 3 | , it suffices to bound ∣ ∣ ∣ ∫ R \ [ µ q -r q ,µ q + r q ] ( x -µ q ) d q ∣ ∣ ∣ . By the triangle inequality, we have that

and we bound the two terms separately.

By Lemmas 33 and 34, we have ∣ ∣ ∣ ∫ R \ [ -1 , 1] ( x -µ q ) d q ∣ ∣ ∣ ≤ 5 | µ p -µ p ∗ n | +5 σ p ∗ n √ log 1 δ n in either case of the construction of q .

To apply Lemma 32, we need to verify that the construction of q is such that | µ q -µ p | ≤ 1 / 4 when the trimming radius for constructing p ∗ n from p is 1. This was shown in Lemmas 30 and 31. Therefore, by the two cases of Lemma 32, we have ∣ ∣ ∣ ∫ 1 -1 ( x -µ q ) d q -∫ µ q + r q µ q -r q ( x -µ q ) d q ∣ ∣ ∣ ≤ max ( | µ p -µ q | + √ 3 5 σ p ∗ n √ 4 . 5log 1 δ n , 2 ϵ n,δ ( p ) ) . From Lemmas 30 and 31 again, we have that | µ p -µ q | ≤ ϵ n,δ ( p ) .

Combining all these bounds, we have

where the last inequality uses the definition of ϵ n,δ ( p ) = | µ p -µ p ∗ n | + σ p ∗ n √ 4 . 5log 1 δ n . □

Lemma 36 Given a distribution p with µ p = 0 and whose trimming radius for constructing p ∗ n is equal to 1, consider any distribution q such that d q d p ≤ 2 and | µ q | ≤ 1 4 . Recall the notation for the distribution q ∗ n/ 3 which is the ( 1 . 35 n log 1 δ ) -trimmed version of q as in Definition 1, and suppose q ∗ n/ 3 is formed by trimming q to some interval [ µ q -r q , µ q + r q ] . Assuming that log 1 δ n is upper bounded by some sufficiently small absolute constant, then σ q ∗ n/ 3 ≤ 50( σ p ∗ n + | µ p -µ p ∗ n | ) .

Proof. Without loss of generality, assume that µ q ≥ 0 . In the following proof, we will aim to upper bound E X ← q ∗ [ X 2 ] by 57(( σ p ∗ n ) 2 + ( µ p ∗ n ) 2 ) , which implies σ q ∗ n/ 3 ≤ √ E X ← q ∗ [ X 2 ] ≤ √ 57 √ ( σ p ∗ n ) 2 +( µ p ∗ n ) 2 ≤ 8( σ p ∗ n + | µ p ∗ n | ) , yielding the lemma statement.

Before we bound E X ← q ∗ [ X 2 ] , we first show that by the assumption of µ q ≥ 0 , it must be the case that µ q -r q ≥ -1 .

Recall that p has a total of 0 . 45log 1 δ n mass outside [ -1 , 1] by the lemma assumption, and that d q d p ≤ 2 . This implies that q has at most 0 . 9log 1 δ n outside [ -1 , 1] . Since 1 . 35log 1 δ n mass is trimmed from q to construct q ∗ n/ 3 , there is thus at least 0 . 45log 1 δ n mass trimmed from q inside the interval [ -1 , 1] .

Since µ q ≥ µ p = 0 , it must be the case that µ q -r q ≥ -1 ; otherwise r q ≥ 1 and hence µ q + r q ≥ 1 , which would mean that [ µ q -r q , µ q + r q ] completely contains [ -1 , 1] , contradicting the fact that q trims non-zero mass from the interval [ -1 , 1] .

We will now start bounding E X ← q ∗ [ X 2 ] .

We can bound the two integrals separately. First:

Second, we will bound ∫ max(1 ,µ q + r q ) 1 x 2 d p . If µ q + r q ≤ 1 then the integral is 0. Otherwise, we have µ q + r q > 1 , and we have to bound the above integral, using the following bounds on r q , µ q and the second moment of p ∗ n .

Consider the mass of q that was trimmed from within [ -1 , 1] . Since µ q + r q > 1 , the (at least 0 . 45log 1 δ n ) mass that was trimmed from q must be within [ -1 , µ q -r q ] . Further recall that d q d p ≤ 2 , which implies that p has at least 0 . 225log 1 δ n mass within [ -1 , µ q -r q ] . Since the trimming interval for constructing p ∗ n from p is [ -1 , 1] , and p ∗ n is constructed from scaling up the trimmed version of p , we conclude that p ∗ n also has at least 0 . 225log 1 δ n mass within [ -1 , µ q -r q ] .

As we assumed that µ q ≤ 1 4 in the lemma statement, we have from µ q + r q > 1 that r q ≥ 3 4 . This furthermore implies that | µ q -r q | ≥ 1 2 . Thus the ≥ 0 . 225log 1 δ n probability mass of p ∗ n in [ -1 , µ q -r q ] contributes at least 1 2 2 0 . 225log 1 δ n to the second moment of p ∗ n ; namely E X ← p ∗ n [ X 2 ] ≥ 0 . 225 4 log 1 δ n . We can also bound r q ≤ 5 4 , since µ q -r q > -1 and µ q ≤ 1 4 ; thus µ q + r q ≤ 3 2 .

Therefore,

Summarizing, when log 1 δ n is bounded by some sufficiently small absolute constant, we have

which yields the lemma, by the argument at the very beginning of the proof.

□

Lemma 37 Consider constructing distribution q from p according to Definition 4. Assuming that log 1 δ n is upper bounded by some sufficiently small absolute constant, then ϵ n/ 3 ,δ ( q ) ≤ 100 ϵ n,δ ( p ) .

Proof. Since the construction of q in Definition 4 satisfies the assumptions of Lemma 36, this lemma follows directly from summing up the bounds of Lemmas 35 and 36, that

ϵ n/ 3 ,δ ( q ) = | µ q -µ q ∗ n/ 3 | + σ q ∗ n/ 3 √ 4 . 5log 1 δ n ≤ 50 ϵ n,δ ( p )+50( σ p ∗ n + | µ p -µ p ∗ n | ) √ 4 . 5log 1 δ n ≤ 100 ϵ n,δ ( p ) where the last inequality uses the definition of ϵ n,δ ( p ) = | µ p -µ p ∗ n | + σ p ∗ n √ 4 . 5log 1 δ n and that log 1 δ n is bounded by a sufficiently small constant. □

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies one-dimensional mean estimation in a beyond-worst case setting.\n\nThe main contributions are twofold. First, it gives indistinguishability results which, given a distribution p of finite mean, construct a “partner” distribution q that preserves the moments yet has mean far from p. This implies the subguassian rate via median-of-means is optimal in general. Hence, this result can be viewed as an impossibility theorem against beyond-worst-case analysis for mean estimation.\n\nSecond, they give a new framework of neighborhood optimality, and show that median-of-means is optimal up to a constant.\n\nSTRENGTHS:\nThe question studied here is well motivated. I believe this is the natural next step following the recent work of Lee-Valiant (2020) and early papers on median-of-means. The paper essentially settles the beyond-worst-case analysis problem in this space (in the negative direction)\n\nThis is a quite solid paper at a technical level. The main contribution is interesting and to me somewhat surprising.  I didn’t not check the proof details from the appendix, but reading the main body the main claims appear correct.\n\nOverall the paper is well-written.\n\nWEAKNESSES:\nI would be most interested in an extension of the result to the multivariate setting. \n\nOther minor comments below.\n\nBy “worst case”, this paper refers to the worst over all distributions (under certain moment constraints). I think this should be remarked early in the introduction, since beyond-worst-case analysis in the TCS community typically refers to non-worst-case data. In our case, the data are drawn from a distribution already. So we are not using this notion in the classic sense.\n\nAbstract line 17:  The construction of q preserves the density of p up to a factor of 2 -> I would just state it formally: “the construction ensures dq / dp <= 2 at all points, thus [...]”\n\nQUESTIONS:\nThis indistinguishability claim reminds one of the sample amplification framework proposed by https://arxiv.org/abs/1904.12053. It might be worth a remark if there’s any formal connection. \n\nIn the related work section, I would suggest citing this survey: https://arxiv.org/abs/1906.04280\n\nLIMITATIONS:\nIt's addressed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is about analyzing the optimal bounds for mean estimation, beyond the worst-case setting given in the recent breakthrough result of Lee and Valiant giving optimal subgaussian error rates for all distributions with unknown, finite variance. This prior work shows only that there exists a single \"problematic\" distribution class, for which the given error rates are best possible. Thus, a natural approach to make progress is to prove instance-dependent bounds for estimating the mean that are able to outperform the worst-case bounds at least for some distributions. The main result of this paper is that, assuming only that the distribution has finite mean, this natural approach cannot asymptotically improve over the worst-case optimal subgaussian error rates. In particular, given a distribution $p$, a failure probability $\\delta$ and a number of samples $n$, the authors construct a distribution $q_{n,\\delta}$ such that: \n1. $q_{n,\\delta}$ cannot be distinguished from $p$ with $n$ samples and success probability $1-\\delta$.\n2. The means of $q_{n,\\delta}$ and $p$ differ by an amount that asymptotically approaches the optimal worst-case subgaussian bound.\nTogether, these two properties show that instance-dependent bounds in this setting offer no improvement. The paper also introduces a framework called \"neighborhood optimality\" which interpolates between the (too weak) notion of admissibility and the (too strong) instance-optimality.\n\nSTRENGTHS:\nThe paper is well-written and gives and intuitive, straight-forward construction showing that it is not possible to improve on the worst-case optimal subgaussian bounds assuming only that the distribution has a finite mean. This is a valuable contribution by itself, and also has to deal with subtle technical difficulties arising in the case that the distribution has small mass very far from the mean, which none-the-less contributes significantly the the variance.\n\nFurther illustrating the utility of such a construction is the fact that follow-up work has already utilized the intuition behind it to prove improved bounds for symmetric distributions. In particular, the construction relies heavily on the ability to consider distributions with tails that are very skewed in one direction, and so symmetric distributions are a natural candidate for improved bounds guided by this construction.\n\nWEAKNESSES:\nThe neighborhood-optimality framework seems interesting, but the results in the paper so far are not entirely convincing that this is the \"right\" way to go beyond the worst-case for mean estimation. However, the construction showing that subgaussian bounds are optimal when there is a finite, unknown mean is in my view the main contribution and quite significant on its own. So any potential limitations of the neighborhood-optimality framework are not much of drawback in my view.\n\nQUESTIONS:\nMinor comment:\nLine 203 states that $q$ satisfies all the properties of definition $N_{n,\\delta}(p)$, but as far as I can tell $N_{n,\\delta}(p)$ hasn't been defined yet.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the fundamental task of 1-dimensional mean estimation. There has been recent interest in the community to understand “beyond worst-case” analyses of statistical problems, where guarantees are able to encompass properties of instances that may make them more tractable than worst-case instances. This work motivates and initiates such analysis for mean estimation. The main result aims to show negative evidence for this perspective: proving that for any distribution $p$ with finite mean there is another distribution $q$ where it is hard to distinguish $p$ vs $q$ and their means are separated by the sub-Gaussian rate. Moreover, in many senses $p$ and $q$ are “similar”, so this indicates hardness to beat the sub-Gaussian rate even when just designing an algorithm to perform on $p$ and “similar” distributions. This perspective is further developed by the formalization of “neighborhood optimality” and median-of-means is shown to be approximately neighborhood optimal.\n\nSTRENGTHS:\nIt is an extremely fundamental pursuit to understand mean estimation and what nice properties of distributions permit better rates. The perspective introduced in this work of analyzing the optimal algorithms that perform on some distribution $p$ and “similar” distributions is both novel and insightful for progress in this pursuit. \n\nWhile this paper mostly focuses on one (relatively well-motivated) notion of “similarity”, the landscape of analogous results clearly is dependent on what similarity notion is used. This is perhaps one of the most appealing aspects of the paper, as it gives a concrete framework within which to use notions of similarity as proxies for the properties that make distributions nicer in the beyond worst-case sense. As an example, the notion of similarity in this paper enforces $\\frac{dq}{dp} \\le 2$, indicating that the similar distributions which are still hard have similar moments and thus limiting how much moment-based guarantees can make distributions more tractable. On the other hand, the later work of [GLP23] (I believe) implies that analogous results cannot hold when similarity preserves both symmetry and Fisher information, highlighting how permissible similarity notions in this framework may help demarcate helpful properties.\n\nWEAKNESSES:\nAlthough likely outside the reasonable scope of this work, the existence of results such as [GLP23] indicate how this perspective would benefit from deeper investigations into various neighborhood functions.\n\nFrom the exposition, it is not immediately clear how conducive the neighborhood optimality perspective is towards thinking about tightness without constant-factor lossiness (the paper seems to hint that proving a guarantee without such lossiness may be a reasonable-sounding future direction).\n\nQUESTIONS:\nCould you further explain your claim on line 70 that your paper shows introducing extra assumptions are necessary for the [GLP23] results? A naive reading of this claim feels strong in that worst-case lower bounds already show the need for additional assumptions, but perhaps the desired claim is that your work shows the assumptions must not be qualities that $q$ preserves in your transformation? Relatedly, does [GLP23] imply that it is open whether a result similar to your Theorem 2 is possible where $q$ additionally preserves (i) symmetry, or (ii) Fisher information, but it is impossible to (iii) simultaneously preserve symmetry and Fisher information?\n\nDo you have more intuition to share regarding whether it is a reasonable goal to naturally hope for $(1+o(1), 1+o(1))$-neighborhood optimality?\n\nAs some of the motivation for neighborhood-optimality originates from failing the hardcoded estimator, are there known references that alternatively tackle this by restricting the estimator to have desirable properties (e.g., being translation invariant)?\n\nMinor details:\n* On line 380, is the mention of $\\log(1-d^2_H(p,q)) \\ge \\frac{1}{2n} \\log 4 \\delta$ redundant from $q \\in N_{n,\\delta}(p)$?\n* The equation above line 391 has an extra pair of parentheses\n\nLIMITATIONS:\nLimitations are well-discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper studies the 1-dimensional robust mean estimation from the perspective of "beyond the worst-case analysis". They show somehow a negative result, that is, for any distribution, we can always construct another distribution that is close in probability distance, but their means are well-separated. Furthermore, to analyze the fine-grained optimality of algorithms, they propose a new definitional framework called “neighborhood optimality”. They show that the classic MoM estimator is neighborhood optimal up to some constant.

STRENGTHS:
- This paper studies the 1-dimensional robust mean estimation from the perspective of "beyond the worst-case analysis", which is a very important and interesting view for this problem.

- Their construction might be adapted in other settings.

WEAKNESSES:
- As discussed in the paragraph (lines 69-76), there exists a class of distributions (symmetric distributions) whose error rate can be better than the sub-Gaussian rate. My question is can we modify the current construction to reflect this result by restricting the feasible domain of the perturbed distributions?

QUESTIONS:
N/A

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07