Centroid Approximation for Bootstrap

Bootstrap is a principled and powerful frequentist statistical tool for uncertainty quantification . Unfortunately , standard bootstrap methods are computationally intensive due to the need of drawing a large i.i.d . bootstrap sample to approximate the ideal bootstrap distribution ; this largely hinders their application in large-scale machine learning , especially deep learning problems . In this work , we propose an efficient method to explicitly optimize a small set of high quality “ centroid ” points to better approximate the ideal bootstrap distribution . We achieve this by minimizing a simple objective function that is asymptotically equivalent to the Wasserstein distance to the ideal bootstrap distribution . This allows us to provide an accurate estimation of uncertainty with a small number of bootstrap centroids , outperforming the naive i.i.d . sampling approach . Empirically , we show that our method can boost the performance of bootstrap in a variety of applications . 1 INTRODUCTION . Bootstrap is a simple and principled frequentist uncertainty quantification tool and can be flexibly applied to obtain data uncertainty estimation with strong theoretical guarantees ( Hall et al. , 1988 ; Austern & Syrgkanis , 2020 ; Chatterjee et al. , 2005 ; Cheng et al. , 2010 ) . In particular , when combined with the maximum likelihood estimator or more general M-estimators , bootstrap provides a generalpurpose , plug-and-play non-parametric inference framework for general probabilistic models without case-by-case derivations ; this makes it a promising frequentist alternative to Bayesian inference . However , the standard bootstrap inference is highly expensive in both computation and memory as it typically requires drawing a large number1 of i.i.d . bootstrap particles ( samples ) to obtain an accurate uncertainty estimation . In the context of this paper , as each bootstrap particle/sample/centroid is a machine learning model , we might directly call a model as particle/sample/centroid . With a small number of particles , bootstrap may perform poorly . As a consequence , when applied to deep learning , we need to store a large number of neural networks and feed the input into a tremendous number of networks every time we make inference , which can be quite expensive and even unaffordable for deep learning problems with huge models2 . For example , in autonomous driving applications , our device can only store a limited number of models and we need to make decisions within a short time , which makes the standard bootstrap with a large number of models no more feasible . Typical ensemble methods in deep learning , such as Lakshminarayanan et al . ( 2017 ) ; Huang et al . ( 2017 ) ; Vyas et al . ( 2018 ) ; Maddox et al . ( 2019 ) ; Liu & Wang ( 2016 ) , can only afford to use a small number ( e.g. , less than 20 ) of models . Therefore , to make bootstrap more accessible in modern machine learning , it is essential to develop new approaches that break the key computation and memory barriers mentioned above . This paper aims to improve the bootstrap when the resource at inference is limited . We are motivated to consider the following problem : How to improve the accuracy of bootstrap inference when the number of particles is limited ? We attack this challenge by presenting an efficient centroid approximation for bootstrap . Our method replaces the i.i.d . bootstrap particles with a set of carefully optimized centroid particles that are 1For example , thousands of , as suggested by Statistics textbooks such as Wasserman ( 2013 ) . 2While training cost is an extra burden , it is small compared with the cost of making prediction as we only need to train the model once but make countless predictions at deployment . guaranteed to provide an accurate and compact approximation to the ideal bootstrap distribution so that only a smaller number of particles is needed to obtain good performance . Our method is based on minimizing a specially designed objective function that is asymptotically equivalent to the Wasserstein distance between the ideal bootstrap distribution and the particle distribution formed by the learned centroids . During the training , each centroid adjusts its location being aware of the locations of the others so that centroids are diversified and well distributed on the domain . Our method is similar to doing K-means on the ideal bootstrap distribution , finding K representative centroids that well represent K separate parts of the target distribution ’ s domain in an optimal way . As centroids are optimized to better approximate the distribution , our approach naturally improves over the vanilla bootstrap with i.i.d . particles . See Figure 1 for illustration . Empirically , we apply the centroid approximation method to various applications , including confidence interval estimation ( DiCiccio et al. , 1996 ) , bootstrap method for contextual bandit ( Riquelme et al. , 2018 ) , bootstrap deep Q-network ( Osband et al. , 2016 ) and bagging method ( Breiman , 1996 ) for neural networks . We find that our method consistently improves over the standard bootstrap . 2 BACKGROUND . Suppose we have a model fθ parameterized by θ in a parameter space Θ ⊆ Rd . Let { xi } ni=1 ⊂ X be a training set with n data points on X . Assume ` ( x , fθ ) is the negative log-likelihood of data point x with model fθ . A standard approach to estimate θ is maximum likelihood estimator ( MLE ) , which minimizes the negative log-likelihood function ( loss ) over the training set θ̂ = arg min θ∈Θ L ( θ ) , L ( θ ) = ∑n i=1 ` ( xi , fθ ) /n . Here the MLE θ̂ provides a point estimation without any information on the data uncertainty . Bootstrap is a simple and effective frequentist method to quantify the uncertainty . The bootstrap loss is a randomly perturbed loss defined as Lw ( θ ) = ∑n i=1wi ` ( xi , fθ ) /n , where w = [ w1 , ... , wn ] > is a set of random weights of data points drawn from some distribution π . A typical choice of π is the multinomial distribution with uniform probability , which corresponds to resampling on the training set with replacement . Given w , one can calculate its associated bootstrap particle by minimizing the bootstrap loss : θ̂w = arg min θ∈Θ Lw ( θ ) . ( 1 ) Let ρπ be the distribution of θ̂w when w ∼ π. Bootstrap theory indicates that we can quantify the data uncertainty of θ or any function g ( θ ) using ρπ . We call ρπ the ideal bootstrap distribution and it is the main object we want to approximate . Denote δθ as the delta measure centered at θ . Standard bootstrap method approximates ρπ by the particle distribution ρ̂π ( · ) = ∑m j=1δθ̂wj ( · ) /m formed by m i.i.d . particles { θ̂wj } mj=1 , which can be obtained by drawing m i.i.d . weights { wj } mj=1 from π and calculating each θ̂wj based on ( 1 ) . However , for deep learning applications , as discussed in the introduction , storing and making inference using a large number m of bootstrap particles can be quite expensive . On the other hand , if m is small , ρ̂π tends to be a poor approximation of ρπ . In this paper , we aim to improve the approximation of the particle distribution when m is small . 3 METHOD . Our idea is simple . Instead of using i.i.d . particles , in which the location of each particle is independent from that of the others , we try to actively optimize the location of each particle so that particles are diversified , better distributed and eventually providing a particle distribution with improved approximation accuracy . A natural way to achieve this goal is to explicitly optimize a set of points { θj } mj=1 ( called centroids ) jointly such that the Wasserstein distance between ρπ and the induced particle distribution is minimized : { θ∗j , v∗j } mj=1 = arg min θ1 , ... , θm∈Θ , [ v1 , ... , vm ] ∈Cm W2 [ ∑m j=1vjδθj , ρπ ] . ( 2 ) Here we consider a Wasserstein distanceW2 equipped with a special data-dependent distance metric || · ||D that will be introduced later in ( 5 ) . We note that here we also optimize the probability weights { vj } mj=1 of the centroids . Finding the optimal centroids and probability weights can be decomposed into two steps : the centroid learning phase and the probability weights learning phase , based on the facts in ( 3,4 ) . W22 [ ∑ j∈ [ m ] v ∗ j δθ∗j , ρπ ] = Jπ ( { θ∗j } mj=1 ) , where Jπ ( { θj } mj=1 ) : =Ew∼π [ minj∈ [ m ] ||θj − θ̂w||2D ] . ( 3 ) Here ( 3 ) implies that , to find the optimal particle distribution in ( 2 ) , we can start with the centroid learning phase where we only need to optimize the centroids . It can be achieved by minimizing Jπ ( { θj } mj=1 ) , which is the averaged distance of bootstrap particles to their closest centroid . After we obtain the optimal centroids , the optimal probability weights can be learned by ( 4 ) : v∗j = ṽ ∗ j / ∑ s∈ [ m ] ṽ ∗ s , where ṽ ∗ j = Pw∼π ( j = arg minj∈ [ m ] ||θ∗j − θ̂w||2D ) . ( 4 ) Here v∗j is the proportion of bootstrap particles that are closest to the centroid j . We emphasize that the optimal solution to two-stage learning is guaranteed to be the global minimizer of the loss in ( 2 ) ( see Lemma 3.1 and 3.2 in Canas & Rosasco ( 2012 ) ) . However , the key issue is that the losses in both ( 2 , 3 ) can not be computed in practice , as they require us to access ρπ ( i.e. , obtain θ̂w first in order to calculate the loss ) . To handle this issue , we seek an easy-to-compute surrogate loss . Our idea is based on the following observation . Assuming the size of training data is large , which is usually the case in deep learning , we can expect that θw will be centered around a small region3 . It implies that we should search the centroid in this small region . Notice that when θ is close to θ̂w , based on Taylor approximation , we have Lw ( θ ) ≈ Lw ( θ̂w ) +∇ > θ Lw ( θ̂w ) ( θ − θ̂w ) + 1/2 ( θ − θ̂w ) > ∇2θLw ( θ̂w ) ( θ − θ̂w ) ≈ L∞ ( θ0 ) + 1/2||θ − θ̂w||2D , where ‖V ‖ 2 D : = V > ∇2θL∞ ( θ0 ) V. ( 5 ) Here L∞ ( θ ) : = Ex ` ( x , fθ ) denotes the population loss ; θ0 is the minimizer of L∞ ( θ ) . In ( 5 ) , we use the facts4 that ∇ > θ Lw ( θ̂w ) = 0 ; and with large training set , the empirical distribution ∑n i=1δxi/n well approximates the whole data population , and hence the bootstrap resampling distribution , i.e. , ∑n i=1wiδxi/n on the empirical distribution also well approximates the whole data population . This implies that Lw ( · ) ≈ L∞ ( · ) and ∇2θLw ( · ) ≈ ∇2θL∞ ( · ) . As the loss are close to each other , their minimizers are also close θ̂w ≈ θ0 . Since L∞ ( θ0 ) is some ( unknown ) constant independent with θ , we can replace the ||θj − θ̂w||2D in ( 3 ) by Lw ( θj ) as it only adds some constant into the loss . 3This can be formally characterized by central limit theorem as discussed in Section 4 . 4We defer the detailed analysis to Section 4 . Intuitively , we can expect that the centroid closest to θ̂w is the one that gives the smallest loss on Lw . It motivates us to learn the centroids via the modified centroid learning phase : { θ∗j } mj=1 = arg min θ1 , ... , θm∈Θ Ew∼π [ minj∈ [ m ] Lw ( θj ) ] . ( 6 ) Similarly , the optimal probability weights can be learned via the modified weight learning phase : v∗j = ṽ ∗ j / ∑ s∈ [ m ] ṽ ∗ s , where ṽ ∗ j = Pw∼π ( j ∈ arg minj∈ [ m ] Lw ( θ∗j ) ) . ( 7 ) We note that here we slightly abuse the notation of θ∗j and v ∗ j in ( 3,4 ) and ( 6,7 ) for simplification . In the later context , θ∗j and v ∗ j are used based on their definitions in ( 6,7 ) . Connection to K-means By viewing the target distribution as a set of particles that we want to cluster , in K-means clustering , each centroid ( i.e. , K-means center ) represents one of the K disjoint groups5 of particles , which is formed by assigning each particle in the whole set to the closest centroid among all the K centroids . K-means learns the optimal K centroids in the way that they can best approximate the whole set . The ‘ closeness ’ for assigning the particles is measured by the distance between the two points . As pointed out by Canas & Rosasco ( 2012 ) , K-means essentially searches the optimal particle distribution formed by the K centroids that minimizes its Wasserstein distance to the target distribution . Our centroid approximation idea follows the same fashion of clustering but our key innovation is to measure the ‘ closeness ’ by examining the bootstrap loss of the centroids so that we can still learn the optimal centroids without obtaining the i.i.d . bootstrap particles first . We also point out that , while we share the same objective as K-means , the optimization algorithms differ . The Expectation-Maximization type of algorithm used by K-means is not applicable to our scenario . Comparing with Other Particle Improving Approach Intuitively , from a high level abstracted perspective , we provide an approach to use K-means type of idea to improve the particle quality without accessing to the true target distribution . This is the key differentiator of this work to other approaches that improve the particle quality , as they all require to access the target distribution . For example Claici et al . ( 2018 ) requires that sampling from target distribution is cheap and easy . Chen et al . ( 2012 ; 2018a ) ; Campbell & Beronov ( 2019 ) need to access the logarithm of the probability density function of the target distribution . In our problem , neither sampling from the target distribution is cheap nor the logarithm of the probability density function is available , making those approaches not more applicable .

Summary: This paper aims to improve the quality of the parametric bootstrap when the number of particles is limited. It does so by constructing a small collection of "centroids" that well-approximate the ideal bootstrap distribution on parameter space in a certain Wasserstein distance. Practically, the centroids are constructed by gradient descent with respect to a loss function involving the negative log-likelihood. Theoretically, this paper shows that the surrogate loss converges to the Wasserstein distance under some assumptions. Empirically, this paper applies the centroid methodology to confidence interval construction, contextual bandits, the bootstrap deep q-network, and ensemble learning .