Centroid Approximation for Bootstrap

Bootstrap is a principled and powerful frequentist statistical tool for uncertainty quantification . Unfortunately , standard bootstrap methods are computationally intensive due to the need of drawing a large i.i.d . bootstrap sample to approximate the ideal bootstrap distribution ; this largely hinders their application in large-scale machine learning , especially deep learning problems . In this work , we propose an efficient method to explicitly optimize a small set of high quality “ centroid ” points to better approximate the ideal bootstrap distribution . We achieve this by minimizing a simple objective function that is asymptotically equivalent to the Wasserstein distance to the ideal bootstrap distribution . This allows us to provide an accurate estimation of uncertainty with a small number of bootstrap centroids , outperforming the naive i.i.d . sampling approach . Empirically , we show that our method can boost the performance of bootstrap in a variety of applications . 1 INTRODUCTION . Bootstrap is a simple and principled frequentist uncertainty quantification tool and can be flexibly applied to obtain data uncertainty estimation with strong theoretical guarantees ( Hall et al. , 1988 ; Austern & Syrgkanis , 2020 ; Chatterjee et al. , 2005 ; Cheng et al. , 2010 ) . In particular , when combined with the maximum likelihood estimator or more general M-estimators , bootstrap provides a generalpurpose , plug-and-play non-parametric inference framework for general probabilistic models without case-by-case derivations ; this makes it a promising frequentist alternative to Bayesian inference . However , the standard bootstrap inference is highly expensive in both computation and memory as it typically requires drawing a large number1 of i.i.d . bootstrap particles ( samples ) to obtain an accurate uncertainty estimation . In the context of this paper , as each bootstrap particle/sample/centroid is a machine learning model , we might directly call a model as particle/sample/centroid . With a small number of particles , bootstrap may perform poorly . As a consequence , when applied to deep learning , we need to store a large number of neural networks and feed the input into a tremendous number of networks every time we make inference , which can be quite expensive and even unaffordable for deep learning problems with huge models2 . For example , in autonomous driving applications , our device can only store a limited number of models and we need to make decisions within a short time , which makes the standard bootstrap with a large number of models no more feasible . Typical ensemble methods in deep learning , such as Lakshminarayanan et al . ( 2017 ) ; Huang et al . ( 2017 ) ; Vyas et al . ( 2018 ) ; Maddox et al . ( 2019 ) ; Liu & Wang ( 2016 ) , can only afford to use a small number ( e.g. , less than 20 ) of models . Therefore , to make bootstrap more accessible in modern machine learning , it is essential to develop new approaches that break the key computation and memory barriers mentioned above . This paper aims to improve the bootstrap when the resource at inference is limited . We are motivated to consider the following problem : How to improve the accuracy of bootstrap inference when the number of particles is limited ? We attack this challenge by presenting an efficient centroid approximation for bootstrap . Our method replaces the i.i.d . bootstrap particles with a set of carefully optimized centroid particles that are 1For example , thousands of , as suggested by Statistics textbooks such as Wasserman ( 2013 ) . 2While training cost is an extra burden , it is small compared with the cost of making prediction as we only need to train the model once but make countless predictions at deployment . guaranteed to provide an accurate and compact approximation to the ideal bootstrap distribution so that only a smaller number of particles is needed to obtain good performance . Our method is based on minimizing a specially designed objective function that is asymptotically equivalent to the Wasserstein distance between the ideal bootstrap distribution and the particle distribution formed by the learned centroids . During the training , each centroid adjusts its location being aware of the locations of the others so that centroids are diversified and well distributed on the domain . Our method is similar to doing K-means on the ideal bootstrap distribution , finding K representative centroids that well represent K separate parts of the target distribution ’ s domain in an optimal way . As centroids are optimized to better approximate the distribution , our approach naturally improves over the vanilla bootstrap with i.i.d . particles . See Figure 1 for illustration . Empirically , we apply the centroid approximation method to various applications , including confidence interval estimation ( DiCiccio et al. , 1996 ) , bootstrap method for contextual bandit ( Riquelme et al. , 2018 ) , bootstrap deep Q-network ( Osband et al. , 2016 ) and bagging method ( Breiman , 1996 ) for neural networks . We find that our method consistently improves over the standard bootstrap . 2 BACKGROUND . Suppose we have a model fθ parameterized by θ in a parameter space Θ ⊆ Rd . Let { xi } ni=1 ⊂ X be a training set with n data points on X . Assume ` ( x , fθ ) is the negative log-likelihood of data point x with model fθ . A standard approach to estimate θ is maximum likelihood estimator ( MLE ) , which minimizes the negative log-likelihood function ( loss ) over the training set θ̂ = arg min θ∈Θ L ( θ ) , L ( θ ) = ∑n i=1 ` ( xi , fθ ) /n . Here the MLE θ̂ provides a point estimation without any information on the data uncertainty . Bootstrap is a simple and effective frequentist method to quantify the uncertainty . The bootstrap loss is a randomly perturbed loss defined as Lw ( θ ) = ∑n i=1wi ` ( xi , fθ ) /n , where w = [ w1 , ... , wn ] > is a set of random weights of data points drawn from some distribution π . A typical choice of π is the multinomial distribution with uniform probability , which corresponds to resampling on the training set with replacement . Given w , one can calculate its associated bootstrap particle by minimizing the bootstrap loss : θ̂w = arg min θ∈Θ Lw ( θ ) . ( 1 ) Let ρπ be the distribution of θ̂w when w ∼ π. Bootstrap theory indicates that we can quantify the data uncertainty of θ or any function g ( θ ) using ρπ . We call ρπ the ideal bootstrap distribution and it is the main object we want to approximate . Denote δθ as the delta measure centered at θ . Standard bootstrap method approximates ρπ by the particle distribution ρ̂π ( · ) = ∑m j=1δθ̂wj ( · ) /m formed by m i.i.d . particles { θ̂wj } mj=1 , which can be obtained by drawing m i.i.d . weights { wj } mj=1 from π and calculating each θ̂wj based on ( 1 ) . However , for deep learning applications , as discussed in the introduction , storing and making inference using a large number m of bootstrap particles can be quite expensive . On the other hand , if m is small , ρ̂π tends to be a poor approximation of ρπ . In this paper , we aim to improve the approximation of the particle distribution when m is small . 3 METHOD . Our idea is simple . Instead of using i.i.d . particles , in which the location of each particle is independent from that of the others , we try to actively optimize the location of each particle so that particles are diversified , better distributed and eventually providing a particle distribution with improved approximation accuracy . A natural way to achieve this goal is to explicitly optimize a set of points { θj } mj=1 ( called centroids ) jointly such that the Wasserstein distance between ρπ and the induced particle distribution is minimized : { θ∗j , v∗j } mj=1 = arg min θ1 , ... , θm∈Θ , [ v1 , ... , vm ] ∈Cm W2 [ ∑m j=1vjδθj , ρπ ] . ( 2 ) Here we consider a Wasserstein distanceW2 equipped with a special data-dependent distance metric || · ||D that will be introduced later in ( 5 ) . We note that here we also optimize the probability weights { vj } mj=1 of the centroids . Finding the optimal centroids and probability weights can be decomposed into two steps : the centroid learning phase and the probability weights learning phase , based on the facts in ( 3,4 ) . W22 [ ∑ j∈ [ m ] v ∗ j δθ∗j , ρπ ] = Jπ ( { θ∗j } mj=1 ) , where Jπ ( { θj } mj=1 ) : =Ew∼π [ minj∈ [ m ] ||θj − θ̂w||2D ] . ( 3 ) Here ( 3 ) implies that , to find the optimal particle distribution in ( 2 ) , we can start with the centroid learning phase where we only need to optimize the centroids . It can be achieved by minimizing Jπ ( { θj } mj=1 ) , which is the averaged distance of bootstrap particles to their closest centroid . After we obtain the optimal centroids , the optimal probability weights can be learned by ( 4 ) : v∗j = ṽ ∗ j / ∑ s∈ [ m ] ṽ ∗ s , where ṽ ∗ j = Pw∼π ( j = arg minj∈ [ m ] ||θ∗j − θ̂w||2D ) . ( 4 ) Here v∗j is the proportion of bootstrap particles that are closest to the centroid j . We emphasize that the optimal solution to two-stage learning is guaranteed to be the global minimizer of the loss in ( 2 ) ( see Lemma 3.1 and 3.2 in Canas & Rosasco ( 2012 ) ) . However , the key issue is that the losses in both ( 2 , 3 ) can not be computed in practice , as they require us to access ρπ ( i.e. , obtain θ̂w first in order to calculate the loss ) . To handle this issue , we seek an easy-to-compute surrogate loss . Our idea is based on the following observation . Assuming the size of training data is large , which is usually the case in deep learning , we can expect that θw will be centered around a small region3 . It implies that we should search the centroid in this small region . Notice that when θ is close to θ̂w , based on Taylor approximation , we have Lw ( θ ) ≈ Lw ( θ̂w ) +∇ > θ Lw ( θ̂w ) ( θ − θ̂w ) + 1/2 ( θ − θ̂w ) > ∇2θLw ( θ̂w ) ( θ − θ̂w ) ≈ L∞ ( θ0 ) + 1/2||θ − θ̂w||2D , where ‖V ‖ 2 D : = V > ∇2θL∞ ( θ0 ) V. ( 5 ) Here L∞ ( θ ) : = Ex ` ( x , fθ ) denotes the population loss ; θ0 is the minimizer of L∞ ( θ ) . In ( 5 ) , we use the facts4 that ∇ > θ Lw ( θ̂w ) = 0 ; and with large training set , the empirical distribution ∑n i=1δxi/n well approximates the whole data population , and hence the bootstrap resampling distribution , i.e. , ∑n i=1wiδxi/n on the empirical distribution also well approximates the whole data population . This implies that Lw ( · ) ≈ L∞ ( · ) and ∇2θLw ( · ) ≈ ∇2θL∞ ( · ) . As the loss are close to each other , their minimizers are also close θ̂w ≈ θ0 . Since L∞ ( θ0 ) is some ( unknown ) constant independent with θ , we can replace the ||θj − θ̂w||2D in ( 3 ) by Lw ( θj ) as it only adds some constant into the loss . 3This can be formally characterized by central limit theorem as discussed in Section 4 . 4We defer the detailed analysis to Section 4 . Intuitively , we can expect that the centroid closest to θ̂w is the one that gives the smallest loss on Lw . It motivates us to learn the centroids via the modified centroid learning phase : { θ∗j } mj=1 = arg min θ1 , ... , θm∈Θ Ew∼π [ minj∈ [ m ] Lw ( θj ) ] . ( 6 ) Similarly , the optimal probability weights can be learned via the modified weight learning phase : v∗j = ṽ ∗ j / ∑ s∈ [ m ] ṽ ∗ s , where ṽ ∗ j = Pw∼π ( j ∈ arg minj∈ [ m ] Lw ( θ∗j ) ) . ( 7 ) We note that here we slightly abuse the notation of θ∗j and v ∗ j in ( 3,4 ) and ( 6,7 ) for simplification . In the later context , θ∗j and v ∗ j are used based on their definitions in ( 6,7 ) . Connection to K-means By viewing the target distribution as a set of particles that we want to cluster , in K-means clustering , each centroid ( i.e. , K-means center ) represents one of the K disjoint groups5 of particles , which is formed by assigning each particle in the whole set to the closest centroid among all the K centroids . K-means learns the optimal K centroids in the way that they can best approximate the whole set . The ‘ closeness ’ for assigning the particles is measured by the distance between the two points . As pointed out by Canas & Rosasco ( 2012 ) , K-means essentially searches the optimal particle distribution formed by the K centroids that minimizes its Wasserstein distance to the target distribution . Our centroid approximation idea follows the same fashion of clustering but our key innovation is to measure the ‘ closeness ’ by examining the bootstrap loss of the centroids so that we can still learn the optimal centroids without obtaining the i.i.d . bootstrap particles first . We also point out that , while we share the same objective as K-means , the optimization algorithms differ . The Expectation-Maximization type of algorithm used by K-means is not applicable to our scenario . Comparing with Other Particle Improving Approach Intuitively , from a high level abstracted perspective , we provide an approach to use K-means type of idea to improve the particle quality without accessing to the true target distribution . This is the key differentiator of this work to other approaches that improve the particle quality , as they all require to access the target distribution . For example Claici et al . ( 2018 ) requires that sampling from target distribution is cheap and easy . Chen et al . ( 2012 ; 2018a ) ; Campbell & Beronov ( 2019 ) need to access the logarithm of the probability density function of the target distribution . In our problem , neither sampling from the target distribution is cheap nor the logarithm of the probability density function is available , making those approaches not more applicable .

Summary: This paper proposes an acceleration technique for bootstrap. Here a bootstrap particle corresponds to a machine learning model and for a large number of bootstrap particles, a huge computational cost is incurred. The authors propose using a k-means-like centroid approximation which optimizes a set of centroids (each of which corresponds to a machine learning model) minimizing a data-dependent Wasserstein distance.