The role of tail dependence in estimating posterior expectations

Venue: NeurIPS_cc_2024_Workshop_BDU
Published: 2024-09-06 13:54:16

Abstract: Many tasks in modern probabilistic machine learning and statistics require estimating expectations over posterior distributions. While many algorithms have been developed to approximate these expectations, reliably assessing their performance in practice, in absence of ground truth, remains a significant challenge.
 In this work, we observe that the well-known k-hat diagnostic for importance sampling (IS) can be unreliable, as it fails to account for the fact that the common self-normalized IS (SNIS) estimator is a ratio. First, we demonstrate that examining separate k-hat statistics for the numerator and denominator can be insufficient. Then, we we propose a new statistic that accounts for the dependence between the estimators in the ratio. In particular, we find that the concept of tail dependence between numerator and denominator weights contains essential information for determining effective performance of the SNIS estimator.

Main paper: 
 introduction and background: 
 Algorithms for Bayesian computation continue to be used for increasingly complex probabilistic models, remaining an active research field [2] . Yet, in the absence of ground truth, it remains challenging in practice to determine how and in which sense an approximate inference algorithm has found a "good" solution, as studied by several recent works, for Markov Chain Monte Carlo (MCMC) [3] [4] [5] , variational inference (VI) [6] [7] [8] , and importance sampling [1, [9] [10] [11] (the latter two being closely connected). In this work, we focus on diagnostics that apply to IS and VI algorithms.Problem statement. Let θ P Θ (commonly, R d θ ) be the parameter of a Bayesian statistical model tppy|θqu θ for data y P Y with posterior PDF πpθ|Dq def " Z ´1 π ¨r πpθ|Dq " Z ´1 π ¨śn ppy n |θq ¨πpθq with D def " ty n u N n"1 , Z π the normalizer and prior PDF πpθq. Formally, we aim at constructing Monte Carlo estimates of a posterior expectation I P R ą0 , defined asEQUATIONwhere f : Θ Ñ R ě0 is a suitably integrable test function. In particular, we are interested in obtaining diagnostics to determine the quality of an estimator p I. As a concrete example, when we set f pθq " ppy pn`1q |θq for a test point y pn`1q , I is often written as ppy pn`1q |Dq, i.e., the evaluation of the posterior predictive PDF ppy|Dq at point y pn`1q . 1Self-normalized IS, combination with VI. Approximating integrals like in Eq. ( 1) accurately is challenging. MCMC is a natural solution, but there are notable cases where it is not appropriate. For1 Such integrals can be used for estimating the predictive performance of a posterior [12] or the influence of a particular observation.Submitted to Workshop on Bayesian Decision-making and Uncertainty, 38th Conference on Neural Information Processing Systems (BDU at NeurIPS 2024). Do not distribute. example, when even exact i.i.d. sampling from πpθ|Dq is inefficient, or when it is too expensive. In these cases one usually resorts to IS [13] , where we obtain samples from a chosen proposal PDF q, as θ psq i.i.d." qpθq, and construct estimators for I asp I SNIS " S ÿ s"1w psq f pθ psq q , w psq def " w psq ř S s 1 "1 w ps 1 q , w psq " wpθ psq q " r πpθ psq |Dq qpθ psq q .(2)Many theoretical properties of this estimator are known (see, e.g., [14] for a review). When the normalizing constant Z π is unknown (i.e., almost always), the normalization of the weights in Eq. ( 2)is not optional. In practice, it is difficult to find a good proposal, i.e., leading to estimates that are close to I in suitable ways. It is natural to use proposals that are the result of a VI algorithm [6] ,which is done implicitly or explicitly in the VI literature. See [6, [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] as examples for the many connections between VI and IS. A consequence of using a bad proposal is that the distribution of the weights w s tends to have a few very large values.Pareto-smoothed IS. Exploiting this observation, [1] proposed Pareto-smoothed IS (PSIS), which replaces the largest M unnormalized weights2 to get SNIS estimators with better behaviour. They fit a generalized Pareto distribution (GPD) to the weights tw psq u S s"1 , whose PDF we denote as ppwq. 3 The new weights introduce bias but reduce variance.. The GPD has three parameters, the most important of which is the shape parameter k. [1] propose to use an estimate of k, i.e., p k, as a diagnostic for IS.The p k diagnostic. [1] use the estimated value of k, i.e., p k, as a diagnostic for deciding whether the SNIS estimates with PSIS-corrected weights are reliable. The GPD has 1{k finite fractional moments when the true k ą 0, which suggests finite variance as soon as k ă 0.5. Note that this guarantees finite variance only for the normalizing constant estimator p Z π " 1{S ř S s"1 w psq , which is implicit in the denominator of SNIS [25] . [1] find empirically that when using S ą 2000, estimation with PSIS-corrected weights is reliable for p k ă 0.7, a threshold less stringent than 0.5.An advantage of p k is that it is not an IS estimate itself, unlike the effective sample size (ESS) [10] ,attempting to address the issues with variance-based diagnostics [9] . 
methodology: 
 Several works [25] [26] [27] have shown theoretically and empirically that accurately estimating posterior expectations such as I in Eq. ( 1) involves more than simply finding a proposal qpθq that is close to the posterior πpθ|Dq. This is because the SNIS estimator is a ratio estimator, as I itself is the ratio of two integrals,EQUATIONwhere we relabelled the normalizing constant I den . Therefore, we can write the SNIS estimator asEQUATIONwhere the two estimators p I num and p I den are unbiased, but p I SNIS is not. As elaborated in [25] , the asymptotic variance of the SNIS estimator is driven by the variance of the numerator estimator, the variance of the denominator, and the covariance between them. For convenience, we define two unnormalized importance weight functions, the one used in the numerator for p I num and the one used in p I den , asEQUATIONWe can then write the SNIS estimator as a ratio of two unbiased IS estimators,EQUATIONGiven that there are two IS weights, w num pθ psq q, w den pθ psq q in the above, it is natural to consider that one may track reliability p I SNIS by computing two diagnostics p k num , p k den separately for weights tw psq num u S s"1 and tw psq den u S s"1 . [1] explored this option empirically, reporting that in their experiments it was sufficient to take maxp p k num , p k den q to determine reliability of the ratio. In this work, we will argue that this heuristic misses useful information and propose a new diagnostic.The diagnostics p k num and p k den describe how well p I num and p I den respectively approximate I num and I den , serving as an (improved) substitute for estimates of variance (like the ESS). Yet, the variance of the SNIS estimator p I SNIS is not only affected by the variance of the numerator of Eq. ( 6), the variance of the denominator. It is also affected by the covariance between them, Cov q r p I num , p I den s [25] . Note that Cov q r p I num , p I den s is proportional to Cov q rw num , w den s, the random variables we will focus on from now.A straightforward idea to capture this missing piece of information from p k num and p k den is to construct an estimate of Cov q r p I num , p I den s, using the same samples from q used to estimate I. Yet, doing so would suffer the same drawbacks of variance-based diagnostics, which was a motivation for p k [1] .Thus, we will develop a diagnostic that is not a direct estimate of Cov q rw num , w den s. Like [1] , we also exploit the fact that the distribution of w num and w den can be well approximated with a powerlaw distribution in the tails. Specifically, we will look at a suitable notion of dependence between the tails of w num and w den . This notion will replace the covariance Cov q rw num , w den s as our target estimate. In fact, covariance, up to normalization, is equivalent to Pearson's correlation ρ, which is only a very specific form of dependence, with many known limitations [28] .Dependence and error cancellation. An intuition for why higher covariance between the estimators Cov q rw num , w den s, or other dependence metrics, can lead to lower error is that, in a ratio, error cancellation can happen. Error cancellation in ratios has been exploited to derive better convergence rates for other numerical integration methods [29] . In IS, it is known that large IS weights lead to high errors. Therefore, error cancellation in the ratio of Eq. ( 6) could happen when a large weight in the numerator is offset by another similarly large weight in the denominator. We now formalize this using the notion of tail dependence.Definition 1 (Upper tail dependence coefficient and tail dependence) Let W 1 , W 2 be two realvalued random variables. Let their (continuous) marginal CDFs be F 1 , F 2 . Then,EQUATIONprovided the limit exists, is known as upper tail dependence coefficient λ U P r0, 1s. If λ U ą 0, we say that W 1 , W 2 are asymptotically tail dependent, with the magnitude of λ U determining the strength of depedence.Next, we discuss how to relate the above concept to the estimation of I.We propose to diagnose whether the estimate in Eq. ( 6) is reliable by examining three quantities: p k num , p k den and a new diagnostic that is constructed as an approximation of the tail dependence coefficient λ U between w num , w den . Our aim is to study how these quantities relate to the effective performance of p I SNIS as an estimator of I, which we define as follows.Definition 2 (Effective performance) We define the effective performance of an estimator p I of I as ensuring that the value of p p I{Iq is close to 1 with high probability. This takes into account the possibility of I being very small, e.g., 10 ´7 following the reccomendation of [9] . In log-space, it is equivalent to look at how log I ´log p I is close to zero (recall I ą 0).In mathematical finance, various estimators of tail dependence have been developed [30] [31] [32] . We begin by studying semi-parametric estimators, following the assumption used by [1] and common in heavy-tailed distribution inference [33] . 3) over 50 replications.We compare SNIS, GenSNIS (see Section 3) with a common random number (CRN) and GenSNIS with independent marginals. From Fig. 1a , we see that GenSNIS with CRN performs best; this cannot be captured by p k values, but by the higher TDC.Specifically, we assume the distribution of w num , w den is well approximated by a GPD in the tails.Similarly, to estimate tail dependence, we assume the copula of their joint distribution is well approximated by an extreme value copula [34] , also only in the tails. 4 We hypothesize that tail dependence between w num and w den improves p I SNIS performance, similar to the effect of Cov q rw num , w den s, but easier to estimate and more reliable. To model this, we fit a Student-t copula, a simple parametric choice that also serves as an extreme value copula [31] . 
preliminary results on bayesian linear regression and conclusions: 
 We look at the distribution of log I ´log p I over different replications. We consider estimating the posterior predictive of a Bayesian linear regression (BLR) model where we can compute the exact value of I. That is, from Eq. ( 1), we set f pθq " ppy pn`1q |θq for a test point y pn`1q , and πpθ|Dq is a Gaussian with known mean and covariance (BLR posterior). 5To validate our hypothesis that tail dependence contains useful information, we check the behaviour of the diagnostics p k num , p k den our tail dependence diagnostic p λ U estimated from a Student-t copula Cpu 1 , u 2 ; ρ, νq, which is given by p λ U " 2t p ν`1 p´?p ν `1a 1 ´p ρ{ a 1 `p ρq where t is the Student-t CDF. 6 We find that, when k-diagnostics between competitors are similar for numerator and denominator, a higher tail dependence coefficient (TDC) explains the better performance. To explain our results, we need to introduce a recent generalization of the SNIS estimator proposed in [25] ,i.e., sampling from an extended space R d θ ˆRd θ , as p I GenSNIS "1 S ř S s"1 wnumpθ psq 1 q 1 S ř S s"1 wdenpθ psq 2 q , rθ psq 1 , θ psq 2 s i.i.d." q 1,2 pθ 1 , θ 2 q. SNIS is a special case where the joint is a degenerate joint with θ 1 " θ 2 . Another special case is taking q 1,2 pθ 1 , θ 2 q " q 1 pθ 1 qq 2 pθ 2 q, which is done in previous works including notably target-aware Bayesian inference [26] . Finally, for these experiments we consider the choice of q 1,2 pθ 1 , θ 2 q that uses a common random number (CRN) for numerator and denominator, but has different marginals. Concretely we used Gaussian proposals N pθ 1 ; µ 1 , Σ 1 q and N pθ 2 ; µ 2 , Σ 2 q for numerator and denominator, respectively. The parameters are set to the optimal ones (given by the BLR true posteriors for numerator and denominator) perturbed by an error term. The SNIS estimator uses only one distribution qpθq, so we take the midpoint between the two optimal IS means and covariances for its parameters. Fig. 1 shows the results. We indeed find in other settings (for d θ , noise variance, and covariate distributions) that when p k values are similar for numerator and denominator, tail dependence explains the remaining performance if a difference exists. We plan to test further TDC metrics and Bayesian models. 
 
 Appendix: See[1] for the choice of M .We use the common abuse of notation of using lowercase letters for both the random variable itself and its realised values.A copula of a bivariate joint distribution is the distribution on r0, 1s 2 after transforming the marginals to the uniform distribution. Many parametric copula families exist[35].See[36] for expressions about BLR including closed form posterior predictives.We use the estimate of p ρ from the Python statsmodels package, while setting ν manually. 



Reviews: review: title: title: Accep,
main_review: main_review: The demo results are somewhat underwhelming, and while the paper covers a lot of ground, the demo itself feels overly simplistic. There is also a lack of comprehensive tests to assess the control of tail dependence, and the mathematical proofs (e.g., provability of upper and lower bounds) are missing.

That said, the writing is solid, and the paper is well-formatted. I believe it offers a promising direction, and I hope the authors can expand on the demos and provide new insights for the community.
title: title: Improved SNIS diagnoistics by assuming tail dependence
main_review: main_review: The author proposed using tail dependence coefficents to improve the current SNIS diagnostics in estimating posterier expectation. The proposed TDC metrics highlighted important posterior diagnostic insights not availlable with the k-hat statistics. The paper generally flows well and is nicely presented overall, in particular section 1 is well-written with detailed mathematical background.

However, the reviewer believes, in its current form, the paper has still several places to work on:
* elaborate the concept of tail dependence in greater details in Definition 1 and Equation 7
* provide more reasoning in the assumptions from line 112-114, to what extend is this applicable in fields beyond mathmatical finance?
* clearly label the y-axis in Figure 1 b), c) and d), are we still in log space?
* one generally expects the experimental results and conclusion are written in separates sections, given there is only one experiment being done, the current merge of the two is understandable.
* still, the case study presented with BLR is perhaps too simple, as the reviewer believes in such case MCMC estimates of the posterior will not be too expensive to compute. It will be better if the author can think if additional case study to better demonstrate the advantages proposed by using the TDC metrics
title: title: An interesting paper and approach but some parts of it are not clear and the experiments are not thorough enough.
main_review: main_review: Pros:
- The paper introduces an interesting hypothesis - tracking tail dependence between estimators in the ratio of the SNIS estimator can capture the information lost in the covariance between them.

Cons:
- As the authors wrote, more experiments should be done to showcase the importance of tail dependence.
- Perhaps I didn't understand something, but I found some parts not clear enough. In particular,
  - The explanation for not estimating $Cov[\hat{I_{num}}, \hat{I_{den}}]$ in lines 78-80.
  - The explanations about the experimental part (lines 123-138) were too succinct. Perhaps the authors could elaborate more in the appendix (e.g., student-t copula, sampling from an extended space)
- Minor:
  - "..we we.." in line 9
  - $w_s$ instead of $w^{(s)}$ in line 39

Questions:
- How does the diagnostic behave with larger dimensionality? Does it have an effect on it?