SlowGuess/NeurIPS_2023_OCR huggingface dataset:
best-fit moment matched Gaussians, and numerical integral approximations based on the unscented transform have been explored [16, 14, 15]. This paper focuses on the Taylor approximation by Huber et al., an alternative Taylor approximation is proposed by Sebastiani [24], which assumes a shared covariance matrix among GMM components in the high variance setting. However, neither Huber et al. or Sebastiani provide theoretical analysis or convergence guarantees offered in our present work. An analysis conducted by Ru et al. [23] explores the efficiency of Huber et al.'s method and demonstrates that deterministic quadrature methods can be equally fast and accurate in a single dimension, however quadrature methods scale poorly with dimension, at  $\mathcal{O}(N^D)$  where  $N$  is the number of quadrature points per dimension and  $D$  is the dimension of the problem.
Variational approximations and bounds are also widely explored for estimating entropy and mutual information (MI). Much of this work is motivated by the use of Gibbs' inequality, which leads to bounds on entropy and MI [1]. Later work explored similar techniques for upper and lower bounds on MI [21, 10]. More recent work uses artificial neural networks (ANNs) as function approximators for a variety of information-theoretic measures based on differential entropy. The MI neural estimator (MINE) uses such an approach for representation learning via the information bottleneck [2] based on the Donsker-Varadhan (DV) lower bound on KL [9]. Related methods use ANNs for optimizing the convex conjugate representation of Nguyen et al. [20]. McAllester and Stratos [19] show that many of these distribution-free approaches based on ANN approximation rely on Monte Carlo approximations that have poor bias-variance characteristics which they provide their own Difference of Entropies (DoE) estimator that achieves the theoretical limit on estimator confidence.
We consider two experiments, a synthetic GMM section where we look at divergence of Huber et al. approximation (Eqn. (10)) and convergence of our three methods, our Taylor (Eqn. (12)), Taylor limit (Eqn. (19)), and our Legendre (Eqn. (16)). Furthermore, we give comparisons of accuracy and computation time across a variety of setting of approximation order, number of GMM components, and dimension for all methods. We then show our how our methods can be applied in practice to Nonparametric Variational Inference [11] where the convergence guarantees of the estimators has a noticeable accuracy improvement on their algorithm.
6.1 Synthetic Multivariate GMM
To highlight the theoretical properties, such as convergence, divergence, accuracy, and lower-bound of methods as discussed in Sec. 4, we will consider some synthetic GMMs. We create two GMMs similar to the example published in [15] (original experiment recreated in A.5). We consider a single and multi-dimensional dimensional case that satisfy the divergence criterion in Theorem 3.1. We also look at a time and accuracy analysis versus dimension, components, and polynomial order.
Figure 3: Dimension (left) of a two-component GMM varies from zero to fifty, for the second order of each method. Our methods show comparable accuracy and computation time to Huber, regardless of dimension. Number of components (middle) in a two-dimensional GMM is considered for the second order approximation of all methods. Huber et al. and our Legendre approximations are nearly equivalent in accuracy, while the Taylor series and Taylor limit serve as lower bounds. Computation time for all our methods is identical and comparable to Huber et al., deviating only at high numbers of components. Order (right) of each approximation is varied for a three-dimensional, two-component GMM. Huber et al. is plotted up to order three, as higher orders are restrictive due to tensor arithmetic and Taylor limit starts at order two as it requires three consecutive terms.
Scalar GMM In this experiment, we consider a scalar GMM as fourth order and above cannot be easily computed in higher dimensions for Huber et al. due to tensor arithmetic. We use a simple two-component GMM with parameters  $w_{1} = 0.35$ ,  $w_{2} = 0.65$ ,  $\mu_{1} = -2$ ,  $\mu_{2} = -1$ ,  $\sigma_{1}^{2} = 2$ , and  $\sigma_{2}^{2} \in (0,1]$ . We are changing the variance of the second Gaussian,  $\sigma_{2}^{2}$ , in the range (0,1] because the condition for divergence in Theorem 3.1 ( $p(x = \mu_1) < \frac{1}{2} p(x = \mu_2)$ ) is satisfied approximately when  $\sigma_2^2 < 0.46$  meaning this experiment should have regions of both convergence and divergence for Huber et al. approximation. Fig. 2 (left) shows the fourth order approximations of all methods. We see that the Huber et al. approximations diverges as expected in the range where  $\sigma_2^2 < .46$ . Our Taylor method remains convergent and accurate for all values while maintaining a lower bound. Again, our limit method gains us some accuracy and still manages to be lower bound. In this case, we see that the Legendre approximation is a near perfect fit for the entropy.
Multivariate GMM To demonstrate that divergence is not limited to single dimension or higher orders, we consider a five-component, two-dimensional GMM with the parameters  $w_{i} = .2 \forall i$ ,  $\mu_{1} = [0,0]^{T}$ ,  $\mu_{2} = [3,2]^{T}$ ,  $\mu_{3} = [1,-.5]^{T}$ ,  $\mu_{4} = [2.5,1.5]^{T}$ ,  $\mu_{5} = c[1,1]^{T}$  for  $c \in [-3,3]$ ,  $\Sigma_{1} = .25\mathbf{I}_{2}$ ,  $\Sigma_{2} = 3\mathbf{I}_{2}$ , and  $\Sigma_{3} = \Sigma_{4} = \Sigma_{5} = 2\mathbf{I}_{2}$  where  $\mathbf{I}_2$  is the two dimensional identity matrix. This examples shifts the mean of the fifth component to show that simply the location of components can make the Huber et al. approximation behave poorly. Fig. 2 (right) shows the third order approximation of each method. We see that Huber et al. is clearly not well behaved in this case even with low order approximation. Furthermore, we continue to see a lower bound by our Taylor method, an increased accuracy from out limit method, and that the Legendre approximation is very close to the true entropy.
6.1.1 Computation Time
In this experiment we empirically analyze the computation time of each method as a function of Gaussian dimension, number of Gaussian components, and the order of each polynomial approximation. The baseline of each method will be compared to the Monte Carlo estimation of entropy using  $L = 1000$  samples  $\{x_{j}\}_{j=1}^{L} \sim p$ . The Monte Carlo estimator is given by  $\hat{H} = \frac{1}{L} \sum_{j} (-\log p(x_{j}))$ .
Dimension In Fig. 3 (left), we evaluate the accuracy and computation time for 30 two-component GMMs per dimension in the range of [1, 50]. Comparing second order approximations of all methods against the Monte Carlo estimator, our polynomial approximations demonstrate similar accuracy and
nearly identical computation time. The results are comparable to Huber, indicating that our methods preserve accuracy and computation efficiency while providing convergence guarantees.
GMM Components In Fig. 3 (middle), accuracy and computation time are presented for 30 two-dimensional GMMs with varying numbers of components (from 1 to 20) using second order approximations. Legendre and Huber methods show slightly higher accuracy compared to our Taylor approximation and Taylor limit. Notice, Huber's standard deviation also increases with more components, due to the increased likelihood of satisfying the divergence condition in Theorem 3.1. Computation time remains similar for all methods, but is more prohibitive for higher components.
Polynomial Order Fig. 3 (right) shows as the order of the polynomial approximation increases for two-component GMMs in three dimensions. Legendre and Huber methods show higher accuracy compared to Taylor approximation and Taylor limit. Huber is limited to order 3 due to Tensor arithmetic, while Taylor limit starts at order 2 as it requires multiple orders. Computation times are similar across all methods. Notice no accuracy is gained from zero to first order and from second to third order in Huber's approximation due to relying on odd moments of Gaussians which are zero.
6.2 Nonparametric Variational Inference
Consider a target density  $p(x,\mathcal{D})$  with latent variables  $x$  and observations  $\mathcal{D}$ . The NPV approach [11] optimizes the evidence lower bound (ELBO),  $\log p(x,\mathcal{D})\geq \max_qH_q(p(x,\mathcal{D})) - H_q(q(x))\equiv \mathcal{L}(q)$  w.r.t. an  $m$ -component GMM variational distribution  $q(x) = \frac{1}{N}\sum_{i = 1}^{m}\mathcal{N}(x|\mu_i,\sigma_i^2 I_d)$ . The GMM entropy lacks a closed-form so NPV applies Jensen's lower bound as an approximation,  $\hat{H}_q^J (q(x))$ . The cross entropy also lacks a closed-form, so NPV approximates this term using the analogous Huber et al. Taylor approximation. Specifically, NPV expands the log density around the means of each GMM component as,
$$
H _ {q} (p (x)) \approx - \sum_ {i = 1} ^ {M} w _ {i} \sum_ {n = 0} ^ {N} \frac {\nabla^ {2} \log \left(p \left(\mu_ {i}\right)\right)}{n !} \mathbb {E} _ {q _ {i}} [ (x - \mu_ {i}) ^ {n} ] = \hat {H} _ {N, q} ^ {H} (p (x)) \tag {20}
$$
However, Eqn. (20) is subject to the divergence criterion of Theorem 3.1 if  $2p(\mu_i)\leq \max (p(x))$ . By replacing the entropy terms with our convergent series approximations we observe significant improvements in accuracy.
In our approach, we will highlight and address two problems with the NPV algorithm; the potential divergence of  $\hat{H}_{N,q}^{H}(p(x))$  and the poor estimation of the GMM entropy via  $\hat{H}_q^J (q(x))$ . To address the potential divergence of  $\hat{H}_{N,q}^{H}(p(x))$ , we will take motivation from the results found in [23] and use a 2 point Gauss-Hermite quadrature method to approximate  $H_{q}(p(x))$ . This method will be a limiting factor in scaling the NPV algorithm in dimension, however it guarantees that the cross-entropy approximation will not diverge. This alteration leads to a solution for the inconsistency of the ELBO approximations. Then, Jensen's inequality is a very poor approximation for entropy in general, instead we will use the three methods we have introduced, our Taylor, Taylor limit, and our Legendre, as the GMM entropy approximations for higher accuracy. Fig. 4 shows an approximation of a two dimensional, three component mixture Student T distribution using a five component GMM in the traditional NPV, our modified NPV algorithm with our Taylor and Legendre approximation.
The results, as seen in Fig. 5, highlight the accuracy of each method versus the number of components, the order of our polynomial approximation, and the dimension of the GMM. In each experiment, we are approximating a multivariate mixture T distribution,  $p(x)$ . We randomize the parameters of  $p(x)$  and the initialization parameters of the variational GMM,  $q(x)$ , for optimization. The KL is approximated using a 100000 Monte Carlo approximation after convergence of each algorithm. We see that in all cases of components, order, and dimension, our method achieves significant accuracy improvements. We see that we can use low order approximations to receive substantial approximation improvement (Fig. 5 (middle)). We see all methods gain accuracy as number of components increase (Fig. 5 (left)) however our methods see most of the accuracy improvements with only a few components, whereas NPV has substantially worse approximations with low components. Finally, we see we maintain a lower variance and KL than NPV with all our methods as the dimension grows (Fig. 5 (right)). For further discussion of the experiment, see A.6.
Our Taylor and Legendre methods ensure convergence and deliver comparable accuracy to that of Huber et al. However, the computational complexity of these methods grows with the number of
Figure 4: A three component mixture Student T distribution PDF (far-left) is approximated by a five component GMM using traditional NPV (left), our algorithm using a  $6^{th}$  order Taylor polynomial (right), and Legendre polynomial (far-right). We see that NPV both has issues with finding correct placement of means and sets the variances of the GMM components to be too narrow. Our methods do a better job of assigning means and the Legendre method seems to set the variances slightly better than our Taylor.
Figure 5: The above figures show the accuracy of each method across varying components, orders, and dimensions in approximating a multivariate mixture T distribution with a GMM. Our method consistently improves accuracy significantly. Low order of the convergent estimators provide substantial approximation improvement (middle). Most accuracy improvements are achieved with a small number of components, unlike NPV (left) which continues to need higher number of components to see good accuracy return. The guaranteed convergence of the approximation in higher dimensions seems to have a drastic improvement on accuracy().
components,  $M$ , in the GMM, following an  $\mathcal{O}((n + M - 1)!)$  time complexity, where  $n$  represents the order of the approximating polynomial. This summation comprises only scalar terms and is amenable to parallelization but may become prohibitively expensive for large  $M$  and  $n$  values. Furthermore, our polynomial methods are specifically tailored to the entropy,  $H_{p}(p(x))$ , and cross-entropy,  $H_{p}(q(x))$ , where both  $p(x)$  and  $q(x)$  are GMMs. In contrast, Huber et al.'s approximation can be more readily extended to cross-entropy scenarios where  $q(x)$  pertains to any distribution amenable to the construction of a second-order Taylor polynomial. This limitation hinders the broader applicability of our approximation.
We have provided novel theoretical analysis of the convergence for the widely used Huber et al. Taylor approximation of GMM entropy and established that the series diverges under conditions on the component means. We address this divergence by introducing multiple novel methods which provably converge. We wish to emphasize that the Huber et al. approximation tends to yield accurate results when it is convergent and the intention of this work is not to dissuade the use of this approximator. Quite the contrary, this work encourages the use of either Huber et al. or our own estimator by providing a solid theoretical foundation for both methods. We acknowledge that there are contexts in which one method may be preferred over the other, for example when bounds are preferred, or when convergence criteria are provably satisfied.
There are several areas that require further investigation. For example, one limitation of both methods is that they scale poorly with polynomial order and number of components. In fact, Huber et al. cannot easily be calculated for fourth order and above, due to tensor arithmetic. Our approximation works well in practice, but is limited solely to GMM densities. Further work is necessary to efficiently apply our convergent series to situations of cross-entropy's that contain non-GMM distributions.
[18] V. Maz'ya and G. Schmidt. On approximate approximations using gaussian kernels. IMA Journal of Numerical Analysis, 16(1):13-29, 1996.
[19] D. McAllester and K. Stratos. Formal limitations on the measurement of mutual information. In International Conference on Artificial Intelligence and Statistics, pages 875–884. PMLR, 2020.
[20] X. Nguyen, M. J. Wainwright, and M. I. Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56 (11):5847-5861, 2010.
[21] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pages 5171-5180. PMLR, 2019.
[22] L. F. Richardson and R. T. Glazebrook. Ix. the approximate arithmetical solution by finite differences of physical problems involving differential equations, with an application to the stresses in a masonry dam. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 210(459-470):307-357, 1911. doi: 10.1098/rsta.1911.0009.
[23] B. Ru, M. McLeod, D. Granziol, and M. A. Osborne. Fast information-theoretic bayesian optimisation, 2018.
[24] P. Sebastiani and H. P. Wynn. Maximum entropy sampling and optimal bayesian experimental design. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 62(1):145-157, 2000. doi: https://doi.org/10.1111/1467-9868.00225. URL https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/1467-9868.00225.
[25] B. Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.
[26] M. C. Shewry and H. P. Wynn. Maximum entropy sampling. Journal of applied statistics, 14 (2):165-170, 1987.
[27] N. Tishby, F. C. Pereira, and W. Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
[28] D. Zwillinger. Crc standard mathematical tables and formulae; 30th edition. Journal of the American Statistical Association, 91:1755, 1995. URL https://api_semanticscholar.org/CorpusID:62740696.
A.1 Section 3 Proofs
Theorem 3.1 (Divergence Criterion for Huber et al.). Let  $p(x) = \sum_{i=1}^{M} w_i \mathcal{N}(x \mid \mu_i, \Sigma_i)$  be a GMM and consider the Taylor series presented by Huber et al. in Eqn. (5). If any mean component,  $\mu_i$ , satisfies the condition  $p(\mu_i) < \frac{1}{2} \max(p(x))$ , then Huber et al.'s approximation diverges, otherwise it converges.
Proof. Let  $f(y) = \log(y)$ ,  $g(x) = p(x)$ , and  $h(x) = f(g(x)) = \log(p(x))$ . Huber et al. creates the Taylor series in Eqn. (5) with  $N^{th}$  order approximation
$$
\sum_ {i = 1} ^ {M} w _ {i} T _ {h, N, \mu_ {i}} (x) = \sum_ {i = 1} ^ {M} w _ {i} \sum_ {n = 0} ^ {N} \frac {h ^ {(n)} (\mu_ {i})}{n !} (x - \mu_ {i}) ^ {n}, \tag {21}
$$
Let us consider just a single one of the Taylor series in Eqn. (21)
$$
T _ {h, N, \mu_ {i}} (x) = \sum_ {n = 0} ^ {N} \frac {h ^ {(n)} (\mu_ {i})}{n !} (x - \mu_ {i}) ^ {n}, \tag {22}
$$
By Theorem 3.4 in Lang<sup>1</sup>, the Taylor series of a composition of function is equivalent to the composition of each components Taylor series, i.e.,  $T_{h,N,\mu_i}(x) = T_{f,N,g(\mu_i)} \circ T_{g,N,\mu_i}(x)$  where  $\circ$  is the composition operation. Since  $p(x)$  is a GMM, is the sum of entire functions, and thus itself is entire, meaning it's Taylor series,  $T_{g,N,\mu_i}(x)$ , converges everywhere. We turn our attention to the Taylor approximation of  $\log T_{f,N,g(\mu_i)}$ ,
$$
T _ {f, N, p \left(\mu_ {i}\right)} (x) = \log \left(p \left(\mu_ {i}\right)\right) + \sum_ {n = 1} ^ {N} \frac {(- 1) ^ {n - 1}}{n p \left(\mu_ {i}\right) ^ {n}} \left(y - p \left(\mu_ {i}\right)\right) ^ {n}, \tag {23}
$$
We can look at the  $n^{th}$  term in the series,  $b_{n} = \frac{(-1)^{n - 1}}{np(\mu_{i})^{n}} (y - p(\mu_{i}))^{n}$ , and use the ratio test to define convergence.
$$
\begin{array}{l} \lim  _ {n \rightarrow \infty} \left| \frac {b _ {n + 1}}{b _ {n}} \right| = \lim  _ {n \rightarrow \infty} \left| \frac {(- 1) ^ {n}}{(n + 1) p (\mu_ {i}) ^ {n + 1}} (y - p (\mu_ {i})) ^ {n + 1} \frac {n p (\mu_ {i}) ^ {n}}{(- 1) ^ {n - 1}} (y - p (\mu_ {i})) ^ {- n} \right| \\ = \lim _ {n \rightarrow \infty} \left| \frac {(- 1) ^ {n}}{(- 1) ^ {n - 1}} \frac {n p (\mu_ {i}) ^ {n}}{(n + 1) p (\mu_ {i}) ^ {n + 1}} \frac {(y - p (\mu_ {i})) ^ {n + 1}}{(y - p (\mu_ {i})) ^ {n}} \right| \\ = \lim  _ {n \rightarrow \infty} \left| \frac {n}{(n + 1) p (\mu_ {i})} (y - p (\mu_ {i})) \right| \\ = \lim  _ {n \rightarrow \infty} \frac {n}{(n + 1)} \left| \frac {y - p \left(\mu_ {i}\right)}{p \left(\mu_ {i}\right)} \right| = \left| \frac {y - p \left(\mu_ {i}\right)}{p \left(\mu_ {i}\right)} \right| = L \\ \end{array}
$$
The ratio test states that the series converges if the limit,  $L$ , is strictly less than 1. However, setting  $L = 1$  and some simple manipulation, we find
$$
\left| \frac {y - p \left(\mu_ {i}\right)}{p \left(\mu_ {i}\right)} \right| <   1 \quad \Rightarrow \quad | y - p \left(\mu_ {i}\right) | <   p \left(\mu_ {i}\right) \quad \Rightarrow \quad y <   2 p \left(\mu_ {i}\right) \tag {24}
$$
This only converges if all  $y < 2(p(\mu_i)$ . Consider the maximum,  $\max(p(x))$ , if it satisfies this condition, so will every other point, if it doesn't satisfy this point, then the series is divergent by the ratio test. So we have the convergent criterion  $\max(p(x)) < 2p(\mu_i)$ , or written in terms of divergence criterion, we diverge if  $p(\mu_i) < \frac{1}{2} \max(p(x))$ .
$$
\log (p (x)) = \log (a) + \sum_ {n = 1} ^ {\infty} \frac {(- 1) ^ {n - 1}}{n a ^ {n}} \sum_ {k = 0} ^ {n} \binom {n} {k} (- a) ^ {n - k} p (x) ^ {k} \tag {25}
$$
$^{1}$ Lang, Serge. Complex Analysis. 4th ed. Springer, 2013. ISBN 978-1-4757-3083-8.
Proof. Consider the  $n^{th}$  term in the sum
$$
b _ {n} = \frac {(- 1) ^ {n - 1}}{n a ^ {n}} (p (x) - a) ^ {n}
$$
The ratio test says if the limit of the absolute value of successive terms converges to a value strictly less than one, then the series converges
$$
\begin{array}{l} \lim  _ {n \rightarrow \infty} \left| \frac {b _ {n + 1}}{b _ {n}} \right| = \lim  _ {n \rightarrow \infty} \left| \frac {(- 1) ^ {n}}{(n + 1) a ^ {n + 1}} (p (x) - a) ^ {n + 1} \frac {n a ^ {n}}{(- 1) ^ {n - 1}} (p (x) - a) ^ {- n} \right| \\ = \lim  _ {n \rightarrow \infty} \left| \frac {(- 1) ^ {n}}{(- 1) ^ {n - 1}} \frac {n a ^ {n}}{(n + 1) a ^ {n + 1}} \frac {(p (x) - a) ^ {n + 1}}{(p (x) - a) ^ {n}} \right| \\ = \lim  _ {n \rightarrow \infty} \left| \frac {n}{(n + 1) a} (p (x) - a) \right| \\ = \lim  _ {n \rightarrow \infty} \frac {n}{(n + 1)} \left| \frac {p (x) - a}{a} \right| = \left| \frac {p (x) - a}{a} \right| = L \\ \end{array}
$$
We see that  $L < 1 \forall x$  iff  $a > \frac{1}{2} \max(p(x))$  in which case the series converges everywhere.
Lemma 3.3 (Convergent Legendre Series of Log). If  $a > \max(p(x))$ , and consider the  $n^{th}$  shifted Legendre polynomial on the interval  $[0, a]$  in Eqn. (8). Then for  $x$  a.e.
$$
\log (p (x)) = \sum_ {n = 0} ^ {\infty} (2 n + 1) \sum_ {j = 0} ^ {n} \frac {(- 1) ^ {n + j} (n + j) ! ((j + 1) \log (a) - 1)}{(n - j) ! ((j + 1) !) ^ {2}} L _ {[ 0, a ], n} (p (x)) \tag {26}
$$
Proof. Orthogonal polynomials can approximate any function a.e. that is continuous and square-integrable (see Trefethen and  $\mathrm{Bau}^2$  and Gustafson). In out case,  $L_{[0,a],n}(y)$  live on  $L_{2}([0,a])$  (referring to the second Lebesgue space on the interval  $[0,a]$ ). This means all we have to show is that  $\log(y)$  lives in this domain which means  $\| \log(y) \|_2^2 < \infty$
$$
\left\| \log (y) \right\| _ {2} ^ {2} = \int_ {0} ^ {a} \log (y) ^ {2} d y = a \left(\left(\log (a) - 2\right) \log (a) + 2\right) <   \infty \tag {27}
$$
So we see that  $\log(y) \in L_2([0, a])$  and therefore it's Legendre series is convergent.
For completeness, we now derive the Legendre series for  $\log (y)$ . We will appeal to Eqn. (8),  $L_{[0,a],n} = \sum_{k = 0}^{n}\frac{(-1)^{n + k}(n + k)!}{(n - k)!(k!)^2a^k} y^k$ , and  $\langle L_{[0,a],n}(y),L_{[0,a],n}(y)\rangle = \frac{a}{2n + 1}$  as found in [4]
$$
\begin{array}{l} \log (p (x)) = \sum_ {n = 0} ^ {\infty} \frac {\left\langle \log (y) , L _ {[ 0 , a ] , n} (y) \right\rangle}{\left\langle L _ {[ 0 , a ] , n} (y) , L _ {[ 0 , a ] , n} (y) \right\rangle} L _ {[ 0, a ], n} (p (x)) \\ = \sum_ {n = 0} ^ {\infty} \frac {2 n + 1}{a} \int_ {0} ^ {a} \log (y) \sum_ {k = 0} ^ {n} \frac {(- 1) ^ {n + k} (n + k) !}{(n - k) ! (k !) ^ {2} a ^ {k}} y ^ {k} d y L _ {[ 0, a ], n} (p (x)) \\ = \sum_ {n = 0} ^ {\infty} \frac {2 n + 1}{a} \sum_ {k = 0} ^ {n} \frac {(- 1) ^ {n + k} (n + k) !}{(n - k) ! (k !) ^ {2} a ^ {k}} \int_ {0} ^ {a} \log (y) y ^ {k} d y L _ {[ 0, a ], n} (p (x)) \\ = \sum_ {n = 0} ^ {\infty} \frac {2 n + 1}{a} \sum_ {k = 0} ^ {n} \frac {(- 1) ^ {n + k} (n + k) !}{(n - k) ! (k !) ^ {2} a ^ {k}} \frac {a ^ {k + 1} ((k + 1) \log (a) - 1)}{(k + 1) ^ {2}} L _ {[ 0, a ], n} (p (x)) \\ = \sum_ {n = 0} ^ {\infty} 2 n + 1 \sum_ {k = 0} ^ {n} \frac {(- 1) ^ {n + k} (n + k) ! ((k + 1) \log (a) - 1)}{(n - k) ! ((k + 1) !) ^ {2}} L _ {[ 0, a ], n} (p (x)) \\ \end{array}
$$
which we know is convergent from the above discussion
$^{2}$ Trefethen, Lloyd N., and David Bau III. Numerical Linear Algebra. SIAM, 1997.