# Comptes Rendus
Mathématique

Christopher S. Withers and Saralees Nadarajah

A note on bias reduction

Volume 358, issue 6 (2020), p. 641-644

Published online: 2 October 2020

https://doi.org/10.5802/crmath.49

<figure><img src="image_1.png" /></figure>

This article is licensed under the
CREATIVE COMMONS ATTRIBUTION 4.0 INTERNATIONAL LICENSE.
http://creativecommons.org/licenses/by/4.0/

<figure><img src="image_2.png" /></figure>
# A note on bias reduction

Christopher S. Withers<sup>a</sup> and Saralees Nadarajah<sup>*, b</sup>

**Abstract.** Let $\hat{w}$ be an unbiased estimate of an unknown $w \in R$. Given a function $t(w)$, we show how to choose a function $f_n(w)$ such that for $w^* = \hat{w} + f_n(w)$, $E t(w^*) = t(w)$. We illustrate this with $t(w) = w^a$ for a given constant $a$. For $a = 2$ and $\hat{w}$ normal, this leads to the convolution equation $c_r = c_r \otimes c_r$.

Manuscript received 10th January 2019, accepted 9th April 2020.

## 1. Introduction

Let $\hat{w}$ be an unbiased estimate of an unknown $w \in R$. Given a function $t(w)$, we show how to choose a function $f_n(w)$ such that $E t(w^*) = t(w)$ for $w^* = \hat{w} + f_n(w)$. We illustrate this in Section 2 with $\hat{w}$ normal and $t(w) = w^a$ for some constant $a$. For $a = 2$ this gives the convolution equation $c_r = c_r \otimes c_r$ to solve.

$w^*$ is not an estimate since it depends on the unknown $w$. The method extends to $\hat{w}$ any standard estimate of an unknown $w \in R^p$ with respect to a given parameter $n$. That is, $E \hat{w} \to w$ as $n \to \infty$ and, for $r \ge 1$, its $r$th order cumulants have magnitude $n^{1-r}$ and can be expanded as power series in $n^{-1}$:

$$ \kappa(\hat{w}_{i_1}, \dots, \hat{w}_{i_r}) = \sum_{e=r-1}^{\infty} n^{-e} k_e^{i_1, \dots, i_r} \quad (1) $$

for $1 \le i_1, \dots, i_r \le p$ and $k_0^{i_1, \dots, i_r} = w_{i_1}$, where $w_i$ is the $i$th component of $w$, and the cumulant coefficients $k_e^{i_1, \dots, i_r}$ are bounded as $n \to \infty$, but may depend on $w$. For $p = 1$, (1) can be written

$$ \kappa(\hat{w}) = \sum_{e=r-1}^{\infty} n^{-e} k_{r,e} $$

for $r \ge 1$, where $k_{1,0} = w$. Cumulant coefficients are the building blocks of analytic methods for statistical inference. For example, methods for constructing estimates of low bias for any smooth function $t(w): R^p \to R$ were given in Mynbaev et al. [3] and Withers and Nadarajah [4–14].

<footnote><sup>a</sup> Industrial Research Limited, Lower Hutt, New Zealand</footnote>

<footnote><sup>b</sup> University of Manchester, Manchester M13 9PL, UK</footnote>

<footnote>E-mails: kit.withers@gmail.com, mbbsssn2@manchester.ac.uk</footnote>

<footnote>* Corresponding author.</footnote>
Given a sequence $a_1, a_2, \dots$, the exponential partial Bell polynomial $B_{i,k}(\mathbf{a})$ is defined by

$$ \left( \sum_{j=1}^{\infty} a_j t^j / j! \right)^k / k! \equiv \sum_{j=k}^{\infty} B_{j,k}(\mathbf{a}) t^j / j! $$

for $t \in \mathbb{R}$ and $k = 0, 1, \dots$. It is tabled on p. 307–308 of Comtet [2] for $1 \le r \le 12$. Given two sequences $a_1, a_2, \dots$ and $b_1, b_2, \dots$, their discrete convolution is defined by

$$ a_r \otimes b_r = \sum_{i=1}^{r} a_i b_{r-i}. $$

Set $\delta_{i,1} = I(i=1)$ and $(a)_j = a!/(a-j)!$.

## 2. Adding bias to $\hat{w}$ to reduce the bias of $t(\hat{w})$

Suppose that $\hat{w} \sim \mathcal{N}(w, v(w)/n)$, a normal estimate. Set

$$ v = v(w), \quad f_n(w) = \sum_{i=1}^{\infty} k_i n^{-i}, \qquad (2) $$

where $k_i = b_i / i!$ may depend on $w$. Theorems 1 and 3 show how to choose $k_i$ or $b_i$ so that for any given $a$,

$$ E(w^*)^a = w^a, \quad w^* = \hat{w} + f_n(w). \qquad (3) $$

Theorem 2.1 considers the case $a=2$. Theorem 3 considers the case $a=3$. Throughout, we set

$$ \delta = \hat{w} - w, \quad v = v/w^2, \quad m_n = E w^* = w + f_n(w). \qquad (4) $$

**Theorem 1.** Take $w^* = \hat{w} + f_n(w)$ with

$$ k_r = -c_r v^r / D^{2r-1}, \\ c_1 = 1, \quad c_r = c_r \otimes c_r. \qquad (5) $$

where $D = 2w$ and, for $r \ge 2$,

$$ \text{Then } E(w^*)^2 = w^2. $$

**Proof.** For $w^*$ of (3),

$$ E(w^*)^2 = m_n^2 + v/n = w^2 + 2wf_n(w) + f_n(w)^2 + v/n = w^2 + \sum_{i=1}^{\infty} T_i n^{-i}, $$

where $T_i = Dk_i + s_i + v\delta_{i,1}$, $s_1 = 0$, and for $i \ge 2$,

$$ s_i = k_i \otimes k_i = \sum_{j=1}^{i-1} k_j k_{i-j}. $$

So, $T_i = 0$ if we take $k_i = -(s_i + v\delta_{i,1})/D$ for $i \ge 1$. This gives the result. $\square$

Corollary 2 gives an explicit form for $c_r$ in (5).

**Corollary 2.** For $r \ge 2$,

$$ c_r = - \left( \frac{1}{2} \right) \left( -4c_1 \right)^r / 2 = 2^{r-1} \frac{1}{3} \cdots \frac{(2r-3)}{r!}. $$

**Proof.** Set

$$ C(t) = \sum_{r=1}^{\infty} c_r t^r, $$

where $c_1$ is now arbitrary. By (5), $C(t) = c_1 t + C(t)^2$. Since $C(0) = 0$, this gives

$$ C(t) = [1 - (1 - 4c_1 t)^{1/2}] / 2 = - \sum_{r=1}^{\infty} \left( \frac{1}{r} \right) \left( -4c_1 t \right)^r / 2 $$
which implies the result.

**Theorem 3.** Take $w^* = \hat{w} + f_n(w)$ with

$$f_n(w) = w \sum_{j=0}^{\infty} A_j (c/n^3)^j / j! + w u \sum_{j=0}^{\infty} B_j (c/n^3)^j / j! - w,$$

where $c = 4v^3$, $a_j = (1/2)_j/2$, $u = -vn^{-1}$, $B_k = a_{k+1}/a_1(k+1)$ and

$$A_j = \sum_{k=0}^{j} (1/3)_k B_{j,k}(\mathbf{a}).$$

Then $E(w^*)^3 = w^3$.

**Proof.** For $w^*$ of (3),

$$E(w^*)^3 = m_n^3 + 3m_n v/n = w^3$$

if $m_n^3 + 3m_n v/n - w^3 = 0$. Set $\gamma = (v/n)^3 + w^6/4$. Since $\gamma > 0$, this cubic has one real root given by Equation (3.8.2) of Abramowitz and Stegun [1]:

$$m_n = S_1^{1/3} + S_2^{1/3},$$

where $S_j = w^3/2 \pm \gamma^{1/2}$. Suppose that $w > 0$. (If not, replace $w$ by $|w|$.) Then for $v$ of (4),

$$\gamma^{1/2} = w^3(1+d)^{1/2}/2,$$

where $d = cn^{-3}$. Furthermore,

$$(1+d)^{1/2} = \sum_{j=0}^{\infty} \binom{1/2}{j} d^j, \quad S_1 = 1+D$$

for

$$D = \sum_{j=1}^{\infty} \binom{1/2}{j} d^j/2 = \sum_{j=1}^{\infty} a_j d^j / j!$$

Furthermore,

$$D^k/k! = \sum_{j=k}^{\infty} B_{j,k}(\mathbf{a}) d^j / j!$$

implies

$$S_1^{1/3} = \sum_{k=0}^{\infty} \binom{1/3}{k} D^k = \sum_{j=0}^{\infty} A_j d^j / j!$$

Also

$$S_2 = 1/2 - (1+d)^{1/2}/2 = -\sum_{j=1}^{\infty} a_j d^j / j! = -a_1 d(1+U)$$

for

$$U = \sum_{k=1}^{\infty} B_k d^k / k!$$

and

$$U^j / j! = \sum_{k=j}^{\infty} B_{k,j}(\mathbf{B}) d^k / k!$$

Then

$$S_2^{1/3} = u(1+U)^{1/3} = u \sum_{j=0}^{\infty} \binom{1/3}{j} U^j = u \sum_{j=0}^{\infty} (1/3)_j U^j / j! = \sum_{k=0}^{\infty} C_k d^k / k!$$

where

$$C_k = \sum_{j=0}^{k} (1/3)_j B_{k,j}(\mathbf{B}).$$

Hence, for the choice of $f_n(w)$, $E(w^*)^3 = w^3$.
The method of Theorems 1 and 3 will not work for $t(w) = w^5$ since there is no explicit solution to a quintic. However, we now show how to obtain an unbiased or bias-reduced estimate of $w^a$ for any $a > 0$. Set $\Delta = \hat{w} - w = w^* - m_n$. Then

$$E(w^*)^a = E(m_n + \Delta)^a = \sum_{j=0}^{\infty} \binom{a}{j} m_n^{a-j} \mu_j(\hat{w}) = \sum_{j=0}^{\infty} \binom{a}{2j} m_n^{a-2j} \mu_{2j}(\hat{w}),$$

where

$$\mu_{2j}(\hat{w}) = N_j v^j, \quad N_0 = 1, \quad N_j = 1 \cdots (2j-1)$$

for $j \ge 1$. By (2),

$$f_n(w)^k/k! = \sum_{i=k}^{\infty} B_{i,k}(\mathbf{b}) n^{-i} / i!$$

So,

$$(m_n/w)^a = [1 + f_n(w)/w]^a = \sum_{k=0}^{\infty} (a)_k w^{-k} f_n(w)^k / k! = \sum_{i=0}^{\infty} D_{a,i} n^{-i} / i!$$

for

$$D_{a,i} = \sum_{k=0}^{i} (a)_k w^{-k} B_{i,k}(\mathbf{b}).$$

This implies

$$E(w^*)^a / w^a = \sum_{k=0}^{\infty} n^{-k} E_k$$

for

$$E_k = \sum_{i+j=k} \binom{a}{2j} N_j v^j D_{a-2j,i} / i!$$

So,

$$b_1 = -(a-1)w/2, \qquad E_1 = 0, \qquad E(w^*)^a = w^a + O(n^{-2}), \\ b_2 = w(a-1)[-(a-1)^2 + (a-1)_2 w v - (a-2)_2 v^2], \qquad E_2 = 0, \qquad E(w^*)^a = w^a + O(n^{-3}).$$

In this way, we can construct $f_n(w)$ so that for any given $a > 0$ and $k \ge 1$, $E[\hat{w} + f_n(w)]^a = w^a + O(n^{-k})$.

## References

[1] M. Abramowitz, I. A. Stegun (eds.), *Handbook of mathematical functions with formulas, graphs and mathematical tables*, Applied Mathematics Series, vol. 55, U.S. Department of Commerce, 1964.

[2] L. Comtet, *Advanced combinatorics: the art of finite and infinite expansions*, Springer, 1974.

[3] K. T. Mynbaev, S. Nadarajah, C. S. Withers, A. S. Aipenova, "Improving bias in kernel density estimation", *Stat. Probab. Lett.* **94** (2014), p. 106-112.

[4] C. S. Withers, S. Nadarajah, "Analytic bias reduction for k-sample functionals", *Sankhyā, Ser. A* **70** (2008), no. 2, p. 186-222.

[5] ——, "The bias and skewness of M-estimators in regression", *Electron. J. Stat.* **4** (2010), p. 1-14.

[6] ——, "Bias-reduced estimates for skewness, kurtosis, L-skewness and L-kurtosis", *J. Stat. Plann. Inference* **141** (2011), no. 12, p. 3839-3861.

[7] ——, "Bias reduction for the ratio of means", *J. Stat. Comput. Simulation* **81** (2011), no. 12, p. 1799-1816.

[8] ——, "Estimates of low bias for the multivariate normal", *Stat. Probab. Lett.* **81** (2011), no. 11, p. 1635-1647.

[9] ——, "Reduction of bias and skewness with applications to second order accuracy", *Stat. Methods Appl.* **20** (2011), no. 4, p. 439-450.

[10] ——, "Nonparametric estimates of low bias", *REVSTAT* **10** (2012), no. 2, p. 229-283.

[11] ——, "Calibration with low bias", *Stat. Pap.* **54** (2013), no. 2, p. 371-379.

[12] ——, "Delta and jackknife estimators with low bias for functions of binomial and multinomial parameters", *J. Multivariate Anal.* **118** (2013), p. 138-147.

[13] ——, "Density estimates of low bias", *Metrika* **76** (2013), no. 3, p. 357-379.

[14] ——, "Bias reduction when data are rounded", *Stat. Neerl.* **69** (2015), no. 3, p. 236-271.