# Modified Moment Method Estimator for the Shape Parameter of Generalized Gaussian Distribution for a Small Sample Size

Robert Krupiński

► To cite this version:

Robert Krupiński. Modified Moment Method Estimator for the Shape Parameter of Generalized Gaussian Distribution for a Small Sample Size. 12th International Conference on Information Systems and Industrial Management (CISIM), Sep 2013, Krakow, Poland. pp.420-429, 10.1007/978-3-642-40925-7_39. hal-01496087

HAL Id: hal-01496087

https://inria.hal.science/hal-01496087

Submitted on 27 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Modified moment method estimator for the
shape parameter of generalized Gaussian
distribution for a small sample size

Robert Krupiński¹

West-Pomeranian University of Technology in Szczecin,
Chair of Signal Processing and Multimedia Engineering,
ul. 26-Kwietnia 10, 71-126 Szczecin, Poland,
rkrupinski@wp.pl

**Abstract.** The moment method (MM) estimator for the shape parame-
ter of generalized Gaussian distribution (GGD) assume asymptotic case
when there is available infinite number of observations, but with the
smaller sample size the variance of the estimator increases and the mo-
ment method equation may not converge to a real solution for some
sample sets. The higher order moments can be expanded into series in
the moment method equation leading to a drop in the relative mean
square error (RMSE) and assuring a solution for a smaller sample size
comparing to the moment method without modification.

**Keywords:** estimation, generalized Gaussian distribution, moment method

# 1 Introduction

The moment method [1] is very popular method used for the estimation of the power shape parameter of GGD. The most popular approach is when the two moments are set to the values $m_1 = 1$ and $m_2 = 2$ (the Mallat's method), but the higher order moments are also used [2]. Most of the already known estimation methods for the shape parameter of GGD assume that the sample size is large, but in the real case, a set of limited size is only available. A very high value of variance for a very small sample size makes the estimation methods very inaccurate.

The computation of GGD parameters on small sample size is different from
on larger ones in a maximum likelihood (ML, [3]) framework as was shown in
[4]. The ML method has a lower variance for a small sample size than the MM
method [5], but it is complex and time consuming. The complexity of ML can
be reduced with the One Moment (OM) method [6]. The comparison of ML and
the Mallat's method for the estimation of GGD shape parameter in the range
0.3 – 3 is presented in [7], where the latter one is not accurate for the whole
range of the shape parameter.

The presented approach in this article will not overcome other estimation
methods for a small sample size, as it is fully based on the MM framework, but
it will enhance the convergence of the MM method for the higher order moments and a small sample size.

In Section 2 the higher order moments are expanded into series in the moment method equation for a small sample size. In Section 3 the numerical results for the integer and real higher order moments are presented.

## 2 Material and methods

The probability density function (PDF) of the continuous random variable of GGD [8] takes the form

$$f(x) = \frac{\lambda(p, \sigma) \cdot p}{2 \cdot \Gamma(\frac{1}{p})} e^{-[\lambda(p, \sigma) \cdot |x-\mu|]^p}, \quad (1)$$

where $\Gamma(z) = \int_0^\infty t^{z-1}e^{-t}dt, z > 0$, is the Gamma function [9], $p$ denotes the shape parameter, $\mu$ is the location parameter and $\lambda$ relates to the variance of the distribution and can be calculated from

$$\lambda(p, \sigma) = \frac{1}{\sigma} \left[ \frac{\Gamma(\frac{3}{p})}{\Gamma(\frac{1}{p})} \right]^{\frac{1}{2}}, \quad (2)$$

where $\sigma$ denotes the standard deviation. The special case of GGD can be observed when the shape parameter equals $p = 1$ and $p = 2$, which corresponds to Laplacian and Gaussian distributions respectively. For $p \to 0$, $f(x)$ approaches an impulse function, and for $p \to \infty$, $f(x)$ approaches a uniform distribution. For $\mu = 0$, the probability density function is centralized around zero.

From the definition of absolute moment $E[|X|^m] = \int_{-\infty}^\infty |x|^m \cdot f(x)dx$, for GGD the following is obtained [5]

$$E_m = \frac{\Gamma\left(\frac{m+1}{p}\right)}{\lambda^m \cdot \Gamma\left(\frac{1}{p}\right)}, \quad (3)$$

where $m$ can be a real number.

The $E_m$ estimated value of the moment can be acquired from the equation

$$\hat{E}_m = \frac{1}{N} \cdot \sum_{i=1}^{N} |x_i|^m \quad (4)$$

and where $N$ denotes the number of observed variables, and $\{x_1, x_2, \dots, x_N\}$ is the collection of $N$ i.i.d zero-mean random variables.

Eq. (3) for two different moment values $m_1$ and $m_2$ and eliminating $\lambda$ leads to:

$$\frac{\Gamma\left(\frac{m_2+1}{p}\right) \Gamma\left(\frac{1}{p}\right)^{\frac{m_2}{m_1}-1}}{\Gamma\left(\frac{m_1+1}{p}\right)^{\frac{m_2}{m_1}}} = \frac{E_{m_2}}{(E_{m_1})^{\frac{m_2}{m_1}}}, \quad (5)$$
which is the method for the estimation of the shape parameter based on two moments.

The moments (Eq. (4)) in the rhs of Eq. (5) can be expanded into series. Each component $|x_i|^m$ in the sum (Eq. (4)) can be represented in the form $(1 \pm y)^m$, which can be expanded into series for $m > 0$ and $|y| \le 1$. For $x_i \in < 0, 1 >$, it can be written (by Binomial Theorem)

$$ (x_i)^m = (1 + (x_i - 1))^m = \frac{1 + m \cdot (x_i - 1) + \frac{m \cdot (m-1)}{2!} \cdot (x_i - 1)^2 + \dots}{\frac{m \cdot (m-1) \cdot (m-2)}{3!} \cdot (x_i - 1)^3 + \dots} \quad (6) $$

For $x_i \in < -1, 0)$, it takes the form

$$ (-x_i)^m = (1 - (x_i + 1))^m = \frac{1 - m \cdot (x_i + 1) + \frac{m \cdot (m-1)}{2!} \cdot (x_i + 1)^2 - \dots}{\frac{m \cdot (m-1) \cdot (m-2)}{3!} \cdot (x_i + 1)^3 + \dots} \quad (7) $$

For $x_i > 1$, it can be written

$$ (x_i)^m = \|\|x_i\|\|^m \cdot \left(1 + \left(\frac{x_i}{\|x_i\|} - 1\right)\right)^m = \|\|x_i\|\|^m \cdot \left\{1 + m \cdot \left(\frac{x_i}{\|x_i\|} - 1\right) + \frac{m \cdot (m-1)}{2!} \cdot \left(\frac{x_i}{\|x_i\|} - 1\right)^2 + \dots\right. \\ \left. \frac{m \cdot (m-1) \cdot (m-2)}{3!} \cdot \left(\frac{x_i}{\|x_i\|} - 1\right)^3 + \dots \right\} \quad (8) $$

The operator $\|x\|$ rounds the elements of $x$ toward zero, resulting in an integer. For $x_i < -1$, it can be written

$$ (-x_i)^m = \|\|x_i\|\|^m \cdot \left(1 - \left(\frac{x_i}{\|x_i\|} + 1\right)\right)^m = \|\|x_i\|\|^m \cdot \left\{1 - m \cdot \left(\frac{x_i}{\|x_i\|} + 1\right) + \frac{m \cdot (m-1)}{2!} \cdot \left(\frac{x_i}{\|x_i\|} + 1\right)^2 - \dots\right. \\ \left. \frac{m \cdot (m-1) \cdot (m-2)}{3!} \cdot \left(\frac{x_i}{\|x_i\|} + 1\right)^3 + \dots \right\} \quad (9) $$

The equations (6)–(9) are used in the approximation of the moment (Eq. (4)) and used in Eq. (5) for the solution of $p$. Taking into account only a limited number of series components, the influence of outliers on the MM equation can be alleviated for a small sample size and the higher order moments and consequently reduce RMSE and improve the convergence.

### 3 Calculation

The equations from the article were validated with the GGD generator with fixed variance to unity and varying shape parameter in the range $p \in < 0.3, 6 >$.
The relative mean square error was applied for the comparison of the estimators output. RMSE was calculated from the equation

$$RMSE = \frac{1}{M} \sum_{i=1}^{M} \frac{(\hat{p} - p)^2}{p^2}, \qquad (10)$$

where $\hat{p}$ is a value estimated by the model and $p$ is a real value of a shape parameter. $M$ denotes the number of repetitions.

A small sample size may lead to difficulties with the root finding of the MM method (Eq. (5)), therefore, the stop condition was set to $tolX = 1e-4$ and $tolY = 1e-4$. When at least one solution is not available for the specified conditions, then the plots have the missing points for such a case.

Fig. 1 depicts RMSE for the MM method of the estimation of the shape parameter $p$ of GGD with a varying shape value $p$ and a selected fixed sample size $N = 10^4$. Each size, RMSE is not The RMSE value small. As an example, the real ones: $m_1$ RMSE perform

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Comparison of RMSE for the MM method of the estimation of the shape parameter $p$ of GGD with a varying shape value $p$ and a selected fixed sample size $N = 10^4$ (a relatively large sample size) for selected moments</figcaption></figure>
In the case of a small sample size, the increase of RMSE as well as the reduc-
tion of a con-
assure conve
shape paran
t there. The 1
also reduced
ation was re
calculations.

... ents
... lower
... tion
... hod
... ter-
... t of

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Comparison of RMSE for the MM method of the estimation of the shape parameter *p* of GGD with a varying shape value *p* and a selected fixed sample size *N* = 100 and *N* = 200 for selected moments</figcaption></figure>

By the application of Eq. (6)–(9) the influence of outliers on the MM equation
can be alleviated for a small sample size and the higher order moments and the
range of convergence can be extended toward the smaller sample sizes. The
expansion of |xᵢ|<sup>m</sup> in Eq. (4) up to and including a k-th component, i.e., (xᵢ/|xᵢ| ± 1)<sup>k</sup> will be denoted as Eₘ = f(k). Fig. 3 depicts that selecting only a few
components in the moments calculation (Eq. (6)–(9)) RMSE is stable and smaller
than in the usual calculation. The selected expansions Eₘ₁ = f(3), Eₘ₂ = f(6)
and Eₘ₁ = f(3), Eₘ₂ = f(8) are plotted for the integer moments m₁ = 4,
m₂ = 8 and for the shape parameter p = 6 and exhibit better RMSE performance
below N = 300.

The same behavior for the integer moments $m_1 = 4$ and $m_2 = 8$ and for the
shape parameter $p = 5$ is observed. For the selected expansions $E_{m1} = f(3)$,
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Comparison of RMSE for the MM method ($m_1 = 4, m_2 = 8$) and selected expansions of the estimation of the shape parameter $p$ of GGD with a varying sample size $N$ and for the selected fixed value $p = 6$ in the GGD generator</figcaption></figure>

$E_{m2} = f(6)$ and $E_{m1} = f(3)$, $E_{m2} = f(8)$, RMSE is stable and smaller than in the usual calculation below $N = 260$ (Fig. 4).

The selected expansions $E_{m1} = f(3)$, $E_{m2} = f(6)$ and $E_{m1} = f(3)$, $E_{m2} = f(8)$ and the MM method for the integer moments $m_1 = 4$ and $m_2 = 8$ for a small sample size $N = 169$ and for a wider range of $p$ are compared in Fig. 5. It can be noticed that the expansions $E_{m1} = f(3)$, $E_{m2} = f(6)$ and $E_{m1} = f(3)$, $E_{m2} = f(8)$ resulted in smaller RMSE and assured convergence for wider range of $p$.

In the case of a small sample size for the real higher order moments, the convergence of the MM equation can be also improved toward the observation smaller in size. Fig. 6 depicts a few components in the moments calculation (Eq. (6)-(9)) comparing to the whole moment calculation ($m_1 = 3.5, m_2 = 5.5$) over a range of varying sample size $N$ for the fixed value $p = 6$ in the GGD generator. It can be also noted that RMSE is longer stable for $E_{m1} = f(3)$ and $E_{m2} = f(6)$ toward smaller $N$ and has a smaller value too.

For the moments $m_1 = 3.5$ and $m_2 = 5.5$ and for the shape parameter $p = 5$, one of the expansions $E_{m1} = f(3)$ and $E_{m2} = f(6)$ has smaller RMSE and is stable for longer range of $N$ than the full moments, whereas another expansion $E_{m1} = f(3)$ and $E_{m2} = f(4)$ gains RMSE smaller and is still stable below $N = 250$ (Fig. 7).

The selected expansions $E_{m1} = f(3)$, $E_{m2} = f(4)$ and $E_{m1} = f(3)$, $E_{m2} = f(6)$ and the MM method for the real moments $m_1 = 3.5, m_2 = 5.5$ for a
<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Comparison of RMSE for the MM method ($m_1 = 4, m_2 = 8$) and selected expansions of the estimation of the shape parameter $p$ of GGD with a varying shape value $p$ and a selected fixed sample size $N = 169$</figcaption></figure>
<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Comparison of RMSE for the MM method ($m_1 = 3.5, m_2 = 5.5$) and selected expansions of the estimation of the shape parameter $p$ of GGD with a varying sample size $N$ and for the selected fixed value $p = 5$ in the GGD generator</figcaption></figure>
small samples
It can be no
convergence
and $E_{m2} =$
moments ca.

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Comparison of RMSE for the MM method ($m_1 = 3.5, m_2 = 5.5$) and selected expansions of the estimation of the shape parameter $p$ of GGD with a varying shape value $p$ and a selected fixed sample size $N = 300$</figcaption></figure>

The same approach can be applied for other higher order moments both
integer and real, but not all expansions lead to better performance.

# 4 Conclusions

The article focuses on the estimation of the shape parameter of GGD when a small size is only available. The experimental results and the comparison with already known MM method, which show that the method proposed is very useful from a practical point of view. The smaller sample size the variance of the MM estimator increases and the moment method equation may not converge to a real solution for some sample sets. For the higher order moments, the presented approach allows to lower RMSE and ensure a real solution for the smaller sample sets of GGD for the modified MM method.
References

1. Mallat, S. G.: A Theory of Multiresolution Signal Decomposition: The Wavelet Representation. IEEE Trans. Pattern Anal. Machine Intell. 11, 674–693 (1989)

2. Novey, M., Adali, T., Roy, A.: A complex generalized Gaussian distribution – characterization, generation, and estimation. IEEE Transactions on Signal Processing 58, 1427–1433 (2010)

3. Du, Y.: Ein sphärisch invariantes Verbunddichtemodell für Bildsignale. Archiv für Elektronik und Übertragungstechnik AEÜ-45, 148–159 (1991)

4. Meignen, S., Meignen, H.: On the modeling of small sample distributions with generalized Gaussian density in a maximum likelihood framework. IEEE Transactions on Image Processing 15, 1647–1652 (2006)

5. Varanasi, M. K., Aazhang, B.: Parametric generalized Gaussian density estimation. J. Acoust. Soc. Amer. 86, 1404–1415 (1989)

6. Krupiński, R., Purczyński, J.: Modeling the distribution of DCT coefficients for JPEG reconstruction. Image Commun. 22, 439–447 (2007)

7. Krupiński, R., Purczyński, J.: Approximated fast estimator for the shape parameter of generalized Gaussian distribution. Signal Process. 86, 205–211 (2006)

8. Box, G., Tiao, G. C.: Bayesian Inference in Statistical Analysis. Addison Wesley, Reading, MA (1973)

9. Olver, F. W. J.: Asymptotics and special functions. Academic Press, New York (1974)