# A Weighted Estimation for Risk Model

Authors: Mei Ling Huang; Ke Zhao
Journal: ISRN Probability and Statistics
 (2013)

---

## Abstract

We propose a weighted estimation method for risk models. Two examples of natural disasters
are studied: hurricane loss in the USA and forest fire loss in Canada. Risk data is often fitted by a
heavy-tailed distribution, for example, a Pareto distribution, which has many applications in economics,
actuarial science, survival analysis, networks, and other stochastic models. There is a difficulty
in the inference of the Pareto distribution which has infinite moments in the heavy-tailed case.
Firstly this paper  applies the truncated Pareto distribution to overcome this difficulty. Secondly,
we propose a weighted semiparametric method to estimate the truncated Pareto distribution.
The idea of the new method is to place less weight on the extreme data values. This paper
gives an exact efficiency function,L1-optimal weights and L2-optimal weights of the new estimator. Monte Carlo simulation results confirm the theoretical conclusions. The two
above mentioned examples are analyzed by using the proposed method. This paper shows that
the new estimation method is more efficient by mean square error relative to several existing
methods and fits risk data well.


---

## Body

## 1. Introduction

### 1.1. Two Motivating Examples

In the recent years, many extreme events have occurred in financial markets, natural disasters, disease control, and industrial quality control. Natural disasters, for example, earthquakes, hurricanes, forest fires, volcanoes, and floods affect human life. It is important to predict and prepare for the next disaster occurrence and to estimate losses to inhabitants, insurance companies, and governments. In this section, we study two examples.


#### 1.1.1. A Hurricane Loss Example

Strong winds, heavy rainfall, and storm surges caused by hurricanes cause death and destroy properties. They generate great losses to insurance companies as well. Figure1 shows the 49 costliest Atlantic hurricane losses for the United States during 1900–2005 [1]. The measurement of this hurricane loss data is in US dollars; all dollars have been adjusted by using the inflation rates from 1900 to 2005.Figure 1
The 49 USA atlantic hurricane loss, 1900–2005.From the data in [1], we note that the most costly hurricane is the 1926 Great Miami Hurricane with cost of damage of 157 billion which is 1.58 times larger than the second worst hurricane, the 1900 Galveston Hurricane. After 1926, on August 28, 2005 Hurricane Katrina caused damage of 81 billion. This is approximately 1.19 times larger than Hurricane Andrew in August 1992, which caused damage of 56 billion. On the other hand, we note that 80% of the hurricane losses are less than 21 billion. Many smaller hurricanes are not listed. These considerations raise a number of questions.(a)
How do we predict the loss of the next hurricane? Will it again be 1.58 times larger than the worst one so far?(b)
Do we keep the traditional approach which is to prepare for the worst event? Should we focus on the extreme events?(c)
What is the value at risk with 5% probability?(d)
How do we set an upper limit of the losses?(e)
How do we estimate the distribution of largest losses?The objective of this study is to find the best model to fit the empirical data and to answer the above questions as accurately as possible. Since the data has large losses, we consider that the data should fit a heavy-tailed distribution. Studies on Pareto-type heavy-tailed distributions are rapidly increasing with applications to extreme values, insurance, survival analysis, networks, and risk analysis [2, 3]. Therefore, we choose a Pareto model. In this paper, we will propose a weighted method to study the hurricane loss data. The statistical analysis results are given in Section 6.


#### 1.1.2. A Forest Fire Loss Example

Large forest fires have a significant impact on natural, social, and economic systems. However, most fires are extinguished in the initial stages and thus remain small. These smaller fires have a large probability of occurrence, but the resulting damage is almost negligible on an individual basis. Large forest fires, however, have a low probability of occurrence, but the damage and the losses are huge. So modelling large fire losses is becoming critical in the analysis of the risk of the next large forest fire. Figure2 shows 30 forest fire losses during 1977 to 2006 in AB, Canada. We are concerned that the losses have been increasing over the last 30 years. The data listed in Table 1 are fire occurrence records from the Forestry Service (Council of Canadian Fire Marshals and Fire Commissioners, 2008, Canadian Fire Statistics, (http://www.ccfmfc.ca/stats.html). The database includes forest fire records for all the insurance losses. The data contains a relatively substantial number of large losses which convince us to use a Pareto model. The statistical analysis results are given in Section 6.Table 1
Forest fire loss data, AB, Canada (in million, 2006 CAD).
Year
Loss
1977
571.7
1978
654.9
1979
756.4
1980
979.0
1981
905.4
1982
998.9
1983
816.0
1984
929.5
1985
926.0
1986
973.5
1987
956.2
1988
1,004.3
1989
1,119.6
1990
1,237.4
1991
1,239.7
1992
1,241.4
1993
1,181.9
1994
1,151.5
1995
1,110.8
1996
1,163.3
1997
1,291.6
1998
1,175.6
1999
1,231.9
2000
1,185.2
2001
1,420.8
2002
1,489.0
2003
1,356.1
2004
1,126.6
2005
1,265.9
2006
1,403.3Figure 2
The 30 forest fire losses, 1977–2006, AB, Canada.


### 1.2. The Truncated Pareto Distribution

Many risk models with heavy tails have been developed using the class of the Pareto distributions. Some of these cases are found in city population sizes, the occurrence of natural resources, the sizes of firms, and personal income [3]. The Pareto distribution is on the list of distributions of Frechet domain which belongs to the extreme value distribution of type II [4]. It is important to explore estimation methods for Pareto distribution. There are theoretical difficulties in studying the Pareto distribution since it has infinite moments in heavy-tailed situations. We propose using the truncated Pareto distribution in these models to overcome these difficulties. Usually, we choose the upper limit as the largest value in the data set. In the recent years, the truncated Pareto distribution has become an alternative model for the original Pareto distribution.There are several kinds of Pareto distributions. We consider a type I Pareto distribution in this paper [3].Definition 1.
The probability density function (p.d.f.) and the cumulative distribution function (c.d.f.) of a random variableX having the Pareto distribution are given by
(1)fp(x)=αγαx(α+1),0<γ≤x<∞,α>0,(2)Fp(x)=1-(γx)α,0<γ≤x<∞,α>0,
where α is the shape parameter.
When0<α≤1, which is a heavy-tailed case, the mean and variance of X are infinite, and the distribution is heavier on the right tail as α decreases.The truncated Pareto distribution was originally used to describe the distribution of oil fields by size. It has a lower limitγ, an upper limit ν, and a shape parameter α. In fact, it has been shown that the truncated Pareto distribution fits better than the nontruncated distribution for positively skewed populations [5].Definition 2.
The p.d.f. and c.d.f. of a random variableX having the truncated Pareto distribution are given by
(3)f(x)=αγαx-α-11-(γ/ν)α,0<γ≤x≤ν<∞,α>0,(4)F(x)=1-γα(x-α-ν-α),1-(γ/ν)α,0<γ≤x≤ν<∞,α>0.
The quantile function of the truncated Pareto distribution is
(5)F-1(u)=(1-uγα+uνα)-1/α,0≤u≤1,0<γ≤ν<∞,α>0.
The mean and variance of X are
(6)μ=αγα(γ1-α-ν1-α)(α-1)(1-(γ/ν)α),0<γ<ν<∞,α>0,(7)σ2=αγα(ν2-α-γ2-α)(2-α)(1-(γ/ν)α)2-α2γ2α(ν1-α-γ1-α)2(1-α)2(1-(γ/ν)α)2,hhhhhhhhhhhhhhhhhhhi0<γ<ν<∞,α>0.


### 1.3. The Weighted Empirical Distribution Function

Recently, some parametric estimation methods for the truncated Pareto distribution has been developed [6]. But there is an efficiency problem in the estimates of the distribution tails. The distribution tail values and their probabilities are important in many fields, for example, value at risk in risk analysis, survival probability in survival analysis, tolerance limits in quality control, prediction intervals, and confidence intervals. Classical statistical inference theory depends on the classical empirical distribution function (EDF) Sn(x):
(8)Sn(x)=1n∑i=1nI(-∞,x](Xi),whereIA={1,ifx∈A;0,ifx∉A,Sn(x) is a minimum variance unbiased estimator for the c.d.f. F(x) based on a random sample X1,X2,…,Xn. Note that Sn(x) uses the equal weight 1/n for each sample point. Should we use equal weights on extreme data values as well? Recently, authors have applied various weights to data points by using different philosophies; that is, the Jackknife method gives zero weight for eliminated data [7]; weighted bootstrap [8] and weighted empirical distribution functions or processes have been discussed [9, 10]. But there are some difficulties to determine what weights should be used for the data points. Huang and Brill [11] introduced a weighted level crossing estimation method from a geometric point of view to visualize random samples in the L1-optimal sense; the method improves the efficiency of the estimation of tails.This paper proposes a semiparametric approach to estimateF(x) in (4) using L1-optimal and L2-optimal weights. Both theoretical and simulation efficiencies are consistently improved when compared with existing methods. This method is based on a symmetric weighted empirical distribution function (SWEDF) of Huang [12], namely,
(9)Fn*^(x)=∑i=1nI(-∞,x](X(i))pn,i,x∈𝔎,n>2,
where the pn,i are symmetric general weights
(10)pn,i≡{w,i=2,…,n-1,0<w<1n-2,w1,n≡12(1-(n-2)w),i=1,n,
and X(1)≤X(2)≤⋯≤X(n) are the order statistics of the random sample. Note that
(11)0<pn,i<1,i=1,…,n;∑i=1npn,i=1.The parameterw in (9) is the weight for the middle n-2 data; w1,n in (9) is the weight for the extreme data. It is interesting to explore how the value of w affects the estimation of a heavy-tailed distribution. We may use w1,n flexibly.In Section2, we propose a weighted method for estimating the shape parameter and the mean of the truncated Pareto distribution. In Section 3, an exact efficiency function of the new mean estimator relative to the classical estimator is derived. Section 4 explores the L1-optimal weights and L2-optimal weights for estimating the mean. Section 5 gives results of Monte Carlo simulations. The simulation efficiencies are consistent with the exact efficiencies in Section 3. In Section 6, we analyze the hurricane loss data and forest fire loss data given in Section 1 by using the proposed method. The statistical inference in these examples shows that the estimated distribution curve by using proposed weighted estimation method fits the tails of data better, relative to several existing methods. Suggestions for further studies are also discussed.


## 1.1. Two Motivating Examples

In the recent years, many extreme events have occurred in financial markets, natural disasters, disease control, and industrial quality control. Natural disasters, for example, earthquakes, hurricanes, forest fires, volcanoes, and floods affect human life. It is important to predict and prepare for the next disaster occurrence and to estimate losses to inhabitants, insurance companies, and governments. In this section, we study two examples.


### 1.1.1. A Hurricane Loss Example

Strong winds, heavy rainfall, and storm surges caused by hurricanes cause death and destroy properties. They generate great losses to insurance companies as well. Figure1 shows the 49 costliest Atlantic hurricane losses for the United States during 1900–2005 [1]. The measurement of this hurricane loss data is in US dollars; all dollars have been adjusted by using the inflation rates from 1900 to 2005.Figure 1
The 49 USA atlantic hurricane loss, 1900–2005.From the data in [1], we note that the most costly hurricane is the 1926 Great Miami Hurricane with cost of damage of 157 billion which is 1.58 times larger than the second worst hurricane, the 1900 Galveston Hurricane. After 1926, on August 28, 2005 Hurricane Katrina caused damage of 81 billion. This is approximately 1.19 times larger than Hurricane Andrew in August 1992, which caused damage of 56 billion. On the other hand, we note that 80% of the hurricane losses are less than 21 billion. Many smaller hurricanes are not listed. These considerations raise a number of questions.(a)
How do we predict the loss of the next hurricane? Will it again be 1.58 times larger than the worst one so far?(b)
Do we keep the traditional approach which is to prepare for the worst event? Should we focus on the extreme events?(c)
What is the value at risk with 5% probability?(d)
How do we set an upper limit of the losses?(e)
How do we estimate the distribution of largest losses?The objective of this study is to find the best model to fit the empirical data and to answer the above questions as accurately as possible. Since the data has large losses, we consider that the data should fit a heavy-tailed distribution. Studies on Pareto-type heavy-tailed distributions are rapidly increasing with applications to extreme values, insurance, survival analysis, networks, and risk analysis [2, 3]. Therefore, we choose a Pareto model. In this paper, we will propose a weighted method to study the hurricane loss data. The statistical analysis results are given in Section 6.


### 1.1.2. A Forest Fire Loss Example

Large forest fires have a significant impact on natural, social, and economic systems. However, most fires are extinguished in the initial stages and thus remain small. These smaller fires have a large probability of occurrence, but the resulting damage is almost negligible on an individual basis. Large forest fires, however, have a low probability of occurrence, but the damage and the losses are huge. So modelling large fire losses is becoming critical in the analysis of the risk of the next large forest fire. Figure2 shows 30 forest fire losses during 1977 to 2006 in AB, Canada. We are concerned that the losses have been increasing over the last 30 years. The data listed in Table 1 are fire occurrence records from the Forestry Service (Council of Canadian Fire Marshals and Fire Commissioners, 2008, Canadian Fire Statistics, (http://www.ccfmfc.ca/stats.html). The database includes forest fire records for all the insurance losses. The data contains a relatively substantial number of large losses which convince us to use a Pareto model. The statistical analysis results are given in Section 6.Table 1
Forest fire loss data, AB, Canada (in million, 2006 CAD).
Year
Loss
1977
571.7
1978
654.9
1979
756.4
1980
979.0
1981
905.4
1982
998.9
1983
816.0
1984
929.5
1985
926.0
1986
973.5
1987
956.2
1988
1,004.3
1989
1,119.6
1990
1,237.4
1991
1,239.7
1992
1,241.4
1993
1,181.9
1994
1,151.5
1995
1,110.8
1996
1,163.3
1997
1,291.6
1998
1,175.6
1999
1,231.9
2000
1,185.2
2001
1,420.8
2002
1,489.0
2003
1,356.1
2004
1,126.6
2005
1,265.9
2006
1,403.3Figure 2
The 30 forest fire losses, 1977–2006, AB, Canada.


## 1.1.1. A Hurricane Loss Example

Strong winds, heavy rainfall, and storm surges caused by hurricanes cause death and destroy properties. They generate great losses to insurance companies as well. Figure1 shows the 49 costliest Atlantic hurricane losses for the United States during 1900–2005 [1]. The measurement of this hurricane loss data is in US dollars; all dollars have been adjusted by using the inflation rates from 1900 to 2005.Figure 1
The 49 USA atlantic hurricane loss, 1900–2005.From the data in [1], we note that the most costly hurricane is the 1926 Great Miami Hurricane with cost of damage of 157 billion which is 1.58 times larger than the second worst hurricane, the 1900 Galveston Hurricane. After 1926, on August 28, 2005 Hurricane Katrina caused damage of 81 billion. This is approximately 1.19 times larger than Hurricane Andrew in August 1992, which caused damage of 56 billion. On the other hand, we note that 80% of the hurricane losses are less than 21 billion. Many smaller hurricanes are not listed. These considerations raise a number of questions.(a)
How do we predict the loss of the next hurricane? Will it again be 1.58 times larger than the worst one so far?(b)
Do we keep the traditional approach which is to prepare for the worst event? Should we focus on the extreme events?(c)
What is the value at risk with 5% probability?(d)
How do we set an upper limit of the losses?(e)
How do we estimate the distribution of largest losses?The objective of this study is to find the best model to fit the empirical data and to answer the above questions as accurately as possible. Since the data has large losses, we consider that the data should fit a heavy-tailed distribution. Studies on Pareto-type heavy-tailed distributions are rapidly increasing with applications to extreme values, insurance, survival analysis, networks, and risk analysis [2, 3]. Therefore, we choose a Pareto model. In this paper, we will propose a weighted method to study the hurricane loss data. The statistical analysis results are given in Section 6.


## 1.1.2. A Forest Fire Loss Example

Large forest fires have a significant impact on natural, social, and economic systems. However, most fires are extinguished in the initial stages and thus remain small. These smaller fires have a large probability of occurrence, but the resulting damage is almost negligible on an individual basis. Large forest fires, however, have a low probability of occurrence, but the damage and the losses are huge. So modelling large fire losses is becoming critical in the analysis of the risk of the next large forest fire. Figure2 shows 30 forest fire losses during 1977 to 2006 in AB, Canada. We are concerned that the losses have been increasing over the last 30 years. The data listed in Table 1 are fire occurrence records from the Forestry Service (Council of Canadian Fire Marshals and Fire Commissioners, 2008, Canadian Fire Statistics, (http://www.ccfmfc.ca/stats.html). The database includes forest fire records for all the insurance losses. The data contains a relatively substantial number of large losses which convince us to use a Pareto model. The statistical analysis results are given in Section 6.Table 1
Forest fire loss data, AB, Canada (in million, 2006 CAD).
Year
Loss
1977
571.7
1978
654.9
1979
756.4
1980
979.0
1981
905.4
1982
998.9
1983
816.0
1984
929.5
1985
926.0
1986
973.5
1987
956.2
1988
1,004.3
1989
1,119.6
1990
1,237.4
1991
1,239.7
1992
1,241.4
1993
1,181.9
1994
1,151.5
1995
1,110.8
1996
1,163.3
1997
1,291.6
1998
1,175.6
1999
1,231.9
2000
1,185.2
2001
1,420.8
2002
1,489.0
2003
1,356.1
2004
1,126.6
2005
1,265.9
2006
1,403.3Figure 2
The 30 forest fire losses, 1977–2006, AB, Canada.


## 1.2. The Truncated Pareto Distribution

Many risk models with heavy tails have been developed using the class of the Pareto distributions. Some of these cases are found in city population sizes, the occurrence of natural resources, the sizes of firms, and personal income [3]. The Pareto distribution is on the list of distributions of Frechet domain which belongs to the extreme value distribution of type II [4]. It is important to explore estimation methods for Pareto distribution. There are theoretical difficulties in studying the Pareto distribution since it has infinite moments in heavy-tailed situations. We propose using the truncated Pareto distribution in these models to overcome these difficulties. Usually, we choose the upper limit as the largest value in the data set. In the recent years, the truncated Pareto distribution has become an alternative model for the original Pareto distribution.There are several kinds of Pareto distributions. We consider a type I Pareto distribution in this paper [3].Definition 1.
The probability density function (p.d.f.) and the cumulative distribution function (c.d.f.) of a random variableX having the Pareto distribution are given by
(1)fp(x)=αγαx(α+1),0<γ≤x<∞,α>0,(2)Fp(x)=1-(γx)α,0<γ≤x<∞,α>0,
where α is the shape parameter.
When0<α≤1, which is a heavy-tailed case, the mean and variance of X are infinite, and the distribution is heavier on the right tail as α decreases.The truncated Pareto distribution was originally used to describe the distribution of oil fields by size. It has a lower limitγ, an upper limit ν, and a shape parameter α. In fact, it has been shown that the truncated Pareto distribution fits better than the nontruncated distribution for positively skewed populations [5].Definition 2.
The p.d.f. and c.d.f. of a random variableX having the truncated Pareto distribution are given by
(3)f(x)=αγαx-α-11-(γ/ν)α,0<γ≤x≤ν<∞,α>0,(4)F(x)=1-γα(x-α-ν-α),1-(γ/ν)α,0<γ≤x≤ν<∞,α>0.
The quantile function of the truncated Pareto distribution is
(5)F-1(u)=(1-uγα+uνα)-1/α,0≤u≤1,0<γ≤ν<∞,α>0.
The mean and variance of X are
(6)μ=αγα(γ1-α-ν1-α)(α-1)(1-(γ/ν)α),0<γ<ν<∞,α>0,(7)σ2=αγα(ν2-α-γ2-α)(2-α)(1-(γ/ν)α)2-α2γ2α(ν1-α-γ1-α)2(1-α)2(1-(γ/ν)α)2,hhhhhhhhhhhhhhhhhhhi0<γ<ν<∞,α>0.


## 1.3. The Weighted Empirical Distribution Function

Recently, some parametric estimation methods for the truncated Pareto distribution has been developed [6]. But there is an efficiency problem in the estimates of the distribution tails. The distribution tail values and their probabilities are important in many fields, for example, value at risk in risk analysis, survival probability in survival analysis, tolerance limits in quality control, prediction intervals, and confidence intervals. Classical statistical inference theory depends on the classical empirical distribution function (EDF) Sn(x):
(8)Sn(x)=1n∑i=1nI(-∞,x](Xi),whereIA={1,ifx∈A;0,ifx∉A,Sn(x) is a minimum variance unbiased estimator for the c.d.f. F(x) based on a random sample X1,X2,…,Xn. Note that Sn(x) uses the equal weight 1/n for each sample point. Should we use equal weights on extreme data values as well? Recently, authors have applied various weights to data points by using different philosophies; that is, the Jackknife method gives zero weight for eliminated data [7]; weighted bootstrap [8] and weighted empirical distribution functions or processes have been discussed [9, 10]. But there are some difficulties to determine what weights should be used for the data points. Huang and Brill [11] introduced a weighted level crossing estimation method from a geometric point of view to visualize random samples in the L1-optimal sense; the method improves the efficiency of the estimation of tails.This paper proposes a semiparametric approach to estimateF(x) in (4) using L1-optimal and L2-optimal weights. Both theoretical and simulation efficiencies are consistently improved when compared with existing methods. This method is based on a symmetric weighted empirical distribution function (SWEDF) of Huang [12], namely,
(9)Fn*^(x)=∑i=1nI(-∞,x](X(i))pn,i,x∈𝔎,n>2,
where the pn,i are symmetric general weights
(10)pn,i≡{w,i=2,…,n-1,0<w<1n-2,w1,n≡12(1-(n-2)w),i=1,n,
and X(1)≤X(2)≤⋯≤X(n) are the order statistics of the random sample. Note that
(11)0<pn,i<1,i=1,…,n;∑i=1npn,i=1.The parameterw in (9) is the weight for the middle n-2 data; w1,n in (9) is the weight for the extreme data. It is interesting to explore how the value of w affects the estimation of a heavy-tailed distribution. We may use w1,n flexibly.In Section2, we propose a weighted method for estimating the shape parameter and the mean of the truncated Pareto distribution. In Section 3, an exact efficiency function of the new mean estimator relative to the classical estimator is derived. Section 4 explores the L1-optimal weights and L2-optimal weights for estimating the mean. Section 5 gives results of Monte Carlo simulations. The simulation efficiencies are consistent with the exact efficiencies in Section 3. In Section 6, we analyze the hurricane loss data and forest fire loss data given in Section 1 by using the proposed method. The statistical inference in these examples shows that the estimated distribution curve by using proposed weighted estimation method fits the tails of data better, relative to several existing methods. Suggestions for further studies are also discussed.


## 2. Estimation Methods

In this section, we discuss the existing and proposed methodologies for the truncated Pareto distribution. Consider a random sampleX1,X2,…,Xn from the distribution in (3), and let X(1)≤X(2)≤⋯≤X(n) denote its order statistics.


### 2.1. Maximum Likelihood Estimators (Hill, Beg, and Aban)

There are several different maximum likelihood estimators (MLE) for estimating the shape parameterα.A popular estimator is the Hill [13] MLE,which uses the r largest order statistics, 1≤r≤n-1, to estimate the original Pareto shape parameter α in (1). When applying it to the truncated Pareto distribution in (3), it is defined as
(12)α^Hill=[r-1∑‍ri=1{lnX(n-i+1)-lnX(n-r)}]-1.Beg [5] developed the MLE method for the truncated Pareto distribution when ν is known. The Beg MLE for α can be obtained by solving the following equation:
(13)n[1α^Beg+ln(X(1)/α^Beg)((X(1)/ν)-α^Beg-1)-ln(gX(1))]=0,
where X(1)=min(X1,X2,…,Xn) and g=(X(1)X(2)⋯X(n))1/n is the sample geometric mean.Aban’s MLE [6] when γ,ν are known is obtained by solving the equation
(14)nα^Aban+n(γ/ν)α^Abanln(γ/ν)1-(γ/ν)α^Aban-∑‍ni=1[lnX(i)-lnγ]=0,
where γ≤X(i)≤ν and n is the sample size.Note that we may use estimatorsγ^=min(X1,X2,…,Xn) and ν^=max(X1,X2,…,Xn) in (14) when γ,ν are unknown. A similar situation is in the following equations (16)and (18). There are other estimators of γ and ν in the literature, for example, Cooke [14].


### 2.2. Moment Estimator

To estimate the population truncated Pareto mean, the sample mean estimator is(15)μ^X¯=X¯=1n∑‍ni=1Xi.
A Moment estimator α^M for estimating α is the solution of the equation
(16)X¯=α^Mγα^M(γ1-α^M-ν1-α^M)(α^M-1)(1-(γ/ν)α^M),α^M>0.


### 2.3. A Proposed Weighted Estimator

Now, to estimate the population truncated Pareto mean, we define a weighted mean based on the weighted empirical distribution function in(9) as
(17)μ^w=X¯w=1n∑i=1npn,iX(i)=∑‍n-1i=2wX(i)+12(1-(n-2)w)[X(1)+X(n)],
where n is the sample size and w is the weight as defined in (9). Then, for estimating α, we define α^w to be the solution of the equation
(18)X¯w=α^wγα^w(γ1-α^w-ν1-α^w)(α^w-1)(1-(γ/ν)α^w),α^w>0.


## 2.1. Maximum Likelihood Estimators (Hill, Beg, and Aban)

There are several different maximum likelihood estimators (MLE) for estimating the shape parameterα.A popular estimator is the Hill [13] MLE,which uses the r largest order statistics, 1≤r≤n-1, to estimate the original Pareto shape parameter α in (1). When applying it to the truncated Pareto distribution in (3), it is defined as
(12)α^Hill=[r-1∑‍ri=1{lnX(n-i+1)-lnX(n-r)}]-1.Beg [5] developed the MLE method for the truncated Pareto distribution when ν is known. The Beg MLE for α can be obtained by solving the following equation:
(13)n[1α^Beg+ln(X(1)/α^Beg)((X(1)/ν)-α^Beg-1)-ln(gX(1))]=0,
where X(1)=min(X1,X2,…,Xn) and g=(X(1)X(2)⋯X(n))1/n is the sample geometric mean.Aban’s MLE [6] when γ,ν are known is obtained by solving the equation
(14)nα^Aban+n(γ/ν)α^Abanln(γ/ν)1-(γ/ν)α^Aban-∑‍ni=1[lnX(i)-lnγ]=0,
where γ≤X(i)≤ν and n is the sample size.Note that we may use estimatorsγ^=min(X1,X2,…,Xn) and ν^=max(X1,X2,…,Xn) in (14) when γ,ν are unknown. A similar situation is in the following equations (16)and (18). There are other estimators of γ and ν in the literature, for example, Cooke [14].


## 2.2. Moment Estimator

To estimate the population truncated Pareto mean, the sample mean estimator is(15)μ^X¯=X¯=1n∑‍ni=1Xi.
A Moment estimator α^M for estimating α is the solution of the equation
(16)X¯=α^Mγα^M(γ1-α^M-ν1-α^M)(α^M-1)(1-(γ/ν)α^M),α^M>0.


## 2.3. A Proposed Weighted Estimator

Now, to estimate the population truncated Pareto mean, we define a weighted mean based on the weighted empirical distribution function in(9) as
(17)μ^w=X¯w=1n∑i=1npn,iX(i)=∑‍n-1i=2wX(i)+12(1-(n-2)w)[X(1)+X(n)],
where n is the sample size and w is the weight as defined in (9). Then, for estimating α, we define α^w to be the solution of the equation
(18)X¯w=α^wγα^w(γ1-α^w-ν1-α^w)(α^w-1)(1-(γ/ν)α^w),α^w>0.


## 3. An Exact Efficiency Function

In this section, an exact efficiency function of the weighted estimatorμ^w in (17) relative to the sample mean X¯ in (15) is given. The mathematical derivations are given in the appendix.Theorem 3.
The mean and mean square error (MSE), ofμ^w in (17) when α>0, are given by
(19)E[μ^w]=γ∑‍i=2n-1∑‍∞k=0{(1α+k-1k)×ρk[Γ(n+1)Γ(k+i)Γ(i)Γ(k+n+1)ffff+12(1-(n-2)w)ffff×ρk(Γ(n+1)Γ(k+1)Γ(k+n+1)+nk+n)]},(20)MSE(μ^w)=aw2+bw+c,where0<w<1,a=A+14(n-2)2B-(n-2)C,b=-12(n-2)B+C-2μD+μ(n-2)E,c=14B-μE+μ2,A=γ∑‍i=2n-1∑‍∞k=0(2α+k-1k)ρkΓ(n+1)Γ(k+i)Γ(i)Γ(k+n+1)+2γ2∑‍i=2n-2∑‍j=i+1n-1∑‍∞k=0∑‍∞l=0{(1α+k-1k)(1α+l-1l)hhhhhhhhhhhhhhhhh×ρk+lhhhhhhhhhhhhhhhhhhh×[Γ(n+1)Γ(k+i)Γ(k+l+j)Γ(i)Γ(k+j)Γ(k+l+n+1)](1α+k-1k)},B=γ∑‍∞k=0{(2α+k-1k)gggggg×ρk[Γ(n+1)Γ(k+1)Γ(k+n+1)+nk+n](2α+k-1k)}+2γ2∑‍∞k=0∑‍∞l=0{(1α+k-1k)(1α+l-1l)hhhhhhhhh×ρk+l[Γ(n+1)Γ(k+1)Γ(k+n)(k+l+n)](2α+k-1k)},C=γ2∑‍i=2n-1∑‍∞k=0∑‍∞l=0{(1α+k-1k)(1α+l-1l)×ρk+l[Γ(n+1)Γ(k+1)Γ(k+l+i)Γ(k+i)Γ(k+l+n+1)hhhhhh+Γ(n+1)Γ(k+i)Γ(i)Γ(k+n)(k+l+n)](1α+l-1l)},D=γ∑‍i=2n-1∑‍∞k=0{(1α+k-1k)×ρkΓ(n+1)Γ(k+i)Γ(i)Γ(k+n+1)(1α+k-1k)},E=γ∑‍∞k=0{(1α+k-1k)×ρk[Γ(n+1)Γ(k+1)Γ(k+n+1)hhhhh+nk+nΓ(n+1)Γ(k+1)Γ(k+n+1)](1α+k-1k)},
where ρ=(gα-1)/gα, g=ν/γ, 0<ρ<1, 0<g<1, 0<γ≤x≤ν<∞,
(21)Γ(z)=∫0∞tz-1e-tdt,z>0,istheGammafunction.Corollary 4.
The efficiency function (EFF) ofμ^w in (17) relative to the sample mean X¯ in (15) is given by
(22)EFF(μ^w)=Var(X¯)MSE(μ^w)=σ2/nMSE(μ^w)=αγα(ν2-α-γ2-α)/{(2-α)(1-(γ/ν)α)}nMSE(μ^w),-α2γ2α(ν1-α-γ1-α)2/{(1-α)2(1-(γ/ν)α)2}nMSE(μ^w),
where σ2 and MSE(μ^w) are given in (7) and (20).


## 4. Optimal Weights

### 4.1.L1-Optimal Weights

Huang and Brill [11] proposed an L1-optimal weight which is based on Manhattan metric for the Fn*^(x) in (9). It is
(23)wL1-opt=1n(n-1)>1n,forthemiddledata.
Huang and Brill [11] proved that the exact efficiency (EFF) of Fn*^(x) in (9)relative to the EDF Sn(x) exceeds 1 on the tails of the distribution. In this paper, we use this wL1-opt weight in order to improve the efficiency of estimating the tail probability of the truncated Pareto distribution. Huang [12] indicates that in general for any distribution if w>1/n, which means putting more weight on the middle data, the efficiency of estimating the tail probability will be improved.Next, we explore an alternativeL2-optimal weight.


### 4.2.L2-Optimal Weights

Corollary 5.
AnL2-optimal weight wL2-opt for the efficiency function of the μ^w given in (17) for estimating the population mean in (3) relative to the sample mean X¯ in (16) when n>4C/B+2 is given by
(24)wL2-opt=-b2a.
The minimum MSEmin(μ^w) and EFFmax(μ^w) are given by
(25)MSEmin(μ^w)=-b24a+c,EFFmax(μ^w)=αγα(ν2-α-γ2-α)/{(2-α)(1-(γ/ν)α)}n(-b2/(4a)+c)-α2γ2α(ν1-α-γ1-α)2/{(1-α)2(1-(γ/ν)α)2}n(-b2/(4a)+c).
In (24) and (25), a, b, c, C, and B are defined in (20).The proof of Corollary5 is in the appendix.Table2 lists the values of wL2-opt, wL1-opt, MSEmin(μ^w), MSEmin(X¯), and the exact EFFmax(μ^w) of μ^w relative to X¯ for n=10,20,30,50, and 100; α=0.8,1.8 and 3, by using (24) and (25). We note that all values of wL2-opt are greater than 1/n, and all exact EFFmax(μ^w) relative to X¯ are greater than 1.Table 2
Exact efficiencies ofμ^w relative to other estimators by using L2-optimal  weights, γ=1, ν=10.
Sample size
α
w
L
2
-opt
w
=
1
/
n
w
L
1
-opt
MSE(μ^w)
MSE(X¯)
EFF(μ^w(X¯))
n
=
10
0.8
0.10396
0.1
0.10541
0.4007
0.4034
1.0068
1.8
0.11446
0.1406
0.1647
1.1717
3
0.11847
0.0365
0.0493
1.3501
n
=
20
0.8
0.05015
0.05
0.05130
0.2017
0.2017
1.0001
1.8
0.05255
0.0766
0.0824
1.0759
3
0.05463
0.0217
0.0247
1.1367
n
=
30
0.8
0.03323
0.03333
0.03390
0.1344
0.1345
1.0003
1.8
0.03425
0.0528
0.0549
1.0399
3
0.03529
0.0155
0.0164
1.0628
n
=
50
0.8
0.02021
0.02
0.02020
0.0968
0.1063
1.0978
1.8
0.02159
0.0514
0.0592
1.1536
3
0.02298
0.0274
0.0352
1.2849
n
=
100
0.8
0.01001
0.01
0.01005
0.0765
0.0830
1.0845
1.8
0.01002
0.0542
0.0576
1.0637
3
0.01002
0.0380
0.0400
1.0526Remark 6.
ThewL1-opt in (23) is totally nonparametric; it is more robust and easy to use. Note that wL2-opt in (24)depends on the parameter α. In practice, we may estimate α first then obtain a wL2-opt, while still keeping the optimal advantage. Of course, we use the given α values in the simulations. However, wL1-opt and wL2-opt are close to each other when n>100.


## 4.1.L1-Optimal Weights

Huang and Brill [11] proposed an L1-optimal weight which is based on Manhattan metric for the Fn*^(x) in (9). It is
(23)wL1-opt=1n(n-1)>1n,forthemiddledata.
Huang and Brill [11] proved that the exact efficiency (EFF) of Fn*^(x) in (9)relative to the EDF Sn(x) exceeds 1 on the tails of the distribution. In this paper, we use this wL1-opt weight in order to improve the efficiency of estimating the tail probability of the truncated Pareto distribution. Huang [12] indicates that in general for any distribution if w>1/n, which means putting more weight on the middle data, the efficiency of estimating the tail probability will be improved.Next, we explore an alternativeL2-optimal weight.


## 4.2.L2-Optimal Weights

Corollary 5.
AnL2-optimal weight wL2-opt for the efficiency function of the μ^w given in (17) for estimating the population mean in (3) relative to the sample mean X¯ in (16) when n>4C/B+2 is given by
(24)wL2-opt=-b2a.
The minimum MSEmin(μ^w) and EFFmax(μ^w) are given by
(25)MSEmin(μ^w)=-b24a+c,EFFmax(μ^w)=αγα(ν2-α-γ2-α)/{(2-α)(1-(γ/ν)α)}n(-b2/(4a)+c)-α2γ2α(ν1-α-γ1-α)2/{(1-α)2(1-(γ/ν)α)2}n(-b2/(4a)+c).
In (24) and (25), a, b, c, C, and B are defined in (20).The proof of Corollary5 is in the appendix.Table2 lists the values of wL2-opt, wL1-opt, MSEmin(μ^w), MSEmin(X¯), and the exact EFFmax(μ^w) of μ^w relative to X¯ for n=10,20,30,50, and 100; α=0.8,1.8 and 3, by using (24) and (25). We note that all values of wL2-opt are greater than 1/n, and all exact EFFmax(μ^w) relative to X¯ are greater than 1.Table 2
Exact efficiencies ofμ^w relative to other estimators by using L2-optimal  weights, γ=1, ν=10.
Sample size
α
w
L
2
-opt
w
=
1
/
n
w
L
1
-opt
MSE(μ^w)
MSE(X¯)
EFF(μ^w(X¯))
n
=
10
0.8
0.10396
0.1
0.10541
0.4007
0.4034
1.0068
1.8
0.11446
0.1406
0.1647
1.1717
3
0.11847
0.0365
0.0493
1.3501
n
=
20
0.8
0.05015
0.05
0.05130
0.2017
0.2017
1.0001
1.8
0.05255
0.0766
0.0824
1.0759
3
0.05463
0.0217
0.0247
1.1367
n
=
30
0.8
0.03323
0.03333
0.03390
0.1344
0.1345
1.0003
1.8
0.03425
0.0528
0.0549
1.0399
3
0.03529
0.0155
0.0164
1.0628
n
=
50
0.8
0.02021
0.02
0.02020
0.0968
0.1063
1.0978
1.8
0.02159
0.0514
0.0592
1.1536
3
0.02298
0.0274
0.0352
1.2849
n
=
100
0.8
0.01001
0.01
0.01005
0.0765
0.0830
1.0845
1.8
0.01002
0.0542
0.0576
1.0637
3
0.01002
0.0380
0.0400
1.0526Remark 6.
ThewL1-opt in (23) is totally nonparametric; it is more robust and easy to use. Note that wL2-opt in (24)depends on the parameter α. In practice, we may estimate α first then obtain a wL2-opt, while still keeping the optimal advantage. Of course, we use the given α values in the simulations. However, wL1-opt and wL2-opt are close to each other when n>100.


## 5. Simulations

Next, we use simulations to compare the performance of the three parametric MLE estimators, that is, Hill’s, Beg’s, and Aban’s estimators in(12), (13), and(14), and the two semiparametric estimators, that is, the Moment and the weighted estimators in (16)and(18),for estimating the shape parameter α in (3). We generate m = 1,000 random samples of size n=100 from the distribution of (3). wL1-opt is only used for the weighted estimator, since Table 2 indicates that the values of wL2-opt and wL1-opt are close to each other when n=100.We know that ifα≤2, the original Pareto distribution has an infinite variance; if α≤1, then the mean is infinite. These cases have inference difficulties. We focus on α=0.5,0.8, and 1.8, and let γ=1, ν=10. Figure 3 contains the box-plots of the comparison of these five estimators. Note that in the cases α=0.5 and 0.8, Beg's and Hill's estimators have large biases, and the weighted, Aban's, and the moment estimators performed very well; but sometimes, Aban's MLE estimates have unstable solutions. When α=1.8, there are similar results except Hill's and Beg's estimators performed better. It is interesting to see that, in all three cases, the median of the weighted estimator for α is relatively larger than Aban's and the Moment's ones. We will discuss how these affect the tail estimation in the next section. The simulations were run by using MAPLE 15 with double precision.Figure 3
Box-plot of the estimators for the shape parameterα, by using L1-optimal weight, γ=1, ν=10, sample size n=100, and generating times m= 1,000.


## 6. Examples

Now we use the proposed method and compare it with existing methods to analyze the data of the two examples outlined in Section1.


### 6.1. Hurricane Loss Example

#### 6.1.1. Comparison of Four Estimation Methods

At first, We look at the hurricane loss example in Section1.1.1. Based on the simulation results in Figure 3, we consider the three better estimators out of the five in the heavy-tailed case, that is, Aban’s, Moment, and the weighted estimators in (27), (28), and (29) for the truncated Pareto p.d.f.f(x) in (3)and the c.d.f.F(x) in (4).We also compare them with Hill’s estimator in (26)for the original Pareto p.d.f. fp(x)  in(1)and the c.d.f. Fp(x) in (2). Here
(26)f^Hill(x)=α^Hillγα^Hillxα^w,F^Hill(x)=1-(γx)α^Hill,
where α^Hill is given in (12) with r=n-1; consider
(27)f^MLE(x)=α^MLEγα^MLEx-α^MLE-11-(γ/ν)α^MLE,F^MLE(x)=1-γα^MLE(x-α^MLE-ν-α^MLE)1-(γ/ν)α^MLE,
where  α^MLE  is Aban’s MLE given in (14); consider
(28)f^M(x)=α^Mγα^Mx-α^M-11-(γ/ν)α^M,F^M(x)=1-γα^M(x-α^M-ν-α^M)1-(γ/ν)α^M,
where  α^Mis given in (16); consider
(29)f^w(x)=α^wγα^wx-α^w-11-(γ/ν)α^w,F^w(x)=1-γα^w(x-α^w-ν-α^w)1-(γ/ν)α^w,
where α^w is given in (18), using the weight wL1-opt=1/n(n-1).The results of these four methods are listed in Table3 by using the hurricane loss data, where γ^=X(min), ν^=X(max), and n=49.Table 3
Estimation of index, mean, median, and VaR for hurricane loss data.
Method
α
^
μ
^
Median
5
% value at risk
Pareto(Hill)
0.8126
∞
8.68 billion
147.68 billion
TPD(Aban)
0.6206
21.74 billion
9.73 billion
85.15 billion
TPD(Moment)
0.6476
20.48 billion
9.47 billion
82.55 billion
TPD(Weighted)
0.6990
19.37 billion
9.02 billion
77.58 billionFigure4 is a log-log plot showing the upper tail for the hurricane loss data. In this plot, the circles represent the real data, and the straight line represents the estimated original Pareto distribution. The dashed line, dotted line, and thick solid line represent the estimated truncated Pareto distributions by using Aban's, Moment, and Huang's estimators, respectively. We observe two conclusions intuitively.(1)
The original Pareto distribution (straight line) does not fit the data well in the tail. The truncated Pareto distribution fits the data very well using all three estimation methods (Aban, Moment, and weighted). Note that the 5% value at risk estimated by the original Pareto model is 147 billion; the three truncated model estimates are about 80 billion. It appears that the original Pareto model overestimates the risk. An insurance company would set a high premium if it is using the Pareto model, with the result that many people cannot afford to buy insurance.(2)
We examine the three truncated Pareto estimates. Around the tail, the weighted estimate fits the data the best; that is, theln(P{X>x}) curve turns downward more quickly following the trend of the data pattern because the weighted estimate α^w is largest among the three truncated model estimates (this is consistent with the simulation results in Figure 4), and the weighted mean estimate μ^w and the 5% value at risk are the smallest among the three methods (this is obtained by placing less weight on the extreme value, e.g., the 1926 great miami hurricane's loss), so its estimated distribution is less heavy in the tail compared with other methods.Figure 4
Log-log plot of the hurricane loss data.Next, in order to confirm these conclusions, we run three goodness of-fit tests. Later, we define the absolute error and integrated error as the measures of the distance from the empirical data points to the estimated Pareto curve and truncated Pareto curves.


#### 6.1.2. Goodness-of-Fit Tests

Our objective is to test if the estimated distributions in(26)–(29)fit the data properly. We test the hypotheses H0:F(x)=F*(x) against H1:F(x)≠F*(x), where F(x) is the true unknown distribution function and F*(x) is the estimated Pareto c.d.f. in(26)or the estimated truncated Pareto c.d.f. in (27)–(29). In this paper we use three EDF goodness-of-fit tests.(1) The Kolmogorov-Smirnov (K-S) test [15]: the test statistic is given by
(30)T=supx|F*(x)-Sn(x)|,-∞<x<∞,
where Sn(x) is the EDF in (8). Under H0, the two-tailed P value for the K-S test is as follows:
(31)P-value=1-[1-T∑j=0[n(1-T)]‍n!j!(n-j)!(1-T-jn)n-j(t+jn)j-1]2,
where [n(1-T)] is the integer part of n(1-T).(2) Anderson and Darling [16] test (A-D test) introduced a measure of “distance” between the empirical distribution Sn(x) and the proposed c.d.f. F*(x) by using a metric function space
(32)Wn2=n∫-∞∞[Sn(x)-F*(x)]2ψ(F*(x))dF,
where ψ(u)=1/(u/(1-u)) is a weight function, with u=F(x). Let ui=F(X(i)),i=1,…,n, and under H0 the test statistic and P-value are given by
(33)Wn2=-n-1n∑j=1n‍(2j-1)(log(uj)+log(1-un-j+1)),P-value=1-2z∑j=0∞‍{(-1)jΓ(j+1/2)(4j+1)j!e-(4j+1)2π2/(8z)=1-2z∑j=0∞‍×∫0∞e((z/8)/(w2+1)-(4j+1)2π2w2/(8z))dw},
where z is the observed value of Wn2 and Γ(x)=∫0∞tx-1e-tdt,  x>0, is the Gamma function.(3) Cramer-von Mises test [16] proposed (C-v-M test) using ψ(u)=1 in (32); thus, under H0, the test statistic and P-value are given by
(34)nω2=112n+∑j=1n‍(uj-2j-12n)2,P-value=1-1πz∑j=0∞‍{(-1)2jΓ(j+1/2)(4j+1)1/2j!πe-(4j+1)2/(16z)P-value=1-1πz∑j=0∞‍1-×K1/4((4j+1)216z)},
where K1/4(x) is the modified Bessel function of the second kind
(35)hhhhhKα(x)=π2I-α(x)-Iα(x)sin(απ),hhhhhIα(x)=∑m=0∞‍1m!Γ(m+α+1)(x2)2m+α.For fitting the 49 losses in the hurricane loss data, for example, after computing estimates ofF*(x) by using (26), (27), (28),and(29), we compute theabsolute error(AE)in (30) and define the integrated error (IE) by
(36)IE=[∫X(n-r+1)X(n)‍(Sn(x)-F*(x))2dx]1/2.Figure5 gives the absolute errors (AE) in (30) for the 10 largest losses and confirms that the original Pareto estimate has relatively larger errors and the weighted estimate has relatively smaller errors in the tail. Figure 5 explains the data fitting of the tail of the distributions in Figure 4.Figure 5
Absolute errors|F*(x)-Sn(x)| for hurricane loss example on the 10 largest losses data.We also compute the AE in(30)and IE in(36) to confirm those tail errors by using r as the number of the largest losses, 1≤r≤n. The AE and IE values are given in Table 4 (the smallest values are bold with *).Table 4
Errors of goodness-of-fit testsn=49 for hurricane example.
Method
Goodness-of-fit tests
Absolute error (AE)
Integrated error (IE)
r
=
49
r
=
18
r
=
10
r
=
49
r
=
18
r
=
10
Pareto(Hill)
0.1340
0.0584
0.0584
0.4844
0.3818
0.3723
TPD(Aban)
0.0948
*
0.0839
0.0832
0.3114
0.2565
0.2161
TPD(Moment)
0.1053
0.0738
0.0737
0.2985
*
0.2171
0.1825
TPD(Weighted)
0.1250
0.0561
*
0.0561
*
0.3098
0.1678
*
0.1453
*The weighted estimator has the smallest AE and IE values forr=18 and 10 largest losses, and its IE value is almost equal to the smallest IE value for r=49, all largest losses. We statistically conclude that the weighted estimated distribution is the best fit in the tail of the hurricane loss data.


### 6.2. Forest Fire Loss Data Example

Next we look at the forest fire loss example in Section1.1.2. The data in Table 1 contains a relatively substantial number of large forest fire losses which convince us to use a truncated Pareto model and compare the four estimators in (26), (27), (28),and (29).We use the 25 largest losses in this study.Figure6 is a log-log plot which shows the upper tail for the forest fire loss data. The circles represent the real data, and the straight line represents the estimated original Pareto distribution. The dashed line, dotted line, and thick solid line represent the estimated truncated Pareto distribution by using Aban’s, Moment, and the weighted estimators, respectively. We can see that the estimated truncated Pareto distributions fit the data very well using all three estimation methods and are much better than the original Pareto distribution. Around the tail, the weighted and the Moment estimation methods perform the best. The fact that the tail of the data curves downward in Figure 6 is the evidence in support of using a truncated Pareto model.Figure 6
Log-log plot for forest fire example.The results of these three estimators are listed in Table5 by using the forest fire loss data, where γ^=X(1), ν^=X(n), and n=25.Table 5
The estimation of index, mean, median, and VaR for forest fire loss data.
Method
α
^
μ
^
Median
5
% value at risk
Pareto(Hill)
4.3044
1.096 billion
1.09 billion
1.855 billion
TPD(Aban)
0.7964
1.167 billion
1.15 billion
1.446 billion
TPD(Moment)
0.8260
1.166 billion
1.15 billion
1.446 billion
TPD(Weighted)
0.8642
1.165 billion
1.15 billion
1.446 billionNote that the weighted estimateα^w is the largest among the three truncated Pareto estimates. The 5% value at risk of the original Pareto estimate is the largest. The 5% VaR may be overestimated compared to the truncated Pareto models.Similarly, as in the hurricane example, we compute the AE in(30)and IE in (36)again for fitting the 25 losses of the forest fire loss data.Figure7 gives the absolute errors (AE) in (30) for the 10 largest losses and confirms that the original Pareto estimate has relatively larger errors and the weighted estimate has relatively smaller errors in the tail. Figure 7 explains the data fitting of the tail of the distributions in Figure 6.Figure 7
Absolute errors|F*(x)-Sn(x)| for forest fire loss example, the 10 largest losses.We also compute the AE in(30)and IE in(36) to confirm those tail errors by using r as the number of the largest losses, 1≤r≤n. The IE values are given in Table 6 (the smallest values are bold with *).Table 6
Errors of goodness-of-fit testsn=25 for forest fire loss data.
Method
Goodness-of-fit tests
Absolute error (AE)
Integrated error (IE)
r
=
25
r
=
10
r
=
5
r
=
25
r
=
10
r
=
5
Pareto(Hill)
0.2656
0.1288
0.1288
2.7125
1.1493
1.0985
TPD(Aban)
0.1503
*
0.1195
0.1195
1.3687
*
0. 7911
0.5430
TPD(Moment)
0.1520
0.0982
0.0982
1.3736
0. 7764
0.5430
TPD(Weighted)
0.1542
0.0964
*
0.0964
*
1.3796
0.7577
*
0.5203
*The weighted estimator has the smallest AE and IE values forr=10 and 5, and its IE value in r=25 is almost equal to the smallest IE value. We statistically conclude that the weighted estimated distribution is the best fit to the tail of the forest fire loss data.


## 6.1. Hurricane Loss Example

### 6.1.1. Comparison of Four Estimation Methods

At first, We look at the hurricane loss example in Section1.1.1. Based on the simulation results in Figure 3, we consider the three better estimators out of the five in the heavy-tailed case, that is, Aban’s, Moment, and the weighted estimators in (27), (28), and (29) for the truncated Pareto p.d.f.f(x) in (3)and the c.d.f.F(x) in (4).We also compare them with Hill’s estimator in (26)for the original Pareto p.d.f. fp(x)  in(1)and the c.d.f. Fp(x) in (2). Here
(26)f^Hill(x)=α^Hillγα^Hillxα^w,F^Hill(x)=1-(γx)α^Hill,
where α^Hill is given in (12) with r=n-1; consider
(27)f^MLE(x)=α^MLEγα^MLEx-α^MLE-11-(γ/ν)α^MLE,F^MLE(x)=1-γα^MLE(x-α^MLE-ν-α^MLE)1-(γ/ν)α^MLE,
where  α^MLE  is Aban’s MLE given in (14); consider
(28)f^M(x)=α^Mγα^Mx-α^M-11-(γ/ν)α^M,F^M(x)=1-γα^M(x-α^M-ν-α^M)1-(γ/ν)α^M,
where  α^Mis given in (16); consider
(29)f^w(x)=α^wγα^wx-α^w-11-(γ/ν)α^w,F^w(x)=1-γα^w(x-α^w-ν-α^w)1-(γ/ν)α^w,
where α^w is given in (18), using the weight wL1-opt=1/n(n-1).The results of these four methods are listed in Table3 by using the hurricane loss data, where γ^=X(min), ν^=X(max), and n=49.Table 3
Estimation of index, mean, median, and VaR for hurricane loss data.
Method
α
^
μ
^
Median
5
% value at risk
Pareto(Hill)
0.8126
∞
8.68 billion
147.68 billion
TPD(Aban)
0.6206
21.74 billion
9.73 billion
85.15 billion
TPD(Moment)
0.6476
20.48 billion
9.47 billion
82.55 billion
TPD(Weighted)
0.6990
19.37 billion
9.02 billion
77.58 billionFigure4 is a log-log plot showing the upper tail for the hurricane loss data. In this plot, the circles represent the real data, and the straight line represents the estimated original Pareto distribution. The dashed line, dotted line, and thick solid line represent the estimated truncated Pareto distributions by using Aban's, Moment, and Huang's estimators, respectively. We observe two conclusions intuitively.(1)
The original Pareto distribution (straight line) does not fit the data well in the tail. The truncated Pareto distribution fits the data very well using all three estimation methods (Aban, Moment, and weighted). Note that the 5% value at risk estimated by the original Pareto model is 147 billion; the three truncated model estimates are about 80 billion. It appears that the original Pareto model overestimates the risk. An insurance company would set a high premium if it is using the Pareto model, with the result that many people cannot afford to buy insurance.(2)
We examine the three truncated Pareto estimates. Around the tail, the weighted estimate fits the data the best; that is, theln(P{X>x}) curve turns downward more quickly following the trend of the data pattern because the weighted estimate α^w is largest among the three truncated model estimates (this is consistent with the simulation results in Figure 4), and the weighted mean estimate μ^w and the 5% value at risk are the smallest among the three methods (this is obtained by placing less weight on the extreme value, e.g., the 1926 great miami hurricane's loss), so its estimated distribution is less heavy in the tail compared with other methods.Figure 4
Log-log plot of the hurricane loss data.Next, in order to confirm these conclusions, we run three goodness of-fit tests. Later, we define the absolute error and integrated error as the measures of the distance from the empirical data points to the estimated Pareto curve and truncated Pareto curves.


### 6.1.2. Goodness-of-Fit Tests

Our objective is to test if the estimated distributions in(26)–(29)fit the data properly. We test the hypotheses H0:F(x)=F*(x) against H1:F(x)≠F*(x), where F(x) is the true unknown distribution function and F*(x) is the estimated Pareto c.d.f. in(26)or the estimated truncated Pareto c.d.f. in (27)–(29). In this paper we use three EDF goodness-of-fit tests.(1) The Kolmogorov-Smirnov (K-S) test [15]: the test statistic is given by
(30)T=supx|F*(x)-Sn(x)|,-∞<x<∞,
where Sn(x) is the EDF in (8). Under H0, the two-tailed P value for the K-S test is as follows:
(31)P-value=1-[1-T∑j=0[n(1-T)]‍n!j!(n-j)!(1-T-jn)n-j(t+jn)j-1]2,
where [n(1-T)] is the integer part of n(1-T).(2) Anderson and Darling [16] test (A-D test) introduced a measure of “distance” between the empirical distribution Sn(x) and the proposed c.d.f. F*(x) by using a metric function space
(32)Wn2=n∫-∞∞[Sn(x)-F*(x)]2ψ(F*(x))dF,
where ψ(u)=1/(u/(1-u)) is a weight function, with u=F(x). Let ui=F(X(i)),i=1,…,n, and under H0 the test statistic and P-value are given by
(33)Wn2=-n-1n∑j=1n‍(2j-1)(log(uj)+log(1-un-j+1)),P-value=1-2z∑j=0∞‍{(-1)jΓ(j+1/2)(4j+1)j!e-(4j+1)2π2/(8z)=1-2z∑j=0∞‍×∫0∞e((z/8)/(w2+1)-(4j+1)2π2w2/(8z))dw},
where z is the observed value of Wn2 and Γ(x)=∫0∞tx-1e-tdt,  x>0, is the Gamma function.(3) Cramer-von Mises test [16] proposed (C-v-M test) using ψ(u)=1 in (32); thus, under H0, the test statistic and P-value are given by
(34)nω2=112n+∑j=1n‍(uj-2j-12n)2,P-value=1-1πz∑j=0∞‍{(-1)2jΓ(j+1/2)(4j+1)1/2j!πe-(4j+1)2/(16z)P-value=1-1πz∑j=0∞‍1-×K1/4((4j+1)216z)},
where K1/4(x) is the modified Bessel function of the second kind
(35)hhhhhKα(x)=π2I-α(x)-Iα(x)sin(απ),hhhhhIα(x)=∑m=0∞‍1m!Γ(m+α+1)(x2)2m+α.For fitting the 49 losses in the hurricane loss data, for example, after computing estimates ofF*(x) by using (26), (27), (28),and(29), we compute theabsolute error(AE)in (30) and define the integrated error (IE) by
(36)IE=[∫X(n-r+1)X(n)‍(Sn(x)-F*(x))2dx]1/2.Figure5 gives the absolute errors (AE) in (30) for the 10 largest losses and confirms that the original Pareto estimate has relatively larger errors and the weighted estimate has relatively smaller errors in the tail. Figure 5 explains the data fitting of the tail of the distributions in Figure 4.Figure 5
Absolute errors|F*(x)-Sn(x)| for hurricane loss example on the 10 largest losses data.We also compute the AE in(30)and IE in(36) to confirm those tail errors by using r as the number of the largest losses, 1≤r≤n. The AE and IE values are given in Table 4 (the smallest values are bold with *).Table 4
Errors of goodness-of-fit testsn=49 for hurricane example.
Method
Goodness-of-fit tests
Absolute error (AE)
Integrated error (IE)
r
=
49
r
=
18
r
=
10
r
=
49
r
=
18
r
=
10
Pareto(Hill)
0.1340
0.0584
0.0584
0.4844
0.3818
0.3723
TPD(Aban)
0.0948
*
0.0839
0.0832
0.3114
0.2565
0.2161
TPD(Moment)
0.1053
0.0738
0.0737
0.2985
*
0.2171
0.1825
TPD(Weighted)
0.1250
0.0561
*
0.0561
*
0.3098
0.1678
*
0.1453
*The weighted estimator has the smallest AE and IE values forr=18 and 10 largest losses, and its IE value is almost equal to the smallest IE value for r=49, all largest losses. We statistically conclude that the weighted estimated distribution is the best fit in the tail of the hurricane loss data.


## 6.1.1. Comparison of Four Estimation Methods

At first, We look at the hurricane loss example in Section1.1.1. Based on the simulation results in Figure 3, we consider the three better estimators out of the five in the heavy-tailed case, that is, Aban’s, Moment, and the weighted estimators in (27), (28), and (29) for the truncated Pareto p.d.f.f(x) in (3)and the c.d.f.F(x) in (4).We also compare them with Hill’s estimator in (26)for the original Pareto p.d.f. fp(x)  in(1)and the c.d.f. Fp(x) in (2). Here
(26)f^Hill(x)=α^Hillγα^Hillxα^w,F^Hill(x)=1-(γx)α^Hill,
where α^Hill is given in (12) with r=n-1; consider
(27)f^MLE(x)=α^MLEγα^MLEx-α^MLE-11-(γ/ν)α^MLE,F^MLE(x)=1-γα^MLE(x-α^MLE-ν-α^MLE)1-(γ/ν)α^MLE,
where  α^MLE  is Aban’s MLE given in (14); consider
(28)f^M(x)=α^Mγα^Mx-α^M-11-(γ/ν)α^M,F^M(x)=1-γα^M(x-α^M-ν-α^M)1-(γ/ν)α^M,
where  α^Mis given in (16); consider
(29)f^w(x)=α^wγα^wx-α^w-11-(γ/ν)α^w,F^w(x)=1-γα^w(x-α^w-ν-α^w)1-(γ/ν)α^w,
where α^w is given in (18), using the weight wL1-opt=1/n(n-1).The results of these four methods are listed in Table3 by using the hurricane loss data, where γ^=X(min), ν^=X(max), and n=49.Table 3
Estimation of index, mean, median, and VaR for hurricane loss data.
Method
α
^
μ
^
Median
5
% value at risk
Pareto(Hill)
0.8126
∞
8.68 billion
147.68 billion
TPD(Aban)
0.6206
21.74 billion
9.73 billion
85.15 billion
TPD(Moment)
0.6476
20.48 billion
9.47 billion
82.55 billion
TPD(Weighted)
0.6990
19.37 billion
9.02 billion
77.58 billionFigure4 is a log-log plot showing the upper tail for the hurricane loss data. In this plot, the circles represent the real data, and the straight line represents the estimated original Pareto distribution. The dashed line, dotted line, and thick solid line represent the estimated truncated Pareto distributions by using Aban's, Moment, and Huang's estimators, respectively. We observe two conclusions intuitively.(1)
The original Pareto distribution (straight line) does not fit the data well in the tail. The truncated Pareto distribution fits the data very well using all three estimation methods (Aban, Moment, and weighted). Note that the 5% value at risk estimated by the original Pareto model is 147 billion; the three truncated model estimates are about 80 billion. It appears that the original Pareto model overestimates the risk. An insurance company would set a high premium if it is using the Pareto model, with the result that many people cannot afford to buy insurance.(2)
We examine the three truncated Pareto estimates. Around the tail, the weighted estimate fits the data the best; that is, theln(P{X>x}) curve turns downward more quickly following the trend of the data pattern because the weighted estimate α^w is largest among the three truncated model estimates (this is consistent with the simulation results in Figure 4), and the weighted mean estimate μ^w and the 5% value at risk are the smallest among the three methods (this is obtained by placing less weight on the extreme value, e.g., the 1926 great miami hurricane's loss), so its estimated distribution is less heavy in the tail compared with other methods.Figure 4
Log-log plot of the hurricane loss data.Next, in order to confirm these conclusions, we run three goodness of-fit tests. Later, we define the absolute error and integrated error as the measures of the distance from the empirical data points to the estimated Pareto curve and truncated Pareto curves.


## 6.1.2. Goodness-of-Fit Tests

Our objective is to test if the estimated distributions in(26)–(29)fit the data properly. We test the hypotheses H0:F(x)=F*(x) against H1:F(x)≠F*(x), where F(x) is the true unknown distribution function and F*(x) is the estimated Pareto c.d.f. in(26)or the estimated truncated Pareto c.d.f. in (27)–(29). In this paper we use three EDF goodness-of-fit tests.(1) The Kolmogorov-Smirnov (K-S) test [15]: the test statistic is given by
(30)T=supx|F*(x)-Sn(x)|,-∞<x<∞,
where Sn(x) is the EDF in (8). Under H0, the two-tailed P value for the K-S test is as follows:
(31)P-value=1-[1-T∑j=0[n(1-T)]‍n!j!(n-j)!(1-T-jn)n-j(t+jn)j-1]2,
where [n(1-T)] is the integer part of n(1-T).(2) Anderson and Darling [16] test (A-D test) introduced a measure of “distance” between the empirical distribution Sn(x) and the proposed c.d.f. F*(x) by using a metric function space
(32)Wn2=n∫-∞∞[Sn(x)-F*(x)]2ψ(F*(x))dF,
where ψ(u)=1/(u/(1-u)) is a weight function, with u=F(x). Let ui=F(X(i)),i=1,…,n, and under H0 the test statistic and P-value are given by
(33)Wn2=-n-1n∑j=1n‍(2j-1)(log(uj)+log(1-un-j+1)),P-value=1-2z∑j=0∞‍{(-1)jΓ(j+1/2)(4j+1)j!e-(4j+1)2π2/(8z)=1-2z∑j=0∞‍×∫0∞e((z/8)/(w2+1)-(4j+1)2π2w2/(8z))dw},
where z is the observed value of Wn2 and Γ(x)=∫0∞tx-1e-tdt,  x>0, is the Gamma function.(3) Cramer-von Mises test [16] proposed (C-v-M test) using ψ(u)=1 in (32); thus, under H0, the test statistic and P-value are given by
(34)nω2=112n+∑j=1n‍(uj-2j-12n)2,P-value=1-1πz∑j=0∞‍{(-1)2jΓ(j+1/2)(4j+1)1/2j!πe-(4j+1)2/(16z)P-value=1-1πz∑j=0∞‍1-×K1/4((4j+1)216z)},
where K1/4(x) is the modified Bessel function of the second kind
(35)hhhhhKα(x)=π2I-α(x)-Iα(x)sin(απ),hhhhhIα(x)=∑m=0∞‍1m!Γ(m+α+1)(x2)2m+α.For fitting the 49 losses in the hurricane loss data, for example, after computing estimates ofF*(x) by using (26), (27), (28),and(29), we compute theabsolute error(AE)in (30) and define the integrated error (IE) by
(36)IE=[∫X(n-r+1)X(n)‍(Sn(x)-F*(x))2dx]1/2.Figure5 gives the absolute errors (AE) in (30) for the 10 largest losses and confirms that the original Pareto estimate has relatively larger errors and the weighted estimate has relatively smaller errors in the tail. Figure 5 explains the data fitting of the tail of the distributions in Figure 4.Figure 5
Absolute errors|F*(x)-Sn(x)| for hurricane loss example on the 10 largest losses data.We also compute the AE in(30)and IE in(36) to confirm those tail errors by using r as the number of the largest losses, 1≤r≤n. The AE and IE values are given in Table 4 (the smallest values are bold with *).Table 4
Errors of goodness-of-fit testsn=49 for hurricane example.
Method
Goodness-of-fit tests
Absolute error (AE)
Integrated error (IE)
r
=
49
r
=
18
r
=
10
r
=
49
r
=
18
r
=
10
Pareto(Hill)
0.1340
0.0584
0.0584
0.4844
0.3818
0.3723
TPD(Aban)
0.0948
*
0.0839
0.0832
0.3114
0.2565
0.2161
TPD(Moment)
0.1053
0.0738
0.0737
0.2985
*
0.2171
0.1825
TPD(Weighted)
0.1250
0.0561
*
0.0561
*
0.3098
0.1678
*
0.1453
*The weighted estimator has the smallest AE and IE values forr=18 and 10 largest losses, and its IE value is almost equal to the smallest IE value for r=49, all largest losses. We statistically conclude that the weighted estimated distribution is the best fit in the tail of the hurricane loss data.


## 6.2. Forest Fire Loss Data Example

Next we look at the forest fire loss example in Section1.1.2. The data in Table 1 contains a relatively substantial number of large forest fire losses which convince us to use a truncated Pareto model and compare the four estimators in (26), (27), (28),and (29).We use the 25 largest losses in this study.Figure6 is a log-log plot which shows the upper tail for the forest fire loss data. The circles represent the real data, and the straight line represents the estimated original Pareto distribution. The dashed line, dotted line, and thick solid line represent the estimated truncated Pareto distribution by using Aban’s, Moment, and the weighted estimators, respectively. We can see that the estimated truncated Pareto distributions fit the data very well using all three estimation methods and are much better than the original Pareto distribution. Around the tail, the weighted and the Moment estimation methods perform the best. The fact that the tail of the data curves downward in Figure 6 is the evidence in support of using a truncated Pareto model.Figure 6
Log-log plot for forest fire example.The results of these three estimators are listed in Table5 by using the forest fire loss data, where γ^=X(1), ν^=X(n), and n=25.Table 5
The estimation of index, mean, median, and VaR for forest fire loss data.
Method
α
^
μ
^
Median
5
% value at risk
Pareto(Hill)
4.3044
1.096 billion
1.09 billion
1.855 billion
TPD(Aban)
0.7964
1.167 billion
1.15 billion
1.446 billion
TPD(Moment)
0.8260
1.166 billion
1.15 billion
1.446 billion
TPD(Weighted)
0.8642
1.165 billion
1.15 billion
1.446 billionNote that the weighted estimateα^w is the largest among the three truncated Pareto estimates. The 5% value at risk of the original Pareto estimate is the largest. The 5% VaR may be overestimated compared to the truncated Pareto models.Similarly, as in the hurricane example, we compute the AE in(30)and IE in (36)again for fitting the 25 losses of the forest fire loss data.Figure7 gives the absolute errors (AE) in (30) for the 10 largest losses and confirms that the original Pareto estimate has relatively larger errors and the weighted estimate has relatively smaller errors in the tail. Figure 7 explains the data fitting of the tail of the distributions in Figure 6.Figure 7
Absolute errors|F*(x)-Sn(x)| for forest fire loss example, the 10 largest losses.We also compute the AE in(30)and IE in(36) to confirm those tail errors by using r as the number of the largest losses, 1≤r≤n. The IE values are given in Table 6 (the smallest values are bold with *).Table 6
Errors of goodness-of-fit testsn=25 for forest fire loss data.
Method
Goodness-of-fit tests
Absolute error (AE)
Integrated error (IE)
r
=
25
r
=
10
r
=
5
r
=
25
r
=
10
r
=
5
Pareto(Hill)
0.2656
0.1288
0.1288
2.7125
1.1493
1.0985
TPD(Aban)
0.1503
*
0.1195
0.1195
1.3687
*
0. 7911
0.5430
TPD(Moment)
0.1520
0.0982
0.0982
1.3736
0. 7764
0.5430
TPD(Weighted)
0.1542
0.0964
*
0.0964
*
1.3796
0.7577
*
0.5203
*The weighted estimator has the smallest AE and IE values forr=10 and 5, and its IE value in r=25 is almost equal to the smallest IE value. We statistically conclude that the weighted estimated distribution is the best fit to the tail of the forest fire loss data.


## 7. Conclusions

In the complicated real world, it is difficult to construct a model combining all the desired features. In general, the final model selection depends on the best fitting model. The criteria are based on goodness-of-fit tests, existence of the moments, characteristic largest values, and log-log plots. The hurricane loss data and forest fire loss data are well fitted by the truncated Pareto distribution. In summary,(a)
we recommend that the truncated Pareto model is appropriate as a loss distribution to be used when analyzing huge risk loss data sets. The upper and lower limits can be set by the largest and smallest losses or other reasonable values;(b)
the estimated loss distribution provides a prediction of the next disaster’s 5% value at risk. The largest loss in the data set plays an important role in the prediction. In Figures4 and 6, the trend of the tail distribution is crucial for an insurance company setting policy, also for inhabitants and government to make plans to minimize damage from natural disasters;(c)
the semiparametric methods (Moment and weighted) are robust, easy to use, more stable, and fit the data better than the MLE methods. In both of the foregoing two examples, the estimated shape parameterα^ is less than 1. The original Pareto model is not able to use the Moment or the weighted methods. This is another advantage of using the truncated Pareto model;(d)
the statistical inference in these examples shows that the estimated distribution curve by using proposed weighted estimation method fits the tails of data better, relative to the Moment and MLE estimators. This is due to the fact that the weighted estimator gives less weight on the extreme values and has good MSE. Based on these studies, we suggest that further studies on the usage of weights may be useful.



---

*Source: 829131-2013-11-17.xml*