# Detection of Heterogeneous Structures on the Gaussian Copula Model Using Projective Power Entropy

Authors: Akifumi Notsu; Yoshinori Kawasaki; Shinto Eguchi
Journal: ISRN Probability and Statistics
 (2013)

---

## Abstract

We discuss a parameter estimation problem for a Gaussian copula model under misspecification. Conventional estimators such as the maximum likelihood estimator (MLE) do not work well if the model is misspecified. We propose the estimator that minimizes the projective power entropy. We call it theγ-estimator, where γ denotes the power index. A feasible form of the projective power entropy is given that suites the Gaussian copula model. It is shown that the γ-estimator is robust against outliers. In addition the γ-estimator can appropriately detect a heterogeneous structure of the underlying distribution, even if the underlying distribution consists of some different copula components while a single Gaussian copula is used as a statistical model. We explore such an ability of the γ-estimator to detect the local structures in the comparison with the MLE. We also propose a fixed point algorithm to obtain the γ-estimator. The usefulness of the proposed methodology is demonstrated in numerical experiments.


---

## Body

## 1. Introduction

Applications of copula models have been increasing in number in recent years. There are a variety of applications on finance, risk management [1] and multivariate time series analysis [2]. With copula models, the specification of the marginal distributions is parameterized separately from the dependence structure of the joint distribution. Hence, it gives a convenient way of the construction of flexible and more general multivariate distributions. As far as we know, there exist only a few works that are tackled with the identification and the statistical estimation of the mixture of copula models and most of them rely on MCMC algorithm. In this paper we focus on a misspecified Gaussian copula model. In other words, a sample follows a distribution mixed with different sources but a statistical model we fit is just a single Gaussian copula. It is very hard to construct multivariate copulas for three or more random variables [3], while the Gaussian is an exception. So we start with the Gaussian copula model, but later in Section 5 we will show that our method is closely related to t-copula. As an example of misspecification, we consider that the underlying distribution is
(1)τcG(u;P1)+(1-τ)cG(u;P2),
where τ is a mixing proportion and cG(u;P) denotes the probability density function of the Gaussian copula with the correlation matrix parameter P. We see that the MLE for P almost surely converges to τP1+(1-τ)P2 under the assumption (1), which means that the MLE fails to detect the structure of the underlying distribution.We make use of theγ-estimator [4, 5] that can be obtained via minimization of the projective power entropy. Here γ denotes the power index, and if γ→0, the γ-estimator reduces to the MLE. So the γ-estimator can be regarded as an extension of the MLE. In [5], the robustness of the γ-estimator was investigated in a general setting of parametric model. In [6], the minimum density power divergence estimator was proposed, which also uses power of density, for the covariance matrix of multivariate time series, and the robustness was shown. Our research shows that even if a single Gaussian copula model is incorrectly fitted to the data from the mixture distribution (1), the γ-estimator can detect both P1 and P2 separately if P1 and P2 are “distinct” enough and τ is close to 0.5.Theγ-estimation for the Gaussian copula model relies on the projective power cross entropy between the underlying distribution and the Gaussian copula model cG(u;P). The projective power cross entropy, which is a function of P, has only one local minimum or some local minima depending on the underlying distribution. We show that if P1 and P2 are “distinct” enough and τ is near 0.5, then the projective power cross entropy between the underlying mixture distribution (1) and the Gaussian copula cG(u,P) has two local minimizers near P1 and P2, respectively, so we propose to use these local minimizers to detect P1 and P2.This paper is organized as follows. Theγ-estimator and the MLE for the Gaussian copula model and a fixed point algorithm to obtain the γ-estimator and the MLE are given in Section 2. Section 3 states the relationship between the projective power entropy and the γ-estimator. We introduce an appropriate measure for the Gaussian copula model since the projective power entropy is defined with respect to some carrier measure. Section 4 reveals the property of the γ-estimator to detect heterogeneous structures. Section 5 elucidates the relationship between maximum entropy distributions and the γ-estimation. The robustness of the γ-estimator is discussed based on its influence function in Section 6. A simulation study is given in Section 7, and discussions are given in the last section. The proofs for all the theoretical results are provided in the appendix.


## 2. Estimation of the Gaussian Copula Model

In Section2.1, the γ-estimator for the Gaussian copula model is discussed, and in the followed subsection the MLE for the Gaussian copula model is given. The last subsection lays out a fixed point algorithm to obtain the γ-estimator and the MLE.


### 2.1. Theγ-Estimator for the Gaussian Copula Model

The density function of the Gaussian copula is given by(2)cG(u;P)=detP-1/2exp(-12xG(u)⊤(P-1-Im)xG(u)),u∈[0,1]m,
where xG(u)=(Φ-1(u1),…,Φ-1(um))⊤, Φ(x) denotes the cumulative distribution function of the standard normal distribution, P is a correlation matrix, and Im is the identity matrix of size m. Let v(P) be the m(m-1)/2-dimensional vector which consists of the column-wise stacked lower diagonal elements of P. For example, v(P)=(p21,p31,p32)⊤ if m=3. The set
(3){v(P):Pisacorrelationmatrixofsizem}
is a parameter space of the Gaussian copula model.Letu(1),…,u(n) be a random sample from a copula with the probability density function c(u) while cG(u;P) is our statistical model. The loss function associated with the projective power entropy introduced in Section 3.1 is given by
(4)Lγ(P)=-det(P)-γ/2(1+γ)1n∑i=1nexp(-γ2x(i)⊤P-1x(i)),
up to constant, where x(i)=xG(u(i)) for i=1,…,n. The γ-estimator is proposed as the set of local minimizers of Lγ(P) and interpreted as follows. If Lγ(P) has a local minimum, the underlying distribution is estimated by cG(u;P^γ) using the minimizer P^γ. If Lγ(P) has ℓ local minima (ℓ≥2), the underlying distribution is estimated by a mixture of ℓ Gaussian copulas. Each Gaussian copula’s parameter is estimated by the corresponding local minimizer.


### 2.2. The MLE for the Gaussian Copula Model

We consider the MLE for the Gaussian copula model on the same setting as in Section2.1. The log-likelihood multiplied by -1/n is given by
(5)L0(P)=-12logdet(P)-1+12n∑i=1nx(i)⊤(P-1-Im)x(i).
It is easy to see that L0(P) and Lγ(P) satisfy
(6)limγ→0Lγ(P)+1γ=L0(P),
up to constant, so the MLE will be deemed to be the 0-estimator in terms of the γ-estimator. Generally the γ-estimator can be regarded as an extension of the MLE. It is well known that the MLE does not work well under model misspecification. For example, in the case of (1) the MLE for the Gaussian copula model almost surely converges to τP1+(1-τ)P2, but we cannot detect neither P1 nor P2. If τ=0.5 and
(7)P1=(10.90.91),P2=(1-0.9-0.91),
then τP1+(1-τ)P2 is equal to the identity matrix, which has no meaning in this situation. We cannot use the MLE in the case of misspecification.


### 2.3. A Fixed Point Algorithm to Obtain theγ-Estimator for the Gaussian Copula Model

We give a fixed point algorithm to obtain theγ-estimator for the Gaussian copula model using the Lagrange-multiplier method. The appendix provides the details of the derivation of the algorithm. We can still make use of this algorithm to obtain the MLE just by setting γ=0.Algorithm(1) Set an appropriate correlation matrixP0.(2) GivenPt, calculate Pt+1 by the following update formula:
(8)Pt+1=Σt+Ptdiag((Pt⊙Pt)-1Diag(Im-Σt))Pt,
where ⊙ denotes the Hadamard product. Σt is defined by
(9)Σt=(1+γ)∑i=1n‍wγ(x(i),Pt)x(i)⊤x(i),
where
(10)wγ(x,P)=exp(-(γ/2)x⊤P-1x)∑j=1nexp(-(γ/2)x(j)⊤P-1x(j)).
Here Diag (M) for a square matrix M denotes the column vector which consists of the diagonal elements of M and diag(a) for a vector a denotes the diagonal matrix whose diagonal elements are the components of a.(3) For sufficient small given numberε, repeat procedure 2 while
(11)∥Pt+1-Pt∥>ε,
where ∥A∥ for a square matrix A denotes the matrix norm defined by tr(A⊤A).(4) For all local minimizers, repeat procedures 1–3 for different initial valuesP0.If we consider the estimation problem on Gaussian distributions with mean0, the update formula for an iteration algorithm to obtain the γ-estimator of the covariance matrix Σ is given by
(12)Σt+1=(1+γ)∑i=1nwγ(x(i),Σt)x(i)⊤x(i).
See [5] for details. If we consider the optimization problem with the objective function Lγ(P) without the constraint that the diagonal elements of P are 1, the same iteration algorithm (12) can be deduced. So the second term of the right hand side of (8) appears because of the existence of the constraint.We make a remark on the algorithm to obtain the MLE, orγ-estimator with γ=0. We find rather complicated solution of the MLE if we consider a simpler case of m=2. In [1], an approximate MLE for the Gaussian copula model is shown because it takes quite a while to solve the constrained optimization problem in order to obtain the MLE in high dimensions. The approximate MLE is given by
(13)diag(S)-1/2Sdiag(S)-1/2,
where diag(S) is the diagonal matrix whose diagonal elements are equal to those of S. We can easily consider an iteration algorithm to obtain an approximate γ-estimator to combine (12) and (13). The update formula of the algorithm is given by
(14)Pt+1*=diag(Σt+1*)-1/2Σt+1*diag(Σt+1*)-1/2,
where
(15)Σt+1*=(1+γ)∑i=1n‍w(x(i),Pt*)x(i)⊤x(i).
If n is infinity, Pt and Pt* converge to the same correlation matrix when t tends to ∞. However Pt and Pt* are different in general. Pt is preferred to Pt* in terms of accuracy.


## 2.1. Theγ-Estimator for the Gaussian Copula Model

The density function of the Gaussian copula is given by(2)cG(u;P)=detP-1/2exp(-12xG(u)⊤(P-1-Im)xG(u)),u∈[0,1]m,
where xG(u)=(Φ-1(u1),…,Φ-1(um))⊤, Φ(x) denotes the cumulative distribution function of the standard normal distribution, P is a correlation matrix, and Im is the identity matrix of size m. Let v(P) be the m(m-1)/2-dimensional vector which consists of the column-wise stacked lower diagonal elements of P. For example, v(P)=(p21,p31,p32)⊤ if m=3. The set
(3){v(P):Pisacorrelationmatrixofsizem}
is a parameter space of the Gaussian copula model.Letu(1),…,u(n) be a random sample from a copula with the probability density function c(u) while cG(u;P) is our statistical model. The loss function associated with the projective power entropy introduced in Section 3.1 is given by
(4)Lγ(P)=-det(P)-γ/2(1+γ)1n∑i=1nexp(-γ2x(i)⊤P-1x(i)),
up to constant, where x(i)=xG(u(i)) for i=1,…,n. The γ-estimator is proposed as the set of local minimizers of Lγ(P) and interpreted as follows. If Lγ(P) has a local minimum, the underlying distribution is estimated by cG(u;P^γ) using the minimizer P^γ. If Lγ(P) has ℓ local minima (ℓ≥2), the underlying distribution is estimated by a mixture of ℓ Gaussian copulas. Each Gaussian copula’s parameter is estimated by the corresponding local minimizer.


## 2.2. The MLE for the Gaussian Copula Model

We consider the MLE for the Gaussian copula model on the same setting as in Section2.1. The log-likelihood multiplied by -1/n is given by
(5)L0(P)=-12logdet(P)-1+12n∑i=1nx(i)⊤(P-1-Im)x(i).
It is easy to see that L0(P) and Lγ(P) satisfy
(6)limγ→0Lγ(P)+1γ=L0(P),
up to constant, so the MLE will be deemed to be the 0-estimator in terms of the γ-estimator. Generally the γ-estimator can be regarded as an extension of the MLE. It is well known that the MLE does not work well under model misspecification. For example, in the case of (1) the MLE for the Gaussian copula model almost surely converges to τP1+(1-τ)P2, but we cannot detect neither P1 nor P2. If τ=0.5 and
(7)P1=(10.90.91),P2=(1-0.9-0.91),
then τP1+(1-τ)P2 is equal to the identity matrix, which has no meaning in this situation. We cannot use the MLE in the case of misspecification.


## 2.3. A Fixed Point Algorithm to Obtain theγ-Estimator for the Gaussian Copula Model

We give a fixed point algorithm to obtain theγ-estimator for the Gaussian copula model using the Lagrange-multiplier method. The appendix provides the details of the derivation of the algorithm. We can still make use of this algorithm to obtain the MLE just by setting γ=0.Algorithm(1) Set an appropriate correlation matrixP0.(2) GivenPt, calculate Pt+1 by the following update formula:
(8)Pt+1=Σt+Ptdiag((Pt⊙Pt)-1Diag(Im-Σt))Pt,
where ⊙ denotes the Hadamard product. Σt is defined by
(9)Σt=(1+γ)∑i=1n‍wγ(x(i),Pt)x(i)⊤x(i),
where
(10)wγ(x,P)=exp(-(γ/2)x⊤P-1x)∑j=1nexp(-(γ/2)x(j)⊤P-1x(j)).
Here Diag (M) for a square matrix M denotes the column vector which consists of the diagonal elements of M and diag(a) for a vector a denotes the diagonal matrix whose diagonal elements are the components of a.(3) For sufficient small given numberε, repeat procedure 2 while
(11)∥Pt+1-Pt∥>ε,
where ∥A∥ for a square matrix A denotes the matrix norm defined by tr(A⊤A).(4) For all local minimizers, repeat procedures 1–3 for different initial valuesP0.If we consider the estimation problem on Gaussian distributions with mean0, the update formula for an iteration algorithm to obtain the γ-estimator of the covariance matrix Σ is given by
(12)Σt+1=(1+γ)∑i=1nwγ(x(i),Σt)x(i)⊤x(i).
See [5] for details. If we consider the optimization problem with the objective function Lγ(P) without the constraint that the diagonal elements of P are 1, the same iteration algorithm (12) can be deduced. So the second term of the right hand side of (8) appears because of the existence of the constraint.We make a remark on the algorithm to obtain the MLE, orγ-estimator with γ=0. We find rather complicated solution of the MLE if we consider a simpler case of m=2. In [1], an approximate MLE for the Gaussian copula model is shown because it takes quite a while to solve the constrained optimization problem in order to obtain the MLE in high dimensions. The approximate MLE is given by
(13)diag(S)-1/2Sdiag(S)-1/2,
where diag(S) is the diagonal matrix whose diagonal elements are equal to those of S. We can easily consider an iteration algorithm to obtain an approximate γ-estimator to combine (12) and (13). The update formula of the algorithm is given by
(14)Pt+1*=diag(Σt+1*)-1/2Σt+1*diag(Σt+1*)-1/2,
where
(15)Σt+1*=(1+γ)∑i=1n‍w(x(i),Pt*)x(i)⊤x(i).
If n is infinity, Pt and Pt* converge to the same correlation matrix when t tends to ∞. However Pt and Pt* are different in general. Pt is preferred to Pt* in terms of accuracy.


## 3. Projective Power Entropy andγ-Estimator

In Section3.1 the projective power entropy and the γ-estimator are given. In the next subsection we discuss an appropriate measure for the Gaussian copula model.


### 3.1. Projective Power Entropy and theγ-Estimator

The projective power entropy off(x) with the index γ and the measure Q is defined as
(16)Hγ(f∣Q)=-[∫f(x)1+γQ(dx)]1/(1+γ).
If Q is the Lebesgue measure denoted by QL and f(x) is a probability density function, then we have
(17)limγ→0Hγ(f∣QL)+1γ=H0(f∣QL),
where H0(f∣QL) is defined by -∫f(x)logf(x)dx, which is equivalent to the Boltzmann-Shannon entropy. The projective power cross entropy between g(x) and f(x) with the index γ and the measure Q is defined as
(18)Cγ(g,f∣Q)=-∫‍g(x)f(x)γQ(dx)[∫‍f(x)1+γQ(dx)]γ/(1+γ).
The projective power divergence is given by
(19)Dγ(g,f∣Q)=Cγ(g,f∣Q)-Hγ(g∣Q).Dγ(g,f∣Q) satisfies Dγ(g,f∣Q)≥0, and Dγ(g,f∣Q)=0 if and only if g(x)=f(x), so Dγ(g,f∣Q) can be seen as a kind of distance between g and f.Letx(1),…, x(n) be a random sample from a probability density function g(x) and f(x;θ) a statistical model. Since we want to find the closest distribution to g(x) in the model f(x;θ), we want to find the minimizer of Dγ(g,f(·;θ)∣Q), which is equal to the minimizer of Cγ(g,f(·;θ)∣Q). If Q has the Radon-Nikodym derivative q, then Cγ(g,f(·;θ)∣Q) is equal to
(20)-∫g(x)κγ(θ∣Q)f(x;θ)γq(x)dx,
where
(21)κγ(θ∣Q)=(∫f(x,θ)1+γq(x)dx)-γ/(1+γ).
Empirically the projective power cross entropy Cγ(g,f(·;θ)∣Q) can be estimated by
(22)Lγ(θ∣Q)=-1n∑i=1nκγ(θ∣Q)f(x(i);θ)γq(x(i)),
which is called the loss function. Note that
(23)E(Lγ(θ∣Q))=Cγ(g,f(·;θ)∣Q).
The original γ-estimator θ^γ is defined by the minimizer of the loss function Lγ(θ∣Q):
(24)θ^γ=argminθLγ(θ∣Q).
Note that we are not necessarily seeking to the global minimizer. Rather, we allow the loss function to be multimodal, so we refer the γ-estimator to the set of the local minimizers. See [4, 5] for details of the γ-estimator.


### 3.2. Choice of the Carrier Measure

In calculating theγ-estimator, the measure Q can be determined by each user. Here we propose, for Gaussian copula models, the use of a measure, denoted by QG, of which Radon-Nikodym derivative is given by J(xG)-γ, where J(xG) is the Jacobian of the transformation xG(u). From now on we refer this choice to QG and explain its rationale by virtue of invariance.We assume thatx=(x1,…,xm)⊤~ϕ(x;P), where ϕ(x;P) denotes the probability density function of the m-dimensional Gaussian distribution with mean 0 and correlation matrix P. Let u=(Φ(x1),…,Φ(xm))⊤; then u ~ cG(u;P). If the underlying distribution of x is g(x), then u ~ c(u), where c(u) is given by g(xG(u))J(xG). It is noteworthy that the projective power cross entropy between g(x) and ϕ(x;P) based on x is not always equal to the projective power cross entropy between c(u) and cG(u;P) based on u. So the γ-estimator based on x does not coincide with the γ-estimator based on u.It is natural for us to require the equivalence of the twoγ-estimators, and therefore we employ the measure QG(u). It is striking that the projective power cross entropy between c(u) and cG(u;P) calculated under the measure QG is equal to the projective power cross entropy between g(x) and ϕ(x;P) calculated under the Lebesgue measure QL,
(25)Cγ(c,cG(·;P)∣QG)=Cγ(g,ϕ(·;P)∣QL),
which is proportional to
(26)-det(P)-γ/2(1+γ)∫g(x)exp(-γ2x⊤P-1x)dx.
Obviously there is equalization of the two γ-estimators. Note that the loss function associated with cross entropy (26) becomes (4).The argument above extends to a general statement. For given one to one transformationy(x):x↦y, x(y) denotes the inverse function of y(x) and J(y↦x) denotes the Jacobian of the transformation x(y). Any nonnegative functions g(x), f(x) satisfy
(27)Cγ(g(x(·))J(y⟼x),f(x(·))J(y⟼x)∣Q)=Cγ(g,f∣QL),
if and only if the Radon-Nikodym derivative of Q is equal to J(y↦x)-γ. When g(x) and f(x) are the probability density functions, to consider the projective power cross entropy on x under the Lebesgue measure is equal to consider the projective power cross entropy on y under the measure having J(y↦x)-γ as its Radon-Nikodym derivative.


## 3.1. Projective Power Entropy and theγ-Estimator

The projective power entropy off(x) with the index γ and the measure Q is defined as
(16)Hγ(f∣Q)=-[∫f(x)1+γQ(dx)]1/(1+γ).
If Q is the Lebesgue measure denoted by QL and f(x) is a probability density function, then we have
(17)limγ→0Hγ(f∣QL)+1γ=H0(f∣QL),
where H0(f∣QL) is defined by -∫f(x)logf(x)dx, which is equivalent to the Boltzmann-Shannon entropy. The projective power cross entropy between g(x) and f(x) with the index γ and the measure Q is defined as
(18)Cγ(g,f∣Q)=-∫‍g(x)f(x)γQ(dx)[∫‍f(x)1+γQ(dx)]γ/(1+γ).
The projective power divergence is given by
(19)Dγ(g,f∣Q)=Cγ(g,f∣Q)-Hγ(g∣Q).Dγ(g,f∣Q) satisfies Dγ(g,f∣Q)≥0, and Dγ(g,f∣Q)=0 if and only if g(x)=f(x), so Dγ(g,f∣Q) can be seen as a kind of distance between g and f.Letx(1),…, x(n) be a random sample from a probability density function g(x) and f(x;θ) a statistical model. Since we want to find the closest distribution to g(x) in the model f(x;θ), we want to find the minimizer of Dγ(g,f(·;θ)∣Q), which is equal to the minimizer of Cγ(g,f(·;θ)∣Q). If Q has the Radon-Nikodym derivative q, then Cγ(g,f(·;θ)∣Q) is equal to
(20)-∫g(x)κγ(θ∣Q)f(x;θ)γq(x)dx,
where
(21)κγ(θ∣Q)=(∫f(x,θ)1+γq(x)dx)-γ/(1+γ).
Empirically the projective power cross entropy Cγ(g,f(·;θ)∣Q) can be estimated by
(22)Lγ(θ∣Q)=-1n∑i=1nκγ(θ∣Q)f(x(i);θ)γq(x(i)),
which is called the loss function. Note that
(23)E(Lγ(θ∣Q))=Cγ(g,f(·;θ)∣Q).
The original γ-estimator θ^γ is defined by the minimizer of the loss function Lγ(θ∣Q):
(24)θ^γ=argminθLγ(θ∣Q).
Note that we are not necessarily seeking to the global minimizer. Rather, we allow the loss function to be multimodal, so we refer the γ-estimator to the set of the local minimizers. See [4, 5] for details of the γ-estimator.


## 3.2. Choice of the Carrier Measure

In calculating theγ-estimator, the measure Q can be determined by each user. Here we propose, for Gaussian copula models, the use of a measure, denoted by QG, of which Radon-Nikodym derivative is given by J(xG)-γ, where J(xG) is the Jacobian of the transformation xG(u). From now on we refer this choice to QG and explain its rationale by virtue of invariance.We assume thatx=(x1,…,xm)⊤~ϕ(x;P), where ϕ(x;P) denotes the probability density function of the m-dimensional Gaussian distribution with mean 0 and correlation matrix P. Let u=(Φ(x1),…,Φ(xm))⊤; then u ~ cG(u;P). If the underlying distribution of x is g(x), then u ~ c(u), where c(u) is given by g(xG(u))J(xG). It is noteworthy that the projective power cross entropy between g(x) and ϕ(x;P) based on x is not always equal to the projective power cross entropy between c(u) and cG(u;P) based on u. So the γ-estimator based on x does not coincide with the γ-estimator based on u.It is natural for us to require the equivalence of the twoγ-estimators, and therefore we employ the measure QG(u). It is striking that the projective power cross entropy between c(u) and cG(u;P) calculated under the measure QG is equal to the projective power cross entropy between g(x) and ϕ(x;P) calculated under the Lebesgue measure QL,
(25)Cγ(c,cG(·;P)∣QG)=Cγ(g,ϕ(·;P)∣QL),
which is proportional to
(26)-det(P)-γ/2(1+γ)∫g(x)exp(-γ2x⊤P-1x)dx.
Obviously there is equalization of the two γ-estimators. Note that the loss function associated with cross entropy (26) becomes (4).The argument above extends to a general statement. For given one to one transformationy(x):x↦y, x(y) denotes the inverse function of y(x) and J(y↦x) denotes the Jacobian of the transformation x(y). Any nonnegative functions g(x), f(x) satisfy
(27)Cγ(g(x(·))J(y⟼x),f(x(·))J(y⟼x)∣Q)=Cγ(g,f∣QL),
if and only if the Radon-Nikodym derivative of Q is equal to J(y↦x)-γ. When g(x) and f(x) are the probability density functions, to consider the projective power cross entropy on x under the Lebesgue measure is equal to consider the projective power cross entropy on y under the measure having J(y↦x)-γ as its Radon-Nikodym derivative.


## 4. Property of theγ-Estimator

Theγ-estimator for the Gaussian copula model under infinite sample size is equal to the set of the local minimizers of Cγ(c,cG(·;P)∣QG). In this section we leave aside the empirical loss function Lγ(P) for the moment and investigate the property of the γ-estimator (at n infinity) through Cγ(c,cG(·;P)∣QG). First we consider the case where there is no misspecification.Theorem 1.
Ifc(u)=cG(u;P0), then Cγ(c,cG(·;P)∣QG) has the local minimum at P0.In this case we note that theγ-estimator is equal to {P0}, which implies Fisher consistency. For asymptotic properties the γ-estimator has asymptotic consistency and normality.Next we consider the misspecification case where the true data generating process is given by (1). We see that
(28)Cγ(c,cG(·;P)∣QG)=τCγ(cG(·;P1),cG(·;P)∣QG)+(1-τ)Cγ(cG(·;P2),cG(·;P)∣QG),
which is proportional to
(29)-detP-γ/2(1+γ)[τdetP1-1/2det(P1-1+γP-1)-1/2+(1-τ)detP2-1/2det(P2-1+γP-1)-1/2].Cγ(c,cG(·;P)∣QG) is a weighted mean of the two projective power cross entropy. Each component is a unimodal function, bounded above by 0 and has one local minimum at P1 and P2, respectively. So we expect that Cγ(c,cG(·;P)∣QG) has two local minima and these local minimizers are near P1 and P2, respectively, if P1 and P2 are sufficiently “distinct.” However it is hard to formulate such a phenomenon mathematically so we show through easy examples and a graph that such a phenomenon occurs. To obtain numerical solutions, we use the expected (or population) version of the algorithm in Section 2.3.Example 2.
In the case with dimension 2,Cγ(c,cG(·;P)∣QG) is a univariate function of ρ, which is the nondiagonal element of P. Let P1 and P2 be
(30)P1=(1ρ*ρ*1),P2=(1-ρ*-ρ*1),γ=1, and τ=0.5. If ρ*>6-28  =˙  0.842, then Cγ(c,cG(·;P)∣QG) has two local minima in the interval (-1,0) and (0,1), respectively.Example 3.
Suppose the true correlation matricesP1 and P2 are given as follows, and P stands for the parameterization of the statistical model we fit:
(31)P1=(10.90.920.910.90.920.91),P2=(1-0.90.92-0.91-0.90.92-0.91),P=(1ρ1ρ1ρ2ρ11ρ2ρ1ρ2ρ21).
We also set τ=0.5 and γ=1. Note that -Cγ(c,cG(·;P)∣QG) is a function of ρ1 and ρ2. Figure 1 shows the graph of -Cγ(c,cG(·;P)∣QG). We can see there exist two local maxima at (0.86,0.86) and (-0.86,-0.86).Figure 1
Illustration of-Cγ(c,cG(·;P)∣QG).Example 4.
Supposeτ=0.4 and γ=1. When P1,P2, and P are given by
(32)P1=(10.90.70.70.910.90.70.70.910.70.70.70.71),P2=(1-0.90.70.7-0.91-0.9-0.70.7-0.910.70.7-0.70.71),P=(1ρ1ρ2ρ3ρ11ρ4ρ5ρ2ρ41ρ6ρ3ρ5ρ61).
Then Cγ(c,cG(·;P)∣QG) has two local minima at
(33)P=(10.8710.6860.6990.87110.8710.6830.6860.87110.6990.6990.6830.6991),P=(1-0.8920.6950.700-0.8921-0.892-0.6960.695-0.89210.7000.700-0.6960.7001).
Like these examples,Cγ(c,cG(·;P)∣QG) has some local minima depending on the underlying distribution. Owing to this property we can detect the heterogeneous structures of the underlying distribution under misspecification.


## 5. Maximum Entropy Distribution

So far we have considered theγ-estimation of the Gaussian copula model. In this section we uncover that the choice of copula model can be characterized in terms of the maximum entropy distribution. In this regard, the most closely related work is in [2], in which the MLE on meta-t-distribution is addressed. A t-copula is deduced from a multivariate t-distribution while the meta-t-distribution is constructed by linking a t-copula to univariate t-distributions as its marginal distributions. In our framework, [2]’s work can be interpreted as the maximum likelihood estimation of t-copulas with the marginals estimated simultaneously. Actually the γ-estimation of Gaussian copulas and the maximum likelihood estimation of t-copulas look very similar and share a common idea.In [4], it is analyzed what the maximum projective power entropy distributions would be under the given (population) mean vector and covariance matrix. The answer depends on the power index γ. When γ=0, the Gaussian distribution emerges as the maximum projective power entropy distribution. If γ<0, the t-distribution comes up. We show that a similar result holds for copulas. Let tν be the cumulative distribution function of the t-distribution with degrees of freedom ν, and xt,ν(u)=(tν-1(u1),…,  tν-1(um))⊤. We suppose that γ=-2/(ν+m),Qt,ν(du)=J(xt,ν)-γdu. Let 𝒞γ(P) be the set of probability density functions c(u) on [0,1]m which satisfy the following equation:
(34)∫[0,1]mc(u)xt,ν(u)xt,ν(u)⊤du=vv-2P.
Let ft(x;ν,P) denote the probability density function of t-distribution with degrees of freedom ν and correlation matrix P, and let ct(u;ν,P) denote the probability density function of its copula (t-copula):
(35)ct(u;ν,P)=ft(xt,ν(u);ν,P)J(xt,ν),ft(x;ν,P)=Γ((ν+m)/2)Γ(ν/2)det(νπP)-1/2×(1+x⊤P-1xν)-(ν+m)/2.
Then we see
(36)argmaxc∈𝒞γ(P)Hγ(c∣Qt,ν)=ct(u;ν,P).
If γ→0, then tν→Φ and Qt,ν→QL. So we see that
(37)argmaxc∈𝒞0(P)H0(c∣QL)=cG(u;P).
That is, t-copula can be characterized as the maximum projective power entropy distribution on [0,1]m. Moreover it has limiting equivalence (by letting γ→0) with the Gaussian copula which is tagged with the maximum Boltzmann-Shannon entropy distribution. We call these maximum projective power entropy copulas the γ-copulas. Let us consider the relationship between the γ-copula and the γ-estimation. Our method is discussed on the pair of the Gaussian copula (0-copula) and γ-estimator. On the other hand [2] discussed on the pair of γ-copula model (γ<0) and the MLE (0-estimator). We see a sort of duality relationship between two choices of the pair.


## 6. Robustness

In this section we examine robustness of theγ-estimator for the Gaussian copula model through its influence function. The influence function measures the asymptotic bias caused by contamination at the x. The boundedness of the influence function means boundedness of the influence from the outlier, hence its robustness. The influence function of the γ-estimator is given in Section 6.1. We show that it is bounded when γ>0. In the next subsection, a brief simulation is performed.


### 6.1. Influence Function

Theγ-estimator for the Gaussian copula model can be regarded as a functional T(g) of a distribution g defined by
(38)argsolveP∫exp(-γ2x⊤P-1x)v(P-1-(1+γ)P-1xx⊤P-1)×g(x)dx=0.
Let ψγ(x;P) be
(39)ψγ(x;P)=exp(-γ2x⊤P-1x)v(P-1-(1+γ)P-1xx⊤P-1).
Then the influence function IF(x;T,g) of the γ-estimator is given by
(40)IF(x;T,g)=-[∫ψ˙γ(x;T(g))g(x)dx]-1ψγ(x;T(g)),
where ψ˙γ(x;P)=(∂/∂v(P))ψγ(x;P). See [7] for details. The boundedness of the influence function is equivalent to the boundedness of ψγ(x;P). The following theorem gives a bound of ψγ(x;P).Theorem 5.
Whenγ=0, that is, for the MLE, the influence function is not bounded. When γ<0, the influence function is not bounded. When γ>0, the influence function is bounded and a bound is given by
(41)∥ψγ(x;P)∥≤∥v(P-1)∥+2(1+γ)eγ∥P-1/2⊗P-1/2∥,
where ⊗ denotes the Kronecker product and ∥h∥ for an m-dimensional vector h denotes the Euclidean norm defined by h⊤h.For example, ifP is equal to Im, then ∥ψγ(x;Im)∥≤(2(1+γ)/eγ)m.


### 6.2. Simulation

This subsection describes the results of Monte Carlo simulations carried out in order to examine the robustness of theγ-estimator for the Gaussian copula model. We generate 500 pseudorandom samples of size 500 from distribution
(42)0.9cG(u,P)+0.1cG(u,I10),
where cG(u,I10) is equal to the independent copula and P is given by
(43)P=(10.846⋯⋯0.84610.846⋯0.846⋱⋱⋮⋱⋮1).
For each sample, we calculate the γ-estimator P^GE for the Gaussian copula model with γ=0.5 and the MLE P^MLE for the Gaussian copula model. We use the norm ∥P^-P∥ as the accuracy measure. Table 1 shows the root mean squared error (RMSE) of the norm for the γ-estimator and MLE. We can see that the norm for the γ-estimator is less than that for the MLE, so we see that the γ-estimator is more robust than the MLE.Table 1
RMSE of the norm for theγ-estimator and MLE.
P
^
GE
P
^
MLE
RMSE
0.155
0.808


## 6.1. Influence Function

Theγ-estimator for the Gaussian copula model can be regarded as a functional T(g) of a distribution g defined by
(38)argsolveP∫exp(-γ2x⊤P-1x)v(P-1-(1+γ)P-1xx⊤P-1)×g(x)dx=0.
Let ψγ(x;P) be
(39)ψγ(x;P)=exp(-γ2x⊤P-1x)v(P-1-(1+γ)P-1xx⊤P-1).
Then the influence function IF(x;T,g) of the γ-estimator is given by
(40)IF(x;T,g)=-[∫ψ˙γ(x;T(g))g(x)dx]-1ψγ(x;T(g)),
where ψ˙γ(x;P)=(∂/∂v(P))ψγ(x;P). See [7] for details. The boundedness of the influence function is equivalent to the boundedness of ψγ(x;P). The following theorem gives a bound of ψγ(x;P).Theorem 5.
Whenγ=0, that is, for the MLE, the influence function is not bounded. When γ<0, the influence function is not bounded. When γ>0, the influence function is bounded and a bound is given by
(41)∥ψγ(x;P)∥≤∥v(P-1)∥+2(1+γ)eγ∥P-1/2⊗P-1/2∥,
where ⊗ denotes the Kronecker product and ∥h∥ for an m-dimensional vector h denotes the Euclidean norm defined by h⊤h.For example, ifP is equal to Im, then ∥ψγ(x;Im)∥≤(2(1+γ)/eγ)m.


## 6.2. Simulation

This subsection describes the results of Monte Carlo simulations carried out in order to examine the robustness of theγ-estimator for the Gaussian copula model. We generate 500 pseudorandom samples of size 500 from distribution
(42)0.9cG(u,P)+0.1cG(u,I10),
where cG(u,I10) is equal to the independent copula and P is given by
(43)P=(10.846⋯⋯0.84610.846⋯0.846⋱⋱⋮⋱⋮1).
For each sample, we calculate the γ-estimator P^GE for the Gaussian copula model with γ=0.5 and the MLE P^MLE for the Gaussian copula model. We use the norm ∥P^-P∥ as the accuracy measure. Table 1 shows the root mean squared error (RMSE) of the norm for the γ-estimator and MLE. We can see that the norm for the γ-estimator is less than that for the MLE, so we see that the γ-estimator is more robust than the MLE.Table 1
RMSE of the norm for theγ-estimator and MLE.
P
^
GE
P
^
MLE
RMSE
0.155
0.808


## 7. Simulation Study

The property of theγ-estimator to detect heterogeneous structures is investigated by a bunch of simulations. A comparison of the γ-estimator with the MLE for a mixture Gaussian copula (1) is also discussed.


### 7.1. Simulation Setup

We conducted two kinds of simulation.Simulation 1. The underlying distribution was constructed based on the one factor Gaussian copula model [8]. Suppose
(44)xi=aiW+1-ai2εi,i=1,…,m,
where W,ε1,…,εm have independently the standard normal distribution. Then we see x~Φ(x;P), u=(Φ-1(x1),…,Φ-1(xm))~cG(u;P), where  P=(pij)ij satisfies
(45)pij={1,i=j,aiaj,i≠j.
Let the underlying distribution be (1), where cG(u;P1) and cG(u;P2) are made from the one factor Gaussian copula model. This model means that the dependence structure is expressed by the mixture of Gaussian copulas. Set τ=0.5. P1 is made with
(46)a=(0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92)⊤,
and P2 with
(47)a=(-0.92,0.92,-0.92,0.92,-0.92,0.92,-0.92,0.92,-0.92,0.92)⊤.
Then we have
(48)P1=(10.846⋯0.8461⋱⋮⋱⋮1),P2=(1-0.8460.846⋯-0.8461⋱⋱⋮⋱⋱⋮1).Theγ-estimator for the Gaussian copula model with γ=0.7 is investigated. Initial values of P which is used in calculating the γ-estimator are AR(±0.1), AR(±0.3),…,AR(±0.9), where AR(ρ) is the correlation matrix whose (i,j) component (i<j) is equal to ρj-i. If the γ-estimator has two components G1 and G2 such that
(49)∥G1-P1∥<∥G2-P1∥,
then G1 is thought of as an estimator of P1 and denoted by P^1,GE. Similarly G2 for P2 and denoted by P^2,GE.We adopt the MLE for a mixture Gaussian copula model (1). Although P1 and P2 are the correlation matrices, we tentatively view them to be the covariance matrices and use EM-Algorithm to obtain an approximate MLE. The obtained estimators P^1 and P^2 are not necessarily the correlation matrices, so they are transformed into the correlation matrices by
(50)diag(P^i)-1/2P^idiag(P^i)-1/2,
which is denoted by P^i,MLE for i=1,2. The initial value of (τ,P1,P2) which is used in calculating the MLE is set to (0.5,AR(0.5),AR(-0.5)).A set of data of sizen (n=500,1000) was generated from (1), and the norm of P^1,GE-P1, P^2,GE-P2, P^1,MLE-P1, and P^2,MLE-P2 were calculated. 500 simulations were carried out, and then, we calculated the RMSE of the norm based on 500 norm values obtained by simulation. The results are shown in Table 3.Simulation 2. Suppose that the underlying distribution is
(51)c(u)=τ1cG(u;P1)+τ2cG(u;P2)+(1-τ1-τ2)cG(u;I10),
where τ1=τ2=0.45 and P1,P2 are the same in Simulation 1. The other settings are the same as in Simulation 1. The results are shown in Table 5.


### 7.2. Result

Result of Simulation 1. Table 2 shows the ratio for the γ-estimator to detect two correlation matrices. For n=500 nearly 80 percent was successful, and for n=1000 it worked out almost perfectly. From Table 3, the MLE had better performance than the γ-estimator. However this is natural because the MLE is used under no misspecification.Table 2
Ratio of the number of success for theγ-estimator to detect two correlation matrices.
n
500
1000
Ratio
0.768
0.968Table 3
Simulation 1. RMSE of the norm ofP^1,GE-P1,P^1,MLE-P1,P^2,GE-P2, and P^2,MLE-P2.
n
P
^
1
,
GE
P
^
1
,
MLE
P
^
2
,
GE
P
^
2
,
MLE
RMSE
500
0.600
0.184
0.476
0.186
1000
0.479
0.127
0.431
0.129Result of Simulation 2. Table 4 shows the ratio for the γ-estimator to detect two correlation matrices. Compared to the result of Simulation 1 the detection rate at n=500 gets worse while at n=1000 the result is almost alike in Table 2. From Table 5, we find the MLE is considerably underperforming and the γ-estimator is much better.Table 4
Ratio of the number of success for theγ-estimator to detect two correlation matrices.
n
500
1000
Ratio
0.61
0.966Table 5
Simulation 2. RMSE of the norm ofP^1,GE-P1,P^1,MLE-P1,P^2,GE-P2, and P^2,MLE-P2.
n
P
^
1
,
GE
P
^
1
,
MLE
P
^
2
,
GE
P
^
2
,
MLE
RMSE
500
0.494
0.946
0.563
0.952
1000
0.468
1.010
0.438
1.032


## 7.1. Simulation Setup

We conducted two kinds of simulation.Simulation 1. The underlying distribution was constructed based on the one factor Gaussian copula model [8]. Suppose
(44)xi=aiW+1-ai2εi,i=1,…,m,
where W,ε1,…,εm have independently the standard normal distribution. Then we see x~Φ(x;P), u=(Φ-1(x1),…,Φ-1(xm))~cG(u;P), where  P=(pij)ij satisfies
(45)pij={1,i=j,aiaj,i≠j.
Let the underlying distribution be (1), where cG(u;P1) and cG(u;P2) are made from the one factor Gaussian copula model. This model means that the dependence structure is expressed by the mixture of Gaussian copulas. Set τ=0.5. P1 is made with
(46)a=(0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92,0.92)⊤,
and P2 with
(47)a=(-0.92,0.92,-0.92,0.92,-0.92,0.92,-0.92,0.92,-0.92,0.92)⊤.
Then we have
(48)P1=(10.846⋯0.8461⋱⋮⋱⋮1),P2=(1-0.8460.846⋯-0.8461⋱⋱⋮⋱⋱⋮1).Theγ-estimator for the Gaussian copula model with γ=0.7 is investigated. Initial values of P which is used in calculating the γ-estimator are AR(±0.1), AR(±0.3),…,AR(±0.9), where AR(ρ) is the correlation matrix whose (i,j) component (i<j) is equal to ρj-i. If the γ-estimator has two components G1 and G2 such that
(49)∥G1-P1∥<∥G2-P1∥,
then G1 is thought of as an estimator of P1 and denoted by P^1,GE. Similarly G2 for P2 and denoted by P^2,GE.We adopt the MLE for a mixture Gaussian copula model (1). Although P1 and P2 are the correlation matrices, we tentatively view them to be the covariance matrices and use EM-Algorithm to obtain an approximate MLE. The obtained estimators P^1 and P^2 are not necessarily the correlation matrices, so they are transformed into the correlation matrices by
(50)diag(P^i)-1/2P^idiag(P^i)-1/2,
which is denoted by P^i,MLE for i=1,2. The initial value of (τ,P1,P2) which is used in calculating the MLE is set to (0.5,AR(0.5),AR(-0.5)).A set of data of sizen (n=500,1000) was generated from (1), and the norm of P^1,GE-P1, P^2,GE-P2, P^1,MLE-P1, and P^2,MLE-P2 were calculated. 500 simulations were carried out, and then, we calculated the RMSE of the norm based on 500 norm values obtained by simulation. The results are shown in Table 3.Simulation 2. Suppose that the underlying distribution is
(51)c(u)=τ1cG(u;P1)+τ2cG(u;P2)+(1-τ1-τ2)cG(u;I10),
where τ1=τ2=0.45 and P1,P2 are the same in Simulation 1. The other settings are the same as in Simulation 1. The results are shown in Table 5.


## 7.2. Result

Result of Simulation 1. Table 2 shows the ratio for the γ-estimator to detect two correlation matrices. For n=500 nearly 80 percent was successful, and for n=1000 it worked out almost perfectly. From Table 3, the MLE had better performance than the γ-estimator. However this is natural because the MLE is used under no misspecification.Table 2
Ratio of the number of success for theγ-estimator to detect two correlation matrices.
n
500
1000
Ratio
0.768
0.968Table 3
Simulation 1. RMSE of the norm ofP^1,GE-P1,P^1,MLE-P1,P^2,GE-P2, and P^2,MLE-P2.
n
P
^
1
,
GE
P
^
1
,
MLE
P
^
2
,
GE
P
^
2
,
MLE
RMSE
500
0.600
0.184
0.476
0.186
1000
0.479
0.127
0.431
0.129Result of Simulation 2. Table 4 shows the ratio for the γ-estimator to detect two correlation matrices. Compared to the result of Simulation 1 the detection rate at n=500 gets worse while at n=1000 the result is almost alike in Table 2. From Table 5, we find the MLE is considerably underperforming and the γ-estimator is much better.Table 4
Ratio of the number of success for theγ-estimator to detect two correlation matrices.
n
500
1000
Ratio
0.61
0.966Table 5
Simulation 2. RMSE of the norm ofP^1,GE-P1,P^1,MLE-P1,P^2,GE-P2, and P^2,MLE-P2.
n
P
^
1
,
GE
P
^
1
,
MLE
P
^
2
,
GE
P
^
2
,
MLE
RMSE
500
0.494
0.946
0.563
0.952
1000
0.468
1.010
0.438
1.032


## 8.  Discussion

We have considered an estimation problem for misspecified Gaussian copula model. By the simulation study our methodology has been found to work well for misspecification. Though we did not consider how to determine the value ofγ, this problem was considered in [9] for independent component analysis. It could be possible to follow their method in our problem, but it is currently a future problem.We choose the measure in terms of invariance. However theγ-estimator obtained is equal to the estimator with normal distribution as a statistical model, so it seems natural. If we use Lebesgue measure in calculating the γ-estimator for Gaussian copula model, we cannot calculate the projective power entropy for all the value of γ and P.Another issue is to what extent the methodology here works for time series data. Because the basic premise of this paper is that we have data as quantiles, our method would fit, for example, the modeling of unconditional loss distribution [1, page 28]. Such a case is of particular interest when the time horizon over which we measure our losses is relatively large. When we are working on the conditional modeling, our method should be regarded as a tool for the post analysis. As a typical case, we may want to apply our mixture copula approach to multivariate log-return series which are appropriately standardized and declustered by the multivariate GARCH model fitted to them. See [2] for more details.



---

*Source: 787141-2013-12-03.xml*