# The Optimal Selection for Restricted Linear Models with Average Estimator

Authors: Qichang Xie; Meng Du
Journal: Abstract and Applied Analysis
 (2014)

---

## Abstract

The essential task of risk investment is to select an optimal tracking portfolio among various portfolios. Statistically, this process can be achieved by choosing an optimal restricted linear model. This paper develops a statistical procedure to do this, based on selecting appropriate weights for averaging approximately restricted models. The method of weighted average least squares is adopted to estimate the approximately restricted models under dependent error setting. The optimal weights are selected by minimizing ak-class generalized information criterion (k-GIC), which is an estimate of the average squared error from the model average fit. This model selection procedure is shown to be asymptotically optimal in the sense of obtaining the lowest possible average squared error. Monte Carlo simulations illustrate that the suggested method has comparable efficiency to some alternative model selection techniques.


---

## Body

## 1. Introduction

The essential task of risk investment aims to select an optimal tracking portfolio among numerous portfolios of stocks. Given a desired target and a series of stocks, a tracking portfolio is comprised by every nonempty subset of the given group of stocks so as to track the target to a certain degree. Because of the number of nonempty subsets of stocks, there exists a mass of possible tracking portfolios. Among all possible portfolios, we should find an optimal tracking portfolio whose return is closest to the targets. Statistically, a tracking portfolio is built by a group of stocks, which is equivalent to fitting a restricted linear model with the target’s return as the dependent variable and returns on stocks in the group as the regressors. Since the coefficient of a regressor indicates the proportion of the investment in the corresponding stock within the total investment in the portfolio, the linear model is restricted such that all coefficients in the model sum to one. Thus, the task of choosing an optimal tracking portfolio can be accomplished by selecting an optimal restricted linear model.In this paper, a model average technique is developed for examining the selection problem of restricted linear models. Model selection has played an important role in econometrics and statistics over the past decades. The goal of model selection is to choose a model which gives the well-posed fit for observational data. So, the investigation of model selection is an indispensable process in empirical analysis. This work proposes a procedure of minimizingk-class generalized information criterion to select the optimal weights for constrained linear models. Under some conditions, we examine the asymptotic behaviors of the selection program.Various methods have been suggested to study the problems of model selection. Knight and Fu [1] discussed the lasso-type estimators with least squares methods. To simultaneously estimate parameters and select important variables, Fan and Peng [2] proposed a method of nonconcave penalized likelihood and demonstrated that this technique had an oracle property when the number of parameters was infinite. Zou and Yuan [3] investigated the oracle theory of model selection based on composite quantile regression. Caner [4] considered model selection by the generalized method of moments estimator. In the empirical likelihood framework, Tang and Leng [5] studied the parametric estimator and variable selection for diverging numbers of parameters. Jennifer et al. [6] explored the ability of automatic selection algorithms to handle the selection problems of both variables and principal components.Model averaging is another popular and widely used technique for model selection. The method is to average the estimators corresponding to different candidate models. Bayesian and frequentist are two main perspectives of thought in model averaging. Although their spirit and objectives are similar, the two techniques are different in inference and selection of models. In view of the Bayesian model averaging, the basic paradigm was introduced by Leamer [7]. Owing to the difficulty of implementing, the approach was basically ignored until the 2000s. About recent developments of this method, the readers can refer to Brown et al. [8] and Rodney and Herman [9]. Compared with Bayesian model averaging, since the method of frequentist model averaging focused on model selection rather than model averaging, it has been considered by many authors, for instance, Hjort and Claeskens [10], Hansen [11], Liang et al. [12], Zhang and Liang [13], and Hansen and Racine [14].Generally speaking, different methods of model selection need to construct distinct model selection criteria including AIC [15], Mallows' CL [16], CV [17], BIC [18], GCV [19], GMM J-statistic [20], and FPEα [21]. Zhang et al. [22] employed the generalized information criterion for selecting the regularization parameters. To choose basis functions of splines, Xu and Huang [23] showed the optimal property of a LsoCV criterion and designed an efficient Newton-type algorithm for this criterion. Focusing on the divergence measure of Kullback-Leibler, So and Ando [24] defined a generalized predictive information criterion using the bias correction of an expected weighted loglikelihood estimator. Groen and Kapetanios [25] examined the criteria of AIC and BIC to discuss consistent estimates of a factor-augmented regression.The literature mentioned above pays more attention to the unconstrained models with independently and identically distributed random errors. Recently, Lai and Xie [26] discussed model selection for constrained models, which were limited to the homoscedastic cases. Instead of using unrestricted models or homoscedastic models, we develop a k-class generalized information criterion (k-GIC) to discuss the selecting problems of approximately constrained linear models with dependent errors. The k-GIC is an extension of the GICλn proposed by Shao [27] and includes some conventional model selection criteria, such as BIC and GIC. We employ the technique of weighted average least squares to estimate the approximately constrained models and choose the weights through minimizing the k-class generalized information criterion. Our main result demonstrates that the k-class generalized information criterion is asymptotically equivalent to the average squared error. In other words, the selected weights from k-GIC are asymptotically optimal. Moreover, we highlight two new results which enrich the works of Lai and Xie [26]. One is that an estimate of variance is given and the estimate is proved to be consistent. Another is that the selected weights from k-GIC are shown to be still asymptotically optimal, when the true variance is replaced by the suggested estimate. The finite sample properties of model selection are performed by Monte Carlo simulation. The results of simulation reveal that the proposed method of model selection is dominant over some alternative approaches.The remainder of this paper begins with an illustration of the model set-up and average estimator in Section2. Section 3 calculates the average squared error of the model average estimator. The k-GIC criterion is introduced and its asymptotic optimality is derived in Section 4. Section 5 states some results from simulation evidence and Section 6 is conclusions.


## 2. Model Set-Up and Average Estimator

The core in risk investment is to build a tracking portfolio of stocks whose return mimics that of a chosen investment target. Letyi be the return from investing in a selected target and xi,j be the historical return of the jth stock at time i. Assume that p stocks are available for building a tracking portfolio of the target. Then, a tracking portfolio consisting of all p stocks can be represented by
(1)yi=∑j=1p‍xi,jθj+ei,
where θj is unknown parameter and ei is random error. The left-hand side of (1) is the return of investing one dollar in the target. The right-hand side is the return of investing one dollar in the portfolio consisting of all p stocks plus random noise.Because each parameterθj stands for the proportion of investment on the corresponding stock to the total investment in the tracking portfolio, the sum of all parameters is one, namely,
(2)θ1+⋯+θp=1,
which means the 100 percent of the whole investment.In practice, there exist a large number of stocks. These stocks compose various portfolios that may track the target to some degree. Among all possible tracking portfolios, an ideal tracking portfolio should be the one whose return is closest to the target’s return. Therefore, we need to find such an optimal tracking portfolio. Because of the dependance between a tracking portfolio and a restricted linear model, the aim of finding the optimal tracking portfolio can be accomplished by choosing an optimal restricted linear model.In the following, we extend models (1) and (2) to consider a generalized constrained linear model for the problem of building an optimal tracking portfolio. Suppose that yi, i=1,…,n is a random observation at fixed value (xi,zi), where xi=(xi,1,…,xi,p) is fixed-dimensional explanatory variable and zi=(zi,1,…,zi,∞) is countably infinite-dimensional explanatory variable. Consider the constrained linear model
(3)yi=∑j=1p‍xi,jθj+∑l=1∞‍zi,lψl+ei,subjectto(4)rsθs=∑k=1∞‍as,kψk+bs,(s=1,…,p),
where p is a positive integer, θj and ψl are parameters, ei is random error, rs is a constant for restricting the sth parameter θs, and as,k and bs are some constants. Assume that ϕ(zi,l,ψl)=∑l=1∞‍zi,lψl and ϕ(as,k,ψk)=∑k=1∞‍as,kψk converge in mean square.In (3), the explanatory variable xi,j is involved in the model on theoretical grounds or other reasons and zi,l is the additional explanatory variable that we need to make sure whether it should be included in the model. In the context of building tracking portfolio, the fixed explanatory variable xi,j stands for the historical return of the jth stock at time i, which must be selected by investors because of their personal preference or the stable earning of this stock. In a tracking portfolio, investors need to select some alternative stocks from numerous stocks to realize their expected return. So, the additional explanatory variable zi,l indicates the historical return of the lth alternative stock at time i. Since zi,l can be viewed as a series expansion, the identity (3) includes semiparametric models as special form. In fact, the model (3) generalizes the models considered by Lai and Xie [26] and Liang et al. [12]. In addition, the parameters θj and ψl denote, respectively, the proportion of the jth required stock and the lth alternative stock in a tracking portfolio. In (4), the parameter θs is adjusted by a linear combination of ψk and bs. The economic significance of (4) is that the proportion of each fixed investment varies with the proportion of all alternative investments. When investors change their preference or have acquired new information on alternative stocks, they are capable of adjusting the proportion between required stocks and alternative stocks according to (4). This implies that the increase or decrease of alternative stocks can affect the proportion of each required stock in a portfolio. Besides, if we assume that ψk=0, k=1,…,∞, the model (4) becomes rsθs=bs which which has been discussed by Lai and Xie [26]. Particularly, if set r1=⋯=rp=1, a1,1+⋯+ap,1=⋯=a1,∞+⋯+ap,∞=-1, and b1+⋯+bp=1, the restricted equation (4) turns into (2).Denote an index setU={K1,…,KM}, where M is a positive integer. Let Θ=(θ1,…,θp)T and Ψ=(ψ1,…,ψ∞)T, where “T” stands for the transpose operation. Due to the uncertain number of zi,l in formula (3), we consider a sequence of approximately restricted models (3) and (4) with Km∈U, where the mth model includes the first Km elements of zi, that is, zi,1,…,zi,Km, and the parameter of θs is restricted by a linear combination with Km elements of Ψ and a constant bs. Hence, the mth approximately restricted models (3) and (4) are
(5)yi=∑j=1p‍xi,jθj+∑l=1Km‍zi,lψl+di+ei,subjectto(6)rsθs=∑k=1Km‍as,kψk+ds★+bs,(s=1,…,p),
where di=∑l=Km+1∞‍zi,lψl and ds★=∑k=Km+1∞‍as,kψk are the approximation errors.SetY=(y1,…,yn)T, asm=(as,1,…,as,Km)T, Aℵm=(a1mT,…,apmT)T, D=(d1,…,dn)T, D★=(d1★,…,dp★)T, Ψm=(ψ1,…,ψKm)T, B=(b1,…,bp)T, and e=(e1,…,en)T. By matrix notation, the mth approximately restricted models (5) and (6) can be rewritten as
(7)Y=XΘ+ZmΨm+D+e,subjectto(8)RΘ=AℵmΨm+D★+B,
where X is a n×p matrix whose ijth element is xi,j, Zm is a n×Km matrix whose ijth element is zi,j, and R is a p×p diagonal matrix whose ith diagonal element is ri.Hypothesize that{ei}i=1n satisfies E(ei∣xi,zi)=0 and its conditional covariance matrix Ωn=E(eeT∣X,Zm) is
(9)Ωn=(σ12ϱ1⋯ϱn-1ϱ1σ22⋯ϱn-2⋮⋮⋯⋮ϱn-1ϱn-2⋯σn2),
in which E(eiei∣xi,zi)=σi2 and E(eiei+j∣xi,zi)=ϱj with i=1,…,n and j=1,…,n-1. Clearly, the random errors follow a heteroscedastic stationary Gaussian process.Substituting (8) into (7), it yields
(10)Yu=Zm,uΨm+U,
where Yu=Y-XR-1B, U=D+XR-1D★+e, and Zm,u=Zm+XR-1Aℵm. By the method of least squares, the estimator of Ψm is
(11)Ψ^m=(Zm,uTZm,u)-1Zm,uTYu,
where (Zm,uTZm,u)-1 denotes the inverse of Zm,uTZm,u. In the mth approximating model (7), we set μm=XΘ+ZmΨm, so that μ=E(Y∣X,Zm)=μm+D. Thus, the estimator of μm is
(12)μ^m=XΘ^+ZmΨ^m=η+PmY,
where η=(I-Pm)XR-1B and Pm=Zm,u(Zm,uTZm,u)-1Zm,uT is the “hat” matrix.Letw=(w1,…,wM)T be a weight vector, where M<n. Define a weight set W as
(13)W={w∣wm∈[0,1],m=1,…,M,∑m=1M‍wm=1}.For allm≤M, the weighted average estimator of Ψm is
(14)Ψ^w=∑m=1M‍wmΨ^m=∑m=1M‍wm(Zm,uTZm,u)-1Zm,uTYu.Naturally, the weighted average estimator ofΘ^ is
(15)Θ^w=R-1B+R-1AℵmΨ^w.Furthermore, the weighted average estimator ofμ is
(16)μ^(w)=∑m=1M‍wmμ^m=η(w)+P(w)Y,
where P(w)=∑m=1M‍wmPm and η(w)=(I-P(w))XR-1B. It can be seen that the weighted estimator μ^(w) is an average estimator of μ^m, m=1,…,M. The weighted “hat” matrix P(w) depends on nonrandom regressor Zm,u and weight vector w. In general conditions, the matrix P(w) is symmetric, but not idempotent.For a positive integerG, let ξ be the maximal value of σι2, ι=1,…,n, and let λ-j, j=1,…,G be the nonzero eigenvalue of Ωn. Assume that both λ-j and ϱι are summable, namely,
(17)Γ=∑j=1G‍λ-j<∞,W=ξ+2∑ι=1n-1‍|ϱι|<∞.Since the covariance matrixΩn determines the algebraic structure of the model average estimator, we discuss its properties in the following.Lemma 1.
For anyn dimensional vectors a=(a1,…,an)T and b=(b1,…,bn)T, one has |aTΩnb|≤∥a∥∥b∥W, where ∥·∥ is the Euclidean norm.Proof.
Through the definition ofΩn, one has
(18)|aTΩnb|=|∑l=1n‍alblσl2+∑i=1n-1‍ϱi∑l=1n-i‍(al+ibl+albl+i)|≤ξ|∑l=1n‍albl|+|∑i=1n-1‍ϱi||∑l=1n-i‍(al+ibl+albl+i)|.
Applying Cauchy-Schwarz inequality, fori∈{1,…,n-1}, one gets
(19)∑l=1n-i‍al+ibl≤∑l=1n-i‍al+i2∑l=1n-i‍bl2≤∑l=1n‍al2∑l=1n‍bl2=∥a∥∥b∥.
Similarly,∑l=1n-i‍albl+i≤∥a∥∥b∥ holds. Therefore,
(20)|aTΩnb|≤∥a∥∥b∥(ξ+2∑i=1n-1‍ϱi)=∥a∥∥b∥W.Because the matrixP(w) takes an important role in analyzing the problems of model selection, we state some of its properties. We set τ~=max⁡{K1,…,KM}. Let λ(P(w)) and λmax⁡(P(w)) denote the eigenvalue and the largest eigenvalue of P(w), respectively.Lemma 2.
One has (i)tr⁡(P(w))≤τ~ and (ii) 0≤λ(P(w))≤1, where tr⁡(·) denotes the trace operation.Proof.
It follows fromtr⁡(Pm)=Km and tr⁡(P(w))=∑m=1M‍wmKm≤τ~ that (i) is established. Next, we consider (ii). Without loss of generality, let λ1,m≥⋯≥λn,m and φ1,m,…,φn,m be the eigenvalues and standardly orthogonal eigenvectors of Pm, respectively. Observe that Pm is idempotent, which implies that λi,m∈{0,1}, i=1,…,n. For any ϖ∈Rn, it can be seen that
(21)λ(P(w))=ϖTP(w)ϖϖTϖ=∑m=1M‍wmϖTPmϖϖTϖ=∑m=1M‍wmχTΞmχχTχ=∑m=1M‍wm∑i=1n‍λi,m|χi|2χTχ,
where Ξm=diag⁡(λ1,m,…,λn,m), ϖ=Φmχ, and Φm=(φ1,m,…,φn,m). Due to λi,m∈{0,1}, i=1,…,n, it means that
(22)λ(P(w))≤∑m=1M‍wm∑i=1n‍1|χi|2χTχ=∑m=1M‍wm=1,λ(P(w))≥∑m=1M‍wm∑i=1n‍0|χi|2χTχ=0.From Lemma2(ii), we know that P(w) is nonnegative definite.Lemma 3.
Letα denote the number of eigenvalues of P(w). Then (α-1tr⁡(P(w)))2≤α-1tr⁡(P2(w)).Proof.
Letλ1,…,λα be the eigenvalues of P(w). (i) If λ1=⋯=λα=0, we know that Lemma 3 holds. (ii) Let λ1≠0,…,λα-≠0 and λα-+1=⋯=λα=0. It is easy to see that ∑i=1α-‍(λi-λ-)2≥0, where λ-=α--1tr⁡(P(w)). Thus, it can be shown that
(23)∑i=1α-‍(λi-λ-)2=∑i=1α-‍λi2-2λ-∑i=1α-‍λi+α-λ-2=tr⁡(P2(w))-α-(α--1tr⁡(P(w)))2≥0,
which implies that (α--1tr⁡(P(w)))2≤α--1tr⁡(P2(w)).Lemma 4.
For anyw∈W, there exists C1=C′2 such that tr⁡(P(w)Ωn)≤C′Γ, tr⁡(ΩnP(w))2≤C1Γ2 and tr⁡(ΩnPT(w)P(w))2≤C1Γtr⁡(P(w)ΩnPT(w)), in which C′=supw∈Wλmax⁡(P(w)).Proof.
Notice thatλmin⁡(A)λmax⁡(B)≤λmax⁡(AB)≤λmax⁡(A)λmax⁡(B) and tr⁡(AB)≤λmax⁡(A)tr⁡(B)≤tr⁡(A)tr⁡(B), where A and B are any positive semidefinite matrices.
SinceP(w) and Ωn are symmetric and nonnegative definite, the first inequality is established by tr⁡(P(w)Ωn)≤λmax⁡(P(w))tr⁡(Ωn). Let λi be eigenvalue of P(w). From the symmetric property of P(w), we know that there exists a n×n orthogonal matrix Ψ- such that P(w)=Ψ-ΥΨ-T, where Υ=diag⁡(λ1,…,λn). Then, it yields tr⁡(ΩnP(w))=tr⁡(Υ1/2Ψ-TΩnΨ-Υ1/2), where Υ1/2Ψ-TΩnΨ-Υ1/2 is symmetric and nonnegative definite. The second inequality holds because
(24)tr⁡(ΩnP(w))2≤[tr⁡(ΩnP(w))]2≤[λmax⁡(P(w))tr⁡(Ωn)]2≤C1Γ2.
For the last formula, we note that(25)tr⁡(ΩnPT(w)P(w))2=tr⁡(P(w)ΩnPT(w))2≤λmax⁡(P(w)ΩnPT(w))tr⁡(P(w)ΩnPT(w))≤λmax⁡(P2(w))λmax⁡(Ωn)tr⁡(P(w)ΩnPT(w))≤λmax⁡2(P(w))λmax⁡(Ωn)tr⁡(P(w)ΩnPT(w))≤C1Γtr⁡(P(w)ΩnPT(w)).
This completes the proof of Lemma4.Lemma 5.
LetA be a symmetric matrix and X be a random vector with zero expectation. Then, one has Var⁡(XTAX)=2tr⁡(Var⁡(X)AVar⁡(X)A), where Var⁡(·) is the operation of variance.Proof.
The proof of this lemma is provided in Lai and Xie [26].


## 3. Average Squared Error

Denote an average squared error by(26)Ln(w)=1n∥μ-μ^(w)∥k2=(μ-μ^(w))Tk(μ-μ^(w))n,
where k is a fixed positive integer which is often used to eliminate the boundary effect. Andrews [28] suggested that the k can take the value of σ-2, when errors obeyed an independent and identical distribution with variance σ2. The most common situation is k=1. The average squared error Ln(w) can be viewed as a measure of accuracy between μ^(w) and μ. Obviously, an optimal estimator can obtain the minimum value of Ln(w). In other words, we should select a weight vector w from W to make that the average squared error Ln(w) takes value that is as small as possible.In order to investigate the problem of weight selection, we give the expression of conditional expected average squared error as follows:(27)Rn(w)=E(Ln(w)∣X,Zm).From the definition ofRn(w), the following lemma can be obtained.Lemma 6.
The conditional expected average squared error can be rewritten as(28)Rn(w)=∥M(w)μ-η(w)∥k2n+ktr⁡(P(w)ΩnPT(w))n,
where M(w)=I-P(w).Proof.
A straight calculation ofLn(w) leads to
(29)nLn(w)=(μ-μ^(w))Tk(μ-μ^(w))=(μ-η(w)-P(w)(μ+e))T×k(μ-η(w)-P(w)(μ+e))=[μTMT(w)-ηT(w)-eTPT(w)]×k[M(w)μ-η(w)-P(w)eM(w)]=∥M(w)μ-η(w)∥k2-k(μ-XR-1B)TMT(w)P(w)e+keTPT(w)P(w)e-keTPT(w)M(w)(μ-XR-1B).
UsingE(eTPT(w)P(w)e∣X,Zm)=tr⁡(P(w)ΩnPT(w)) and taking conditional expectations on both sides of the above equality give rise to Lemma 6.


## 4. Thek-Class Generalized Information Criterion and Asymptotic Optimality

As the value ofμ is unknown, the average squared error Ln(w) cannot be used directly to select the weight vector w. Thus, we suggest a k-class generalized information criterion (k-GIC) to choose the weight vector. Further, we will prove that the selected weight vector from k-GIC minimizes the average squared error Ln(w).Thek-GIC for the restricted model average estimator is
(30)Cn(w)=∥Y-μ^(w)∥k2n+2ħntr⁡(P(w)Ωn),
where ħ is larger than one and satisfies assumption (35) mentioned below. The k-class generalized information criterion extends the generalized information criterion (GICλn) proposed by Shao [27]. Because the ħ can take different values, the k-GIC includes some common information criteria for model selection such as the Mallows CL criterion (k=ħ=1), the GIC criterion (k=1,ħ→∞), the FPEα criterion (k=1,ħ>1), and the BIC criterion (k=1,ħ=log⁡(n)).Lemma 7.
Ifk=ħ, we have E(Cn(w)∣X,Zm)=Rn(w)+kΓ/n.Proof.
Recalling the definition ofCn(w), one gets
(31)nCn(w)=(Y-μ^(w))Tk(Y-μ^(w))+2ktr⁡(P(w)Ωn)=[M(w)Y-η(w)]Tk[M(w)Y-η(w)]+2ktr⁡(P(w)Ωn)=2ktr⁡(P(w)Ωn)+keTMT(w)M(w)e+k(M(w)μ-η(w))TM(w)e+keTMT(w)(M(w)μ-η(w))+∥M(w)μ-η(w)∥k2.
Observe that
(32)keTMT(w)M(w)e=eT(I-P(w))Tk(I-P(w))e=keTe-keTPT(w)e-keTP(w)e+keTPT(w)P(w)e.
Notice thatE(keTe∣X,Zm)=ktr⁡(Ωn) and E(keTPT(w)e∣X,Zm)=ktr⁡(PT(w)Ωn). Moreover, we have that E(keTP(w)e∣X,Zm)=ktr⁡(P(w)Ωn) and E(keTPT(w)P(w)e∣X,Zm)=ktr⁡(P(w)ΩnPT(w)).
Thus, taking conditional expectations on both sides of (31), we obtain
(33)nE(Cn(w)∣X,Zm)=ktr⁡(Ωn)+∥M(w)μ-η(w)∥k2+ktr⁡(P(w)ΩnPT(w)).
It follows from (33) and Lemma 6 that Lemma 7 is established as desired.Lemma7 shows that Cn(w) is equivalent to the conditional expected average squared error plus an error bias. Particularly, when n approaches to infinite, Cn(w) is an unbiased estimation of Rn(w).Thek-GIC criterion is defined so as to select the optimal weight vector w^. The optimal weight vector w^ is chosen by minimizing Cn(w).Obviously, the well-posed estimators of parameters areΘ^w and Ψ^w with the weight vector w^. Under some regular conditions, we intend to demonstrate that the selection procedure is asymptotically optimal in the following sense:
(34)Ln(w^)infw∈WLn(w)→p1.If the formula (34) holds, we know that the selected weight vector w^ from Cn(w) can realize the minimum value of Ln(w). In other words, the weight vector w^ is equivalent to the selected weight vector by minimizing Ln(w) and is the optimal weight vector for μ^(w). The asymptotic optimality of Cn(w) can be established under the following assumptions.Assumptions.
Writeα-n=infw∈WnRn(w). As n→∞, we assume that
(35)lim⁡_inf⁡w∈Wħ2α-n⟶0,(36)∑w∈W‍1(nRn(w))N′⟶0
for a large ħ and any positive integer 1≤N′<∞.
The above assumptions have been employed by many literatures of model selection. For instance, the expression (35) was used by Shao [27] and the formula (36) was adopted by Li [29], Andrews [28], Shao [27], and Hansen [11].
The following lemma offers a bridge for proving the asymptotic optimality ofCn(w).Lemma 8.
We have(37)∥M(w)Y-η(w)∥k2=nLn(w)-2〈eT,P(w)e〉k+2〈eT,M(w)μ-η(w)〉k+∥e∥k2.Proof.
Usingμ^(w)=P(w)Y+η(w), Y=μ+e, and the definition of Ln(w), one obtains
(38)∥M(w)Y-η(w)∥k2=∥μ+e-P(w)Y-η(w)∥k2=∥e∥k2+∥μ-P(w)Y-η(w)∥k2+2〈eT,μ-η(w)-P(w)Y〉k=∥e∥k2+nLn(w)+2〈eT,μ-η(w)-P(w)(μ+e)〉k=∥e∥k2+nLn(w)+2〈eT,(I-P(w))μ-η(w)-P(w)e〉k=∥e∥k2+nLn(w)+2〈eT,M(w)μ-η(w)〉k-2〈eT,P(w)e〉k.
This completes the proof of Lemma8.Lemma8 and ∥Y-μ^(w)∥k2=∥M(w)Y-η(w)∥k2 imply that
(39)Cn(w)=Ln(w)+2〈eT,M(w)μ-η(w)〉kn+∥e∥k2n+2ħtr⁡(P(w)Ωn)n-2〈eT,P(w)e〉kn.The goal is to choosew^ by minimizing Cn(w). From (39), one only needs to select w^ through minimizing
(40)Ln(w)+2〈eT,M(w)μ-η(w)〉kn+2ħtr⁡(P(w)Ωn)-2〈eT,P(w)e〉kn,
where w∈W.Compared withLn(w), it is sufficient to establish that n-1〈eT,M(w)μ-η(w)〉k and n-1ħtr⁡(P(w)Ωn)-n-1〈eT,P(w)e〉k are uniformly negligible for any w∈W. More specifically, to prove (34), we need to check
(41)sup⁡w∈W〈eT,M(w)μ-η(w)〉knRn(w)→p0,(42)sup⁡w∈W|ħtr⁡(P(w)Ωn)-〈eT,P(w)e〉k|nRn(w)→p0,(43)sup⁡w∈W|Ln(w)-Rn(w)Rn(w)|→p0,
where “→p” denotes the convergence in probability.Following the idea of Li [29], we testify the main result of our work that the minimizing of k-class generalized information criterion is asymptotically optimal. Now, we state the main theorem.Theorem 9.
Under assumptions (35) and (36), the minimizing of k-class generalized information criterion Cn(w) is asymptotically optimal, namely, (34) holds.Proof.
The asymptotically optimal property ofk-GIC needs to show that (41), (42), and (43) are valid.
Firstly, we prove that (41) holds. For any ζ>0, by Chebyshev's inequality, one has
(44)Pr{sup⁡w∈W|〈eT,M(w)μ-η(w)〉k|nRn(w)>ζ}≤∑w∈W‍E[〈eT,M(w)μ-η(w)〉k]2(ζnRn(w))2,
which, by Lemma 1, is no greater than
(45)ζ-2kW∑w∈W‍n-2∥M(w)μ-η(w)∥k2(Rn(w))-2.
Recalling the definition ofRn(w), we get ∥M(w)μ-η(w)∥k2≤nRn(w). Then, (45) does not exceed ζ-2kW∑w∈W‍(nRn(w))-1. By assumption (36), one knows that (45) tends to zero in probability. Thus, (41) is established.
To prove (42), it suffices to testify that
(46)ktr⁡(P(w)Ωn)-〈eT,P(w)e〉knRn(w)→p0,(47)(ħ-k)tr⁡(P(w)Ωn)nRn(w)→p0.
By Chebyshev's inequality and Lemmas4 and 5, for any ϵ>0, we have
(48)Pr{sup⁡w∈W|tr⁡(P(w)Ωn)-〈eT,P(w)e〉nRn(w)|>ϵ}≤∑w∈W‍E[tr⁡(P(w)Ωn)-〈eT,P(w)e〉]2(ϵnRn(w))2=1ϵ2∑w∈W‍Var⁡(eTP(w)e)(nRn(w))2≤2C1Γ2ϵ2∑w∈W‍1(nRn(w))2.
From (36), we know that (48) is close to zero. Thus, (46) is reasonable.
Recalling Lemma4 and (35), it yields
(49)|(ħ-k)tr⁡(P(w)Ωn)|nRn(w)≤|C′(ħ-k)Γ|nRn(w)⟶0,
which derives (47). Then, (42) is proved.
Next, we show that the expression (43) also holds. A straightforward calculation leads to
(50)∥μ-μ^(w)∥k2-∥M(w)μ-η(w)∥k2=∥μ-η(w)-P(w)Y∥k2-∥M(w)μ-η(w)∥k2=∥μ-η(w)-P(w)μ-P(w)e∥k2-∥M(w)μ-η(w)∥k2=∥M(w)μ-η(w)-P(w)e∥k2-∥M(w)μ-η(w)∥k2=∥P(w)e∥k2-2〈μTMT(w)-ηT(w),P(w)e〉k.
From the identity (50), we know that
(51)Ln(w)-Rn(w)=-2〈μTMT(w)-ηT(w),P(w)e〉kn+∥P(w)e∥k2-ktr⁡(P(w)ΩnPT(w))n.
Obviously, the proof of (43) needs to verify that
(52)sup⁡w∈W∥P(w)e∥k2-ktr⁡(P(w)ΩnPT(w))nRn(w)→p0,sup⁡w∈W〈μTMT(w)-ηT(w),P(w)e〉knRn(w)→p0.
To prove (52), it should be noticed that tr⁡(P(w)ΩnPT(w))=E(eTPT(w)P(w)e) and ∥P(w)e∥k2=〈eTPT(w),P(w)e〉k. In addition,
(53)∥P(w)(M(w)μ-η(w))∥2≤λmax⁡2(P(w))∥M(w)μ-η(w)∥2≤∥M(w)μ-η(w)∥2.
Thus, there exist anyζ′>0 and ϵ′>0 such that
(54)Pr{sup⁡w∈W∥P(w)e∥k2-ktr⁡(P(w)ΩnPT(w))nRn(w)>ζ′}≤∑w∈W‍E[∥P(w)e∥k2-kE(eTPT(w)P(w)e)]2(nRn(w)ζ′)2=k2ζ′2∑w∈W‍Var⁡(eTPT(w)P(w)e)(nRn(w))2≤2kC1Γζ′2∑w∈W‍ktr⁡(P(w)ΩnPT(w))(nRn(w))2≤2kC1Γζ′2∑w∈W‍1(nRn(w)),(55)Pr{sup⁡w∈W〈μTMT(w)-ηT(w),P(w)e〉knRn(w)>ϵ′}≤∑w∈W‍E[〈μTMT(w)-ηT(w),P(w)e〉k]2(nRn(w)ϵ′)2≤∑w∈W‍∥P(w)(M(w)μ-η(w))∥k2kW(nRn(w)ϵ′)2≤ϵ′-2kW∑w∈W‍n-2∥M(w)μ-η(w)∥k2(Rn(w))-2.
Combining (36) and (45), one knows that both (54) and (55) tend to zero. In other words, (43) is confirmed. We conclude that the expressions (41), (42), and (43) are reasonable. This completes the proof of Theorem 9.In practice, the covariance of errors is usually unknown and needs to be estimated. However, it is difficult to build a good estimate forΩn in virtue of the special structure of Ωn. In the special case, when the random errors are independent and identical distribution with variance σ2, the consistent estimator of σ2 can be built for the constrained models (7) and (8). Let m=Q in μ^m and τ′=tr⁡(PQ), where Q corresponds to a “large” approximating model. Denote σ^Q2=(n-τ′)-1(Y-μ^Q)T(Y-μ^Q). The coming theorem will show that σ^Q2 is a consistent estimate of σ2.Theorem 10.
Ifτ′/n→0 when τ′→∞ and n→∞, we have σ^Q2→pσ2 as n→∞.Proof.
WritingΛ=D+XR-1D★, one obtains
(56)(Y-μ^Q)T(Y-μ^Q)n-τ′=eTMQen-τ′+∥MQΛ∥2n-τ′+2eTMQΛn-τ′,
where MQ=I-PQ.
SinceE(eTMQe)=σ2(n-τ′), it leads to
(57)E|eTMQe-σ2(n-τ′)|2=E|eTMQe-E(eTMQe)|2=Var⁡(eTMQe)=2σ4tr⁡(MQ).
For anyε>0, it follows from (57) that
(58)Pr{|eTMQen-τ′-σ2|>ε}≤E|eTMQe-(n-τ′)σ2|2ε2(n-τ′)2≤2σ4ε2(n-τ′)⟶0.
Equation (58) implies that (n-τ′)-1(eTMQe)→pσ2. Let IQ,t be the tth diagonal element of the “hat” matrix PQ. Then, 0≤1-IQ,t<1 is the tth diagonal element of MQ and satisfies ∑t=1n‍(1-IQ,t)=n-τ′.
Becauseϕ(zi,l,ψl) and ϕ(as,k,ψk) converge to mean square, we have E(ΛQ,t2)→0 as τ′→∞, where ΛQ,t is the tth element of Λ, t=1,…,n. Notice that
(59)E(ΛTMQΛ)(n-τ′)=1n-τ′∑t=1n‍E(ΛQ,t2MQ,t)=1n-τ′∑t=1n‍E(ΛQ,t2)[1-IQ,t]≤max⁡t{E(ΛQ,t2)}1n-τ′∑t=1n‍[1-IQ,t]=max⁡t{E(ΛQ,t2)}⟶0.
The above expression implies that the second term on the right hand of (56) approaches to zero. By the similar proof of (59), we obtain that the final term on the right-hand of (56) also tends to zero. The proof of Theorem 10 is complete.In the case of independently and identically distributed errors, if we replaceσ2 by σ^Q2 in the k-GIC, the k-GIC can be simplified to
(60)C-n(w)=∥Y-μ^(w)∥k2n+2ħσ^Q2tr⁡(P(w))n.Here, we intend to illustrate that the model selection procedure of minimizingC-n(w) is also asymptotically optimal.Theorem 11.
Assume that the random errore is i.i.d with mean zero and variance σ2. Under the conditions (35) and (36), the C-n(w) is still asymptotically valid.Proof.
Using a similar technique of deriving (39), we obtain
(61)nC-n(w)=∥Y-μ^(w)∥k2+2ħσ^Q2tr⁡(P(w))=nLn(w)+2〈eT,M(w)μ-η(w)〉k+2ħσ^Q2tr⁡(P(w))-2〈eT,P(w)e〉k+∥e∥k2=2ħ(σ^Q2-σ2)tr⁡(P(w))+2〈eT,M(w)μ-η(w)〉k+nLn(w)+2ħσ2tr⁡(P(w))-2〈eT,P(w)e〉k+∥e∥k2.
With an appropriate modification of the proofs (41)–(43), one only needs to verify
(62)sup⁡w∈Wħ|σ2-σ^Q2|tr⁡(P(w))nRn(w)→p0,
which is equivalent to showing
(63)sup⁡w∈Wħ2((1/n)|σ2-σ^Q2|tr⁡(P(w)))2Rn2(w)→p0.
From the proof of Theorem10, we have
(64)Pr{|ħeTMQen-τ′-ħσ2|>ε′Rn-1/2(w)}≤ħ2E|eTMQe-(n-τ′)σ2|2ε′2(n-τ′)2Rn(w)≤2ħ2σ4ε′2(n-τ′)Rn(w),
where ε′>0.
By assumption (35), one knows that the above equation is close to zero. Thus, we obtain {Rn(w)}-1|ħσ2-ħσ^Q2|2→p0. It follows from Lemma 3 and the definition of Rn(w) that (n-1tr⁡(P(w)))2 is no greater than Rn(w). Then, the formula (63) is confirmed. Therefore, we conclude that the minimizing of C-n(w) is also asymptotically optimal.


## 5. Monte Carlo Simulation

In this section, Monte Carlo simulations are performed to investigate the finite sample properties of the proposed restricted linear model selection. This data generating process is(65)yi=xiθ+∑l=11000‍zi,lψl+ei,subjecttoθ=-∑l=11000‍ψl+1,i=1,…,n,
where xi follows t(3) distribution, zi,1=1, and zi,l, l=2,…,1000 is independently and identically distributed N(0,1). The parameter ψl, l=1,…,1000 is determined by the rule ψl=ζl-1, where ζ is a parameter which is selected to control the population R~2=ζ2/(1+ζ2). The error ei is independent of xi and zi,l. We consider two cases of the error distribution.Case 1.
e
i is independently and identically distributed N(0,1).Case 2.
e
1
,
…
,
e
n obey multivariate normal distribution N(0,Ωn), where Ωn is a n×n dimensional covariance matrix. The jth diagonal element of Ωn is σj2 generated from uniform distribution on (0,1). The jlth (j≠l) nondiagonal element of Ωn is Ωnjl=exp⁡(-0.5|xij-xil|2), where xij and xil denote the jth and lth elements of xi, respectively.
The sample size is varied betweenn=50,100,150 and 200. The number of models is determined by M=⌊3n1/3⌋, where ⌊s⌋ stands for the integer part of s. We set ζ so that R~2 varies on a grid between 0.1 and 0.9. The number of simulation trials is Π=500. For the k-GIC, the value of k takes one and ħ adopts the effective number of parameters.
To assess the performance ofk-GIC, we consider five estimators which are (1) AIC model selection estimators (AIC), (2) BIC model selection estimators (BIC), (3) leave-one-out cross-validated model selection estimator (CV, [17]), (4) Mallows model averaging estimators (MMA, [11]), and (5)  k-GIC model selection estimator (k-GIC). Following Machado [30], the AIC and BIC are defined, respectively, as
(66)AICm=2nln⁡{1n∥Y-μ^m∥2}+2Km,BICm=2nln⁡{1n∥Y-μ^m∥2}+ln⁡(n)Km.We employ the out-of-sample prediction error to evaluate each estimator. For each replication,{yl,xl,zl}l=1100 are generated as out-of-sample observations. In the πth simulation, the prediction error is
(67)PE(π)=1100∑l=1100‍(yl-μ^l(w^))2,
where w^ is selected by one of the five methods. Then, the out-of-sample prediction error is calculated by
(68)PE=1Π∑π=1Π‍PE(π),
where Π=500 is the number of replication. Obviously, the smaller PE implies the better method of model estimator. We consider PE under homoscedastic errors at first. The prediction error calculations are summarized in Figure 1. The four panels in each graph depict results for a variety of sample sizes. In each panel, PE is displayed on the y-axis and R~2 is displayed on the x-axis.Results for Monte Carlo design.
(a)
(b)
(c)
(d)We find that thek-GIC estimators are almost the best estimators among those considered. When R~2 is very large, the MMA estimators can sometimes be marginally preferred to the k-GIC estimators. In each panel, the AIC and CV have quite similar prediction errors. For a smaller R~2, the AIC obtains a higher prediction error than CV. However, the AIC estimators yield smaller PEs than the CV estimators, when R~2 is increasing. In many situations, the PEs of BIC estimator with a large R~2 are quite poor relative to the other methods.Next, we discussPE under correlative errors and the PE calculation is summarized in Figure 2. Broadly speaking, the conclusions are similar to those found in homoscedastic cases. The k-GIC estimator frequently yields the most accurate estimators followed by the MMA estimator, and both average estimators enjoy significantly smaller PEs than the other three estimators over a large portion of the R~2 space. When R~2≤0.2, the BIC estimator outperforms the k-GIC estimator. Again, the AIC estimator is habitually the worst performing estimator with the CV being a close second in a large region of the R~2 space. Besides, their relative efficiency relies closely on sample size with the BIC estimator revealing increasing PE and the remaining four estimators showing decreasing PE, as n increases.Results for Monte Carlo design.
(a)
(b)
(c)
(d)


## 6. Conclusions

In risk investment, an important subject is to find an optimal portfolio. The commonly used techniques are the optimization methods based on the scheme of mean-variance. However, those methods are cumbersome in computing and cannot obtain the closed solutions for some complex problems. To make up the defects of mean-variance, an alternative methodology for obtaining an optimal portfolio is to use model selection. This paper attempts to develop a statistical program to consider the selection problem of optimal tracking portfolio. We build the theoretical models of tracking portfolios by constrained linear models. Then, the selection problems of optimal portfolio boil down to choosing an optimal constrained linear model.In the setting of unrestricted models or homoscedastic models, a large number of works investigate the problems of model selection. In distinction, we discuss the model selection for constrained models with dependent errors. The restricted models are estimated by the method of weighted average least squares. Thus, the selection of an optimal constrained model is equivalent to finding a series of optimal weights. We select the weights by minimizing ak-class generalized information criterion (k-GIC), which is an estimate of the average squared error from the model average fit. The procedure of selecting weights is proved to be asymptotically optimal. Through Monte Carlo simulation, the performance of k-GIC is compared against that of four other methods. It is found that the k-GIC gives the best performance in most cases.There are two limitations of our results which are open for further research. First, what is the asymptotic distribution of the parametric estimators? Second, can the theory be generalized to allow for continuous weights? These questions remain to be answered by future research. In this work, we mainly adopt the method of regression analysis to solve the selection problem. In fact, some alternative mathematical tools can also be employed to explore the theoretical properties of model selection. For example, the optimal model can be selected by the methods of linear optimization or quadratic programming and we can apply the techniques of linear functional analysis and stochastic control to consider the inferences of parametric estimator. Besides, we mention that the applications of this study can also be extended to some other fields including risk management, ruin theory, and factor analysis.



---

*Source: 692472-2014-05-21.xml*