Non-Asymptotic Properties of Sparse Approximate Factor
ModelsBenjamin Poignard, Osaka University, Graduate School of Engineering ScienceYoshikazu Terada, Osaka University, Graduate School of Engineering Science - RIKEN AIPThe factor modelling approach consists in summarizing the information through a small number ofvariables called factors. The quantity of interest is the variance covariance matrix Σ which is decomposedasΣ = ΛΛT + Ψ,
where Ψ is the p × p variance covariance matrix of the idiosyncratic variables and Λ is the p × m matrix ofthe loading factors. Although the factor model approach aims at reducing the statistical complexity, thetotal number of parameters remains significant for large p-dimensional random vectors. As a consequence,
the total number of parameters to estimate in the idiosyncratic variance covariance matrix is O(p2 ). To
foster sparsity, Bai and Liao (2016) proposed a Lasso/Scad penalization procedure based on a Gaussianquasi maximum likelihood approach for estimating a sparse idiosyncratic variance covariance matrix.
They derive asymptotic bounds and consistency results for the penalized estimate Ψ̂.
In our study, we provide a finite-sample theoretical analysis of sparse idiosyncratic variance covariancematrices Ψ together with the conditions to satisfy the support recovery property using the theoreticalframework developed by Loh and Wainwright (2017). To do so, we propose a two-step estimation, where
the statistical criterion is given by
1log(|Σ(Λ, Ψ)|) + tr(ŜΣ(Λ, Ψ)−1 ) ,(Λ̃, Ψ̃)= arg min {Gn (Λ, Ψ)}, with Gn (Λ, Ψ) = 2nΛ,ΨΨ̂=arg min{Fn (Ψ; Λ̃, Ψ̃) + p(γn , vec(Ψ))}, whereΨ:Ψ0,g(vec(Ψ))≤R F (Ψ; Λ̃, Ψ̃) = vec(Ψ − Ψ̃)0 {∇2nvec(Ψ)vec(Ψ)0 Gn (Λ̃, Ψ̃)}vec(Ψ − Ψ̃),
where Ŝ is the sample variance covariance estimate and vec(.) is the vectorization operator that stacks thecolumns of any matrix on top of one another into a vector. In a first step, both the loading factor matrix
and idiosyncratic matrix are obtained based on a Gaussian maximum likelihood estimator. Conditionallyon these first step estimates, the regularized idiosyncratic matrix corresponds to the solution of a minimum
distance criterion plus the regularizer. This sparse estimator is thus based on a least square type loss
function weighted by the Hessian matrix of the first step Gaussian likelihood function. The regularization2
procedure is performed by the regularizer p(γn , .) : Rp → R, where γn is the regularization parameter,
which depends on the sample size, and enforce a particular type of sparse structure in the solution Ψ̂ andR > 0 is a supplementary regularization parameter. Due to the potential non-convexity of this penalty,
2
we include the side condition g(vec(Ψ)) ≥ kvec(Ψ)k1 with g : Rp → R a convex function, and R toensure the existence of local/global optima.
Under the restricted strong convexity of the unpenalized loss function Fn (.; Λ̃, Ψ̃) and regularity conditions on the regularizer, we prove non-asymptotic error bounds on the regularized idiosyncratic matrix
estimator. Moreover, based on the primal-dual witness method, we establish variable selection consistency, including the case when the regularizer is non-convex. These theoretical results are supported byempirical studies.

References
Bai, J. and Liao, K. (2016). Efficient estimation of approximate factor models via penalized maximum
likelihood. Journal of Econometrics, Vol. 191, 1-18.
Loh, P.L. and Wainwright, M.J. (2017). Support recovery without incoherence: a case for non-convex
regularization. The Annals of Statistics, Vol. 45, No. 6, 2455-2482.