Let (X, Y ) be a random pair taking values in R p × R. In the socalled single-index model, one has Y = f (θ T X) + W , where f is an unknown univariate measurable function, θ is an unknown vector in R d , and W denotes a random noise satisfying E[W |X] = 0. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes 1 Corresponding author. 2 Research partially supported by the French "Agence Nationale pour la Recherche" under grant ANR-09-BLAN-0128 "PARCIMONIE". larger than the number of observations ("p larger than n" paradigm). To circumvent this difficulty, we consider the single-index model estimation problem from a sparsity perspective using a PAC-Bayesian approach. On the theoretical side, we offer a sharp oracle inequality, which is more powerful than the best known oracle inequalities for other common procedures of single-index recovery. The proposed method is implemented by means of the reversible jump Markov chain Monte Carlo technique and its performance is compared with that of standard procedures.
In high-dimensional settings where β * is sparse, REF establish PAC-Bayesian analysis for sparse single index models.
Sparse single-index model
Journal of Machine Learning Research 14 (2013) 243-280
Mathematics Computer Science