Fast Algorithms for Sparse ReducedRank Regression We consider a reformulation of ReducedRank Regression (RRR) and Sparse ReducedRank Regression (SRRR) as a nonconvex nondifferentiable function of a single of the two matrices usually introduced to parametrize lowrank matrix learning problems. We study the behavior of proximal gradient algorithms for the minimization of the objective. In particular, based on an analysis of the geometry of the problem, we establish that a proximal Polyakxc5x81ojasiewicz inequality is satisfied in a neighborhood of the set of optima under a condition on the regularization parameter. We consequently derive linear convergence rates for the proximal gradient descent with line search and for related algorithms in a neighborhood of the optima. Our experiments show that our formulation leads to much faster learning algorithms for RRR and especially for SRRR.

A Stochastic Variance Reduced Extragradient Method for Sparse Machine Learning Problems This paper mainly considers the optimization problem of minimizing the average of a number of smooth convex functions and a general convex function capable of proximal mapping. In particular, we consider nonsmooth functions, and assume that the smooth convex function is strongly convex, which often occur in data mining and deep learning. For nonsmooth largescale highdimensional machine learning problems, this paper introduces both the variance reduction technique in SVRG and extragradient descent, and proposes an efficient proximal stochastic variance reduced extragradient algorithm, called EProxSVRG. From our theoretical results, we know that in the expected form, EProxSVRG converges to the optimum at a linear convergence rate for strongly convex problems, and the convergence guarantee for nonstrongly convex problems is also provided. Moreover, EProxSVRG achieves the same oracle complexity as ProxSVRG. Finally, our experimental results show that our algorithm can obtain an improved performance compared with ProxSVRG and Katyusha.