Efficient Parallel Stochastic Variance Reduction Algorithms for LargeScale SVD In recent years, the stochastic variance reduced gradient (SVRG) method has been used for singular vector decomposition (SVD), called VRSVD, and VRSVD only uses simple variance reduction iterations, yet can obtain an exponential convergence rate to the optimal solution. Considering that recent years have witnessed great application prospects in largescale SVD, this paper proposes two efficient asynchronous parallel stochastic variance reduction algorithms. Moreover we analyze their convergence properties in theory, and both dense and sparse asynchronous parallel variants can converge with the rate of O(1T). Finally, we extend our algorithms for the SVD problems with multiple singular vectors. Our extensive experimental results show that our algorithms attain ideal parallel speedup ratios and achieve the same PSNR values in a shorter time, and thus they can be widely used in practice.

A Stochastic Variance Reduced Extragradient Method for Sparse Machine Learning Problems This paper mainly considers the optimization problem of minimizing the average of a number of smooth convex functions and a general convex function capable of proximal mapping. In particular, we consider nonsmooth functions, and assume that the smooth convex function is strongly convex, which often occur in data mining and deep learning. For nonsmooth largescale highdimensional machine learning problems, this paper introduces both the variance reduction technique in SVRG and extragradient descent, and proposes an efficient proximal stochastic variance reduced extragradient algorithm, called EProxSVRG. From our theoretical results, we know that in the expected form, EProxSVRG converges to the optimum at a linear convergence rate for strongly convex problems, and the convergence guarantee for nonstrongly convex problems is also provided. Moreover, EProxSVRG achieves the same oracle complexity as ProxSVRG. Finally, our experimental results show that our algorithm can obtain an improved performance compared with ProxSVRG and Katyusha.