A coordinate majorization descent algorithm for l(1) penalized learning
The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving (1) penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the (1) penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.
glmnet coordinate decent majorization-minimization lasso