Natural coordinate descent algorithm for L1-penalised regression in generalised linear models
The problem of finding the maximum likelihood estimates for the regression coefficients in generalised linear models with an ?1 ? 1 sparsity penalty is shown to be equivalent to minimising the unpenalised maximum log-likelihood function over a box with boundary defined by the ?1 ? 1 -penalty parameter. In one-parameter models or when a single coefficient is estimated at a time, this result implies a generic soft-thresholding mechanism which leads to a novel coordinate descent algorithm for generalised linear models that is entirely described in terms of the natural formulation of the model and is guaranteed to converge to the true optimum. A prototype implementation for logistic regression tested on two large-scale cancer gene expression datasets shows that this algorithm is efficient, particularly so when a solution is computed at set values of the ?1 ? 1 -penalty parameter as opposed to along a regularisation path. Source code and test data are available from http://tmichoel.github.io/glmnat/.
coordinate descent algorithm generalised linear model logistic regression penalised regression