Ridge regression optimization using a harmonious approach
A critical component of ridge regression (RR) is determining the optimal ridge parameter value, lambda, where lambda >= 0. Improper selection of lambda not only generates an under- or overfitted model but also leads to incorrect conclusions in inter-model comparison studies such as between RR, PLS, PCR and other modeling methods. Several methods for determining the optimal RR model are evaluated in this paper. For example, the commonly used ridge trace is identified as subjective and impractical. A direct calculation method from the literature yields over- or underfitted RR models with lambda either too small or to large respectively. Methods for determining lambda based on a harmonious approach are discussed. The harmonious approach optimizes lambda by inspecting the bias/variance tradeoff. Of the methods investigated, plotting a variance indicator against a bias measure to yield an L-curve appears not only to simplify selection of lambda but also to reduce the chance of obtaining an under- or overfitted RR model. It is shown with four data sets that the L-curve harmonious approach consistently provides good models. The effective rank of models is also discussed in conjunction with the harmony/parsimony tradeoff. Copyright (C) 2005 John Wiley A Sons, Ltd.
ridge regression bias/variance tradeoff l-curve effective rank parsimony harmonious model multivariate calibration degrees of freedom