Fast exact leave-one-out cross-validation of sparse least-squares support vector machines
Leave-one-out cross-validation has been shown to give an almost unbiased estimator of the generalisation properties of statistical models, and therefore provides a sensible criterion for model selection and comparison. In this paper we show that exact leave-one-out cross-validation of sparse Least-Squares Support Vector Machines (LS-SVMs) can be implemented with a computational complexity of only O(?n2) O ( ? n 2 ) floating point operations, rather than the O(?2n2) O ( ? 2 n 2 ) operations of a nave implementation, where ? is the number of training patterns and n is the number of basis vectors. As a result, leave-one-out cross-validation becomes a practical proposition for model selection in large scale applications. For clarity the exposition concentrates on sparse least-squares support vector machines in the context of non-linear regression, but is equally applicable in a pattern recognition setting.
cross-validation least-squares support vector machine model selection