For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all $2^p$ subsets? In 'Applied Survival Analysis,' Hosmer &amp; Lemeshow make reference to Kuk's method, but I cannot find the original paper. Can anyone describe this method, or, even better, a more modern technique? One may assume normally distributed errors.


Question: numpy float: 10x slower than builtin in arithmetic operations?
Answer: The answer is quite simple:  the memory allocation might be part of it, but the biggest problem is that arithmetic operations for numpy scalars is done using "ufuncs" which are meant to be fast for several hundred values not just 1.  There is some overhead in choosing the correct function to call and setting up the loops.  Overhead which is un-necessary for scalars.  

It was easier to just have the scalars be converted to 0-d arrays and then passed to the corresponding numpy ufunc then write separate calculation methods for each of the many different scalar types that NumPy supports.  

The intent was that optimized versions of the scalar math would be added to the type-objects in C.  This could still happen, but it never has happened because no-one has been motivated enough to do it.   Possibly because the work-around is to convert numpy scalars to Python scalars which do have optimized arithmetic.