Improve memory usage when checking for NaNs

To resolve issue #1053 by @cancan101.

> Currently, this appears in numerous places: np.any(np.isnan(X)). This requires creating a large array of booleans matching the shape of X. More efficient would be: np.isnan(np.min(X)).

> See: http://stackoverflow.com/questions/6736590/fast-check-for-nan-in-numpy
Something bad happened here. You lost the inf check and instead doubled up on the nan check,