NaN, Inf, and -Inf are no longer converted to NA