Make sure we use std::isfinite and not the macro on GCC/Clang