Manual revert of 0f83c5a4.

(Automatic revert didn't work any more because of too many intervening
overlapping changes.)

The 0f83c5a4 tried avoiding coercing Inf/NaN to IV/UV, but it also
introduced 20-25% slowdown in floating point operations, because of
repeated redundant calling of Perl_isinfnan().  There was noticeable
compiler dependency: the effect was especially visible with gcc 4.8.
Please see the thread "significant slowdown in float arithmetic" by
Dave Mitchell, Message-Id: 20141025155040.GR5204@iabyn.com