math: support invalid ld80 representations in fpclassify

apparently gnulib requires invalid long double representations
to be handled correctly in printf so we classify them according
to how the fpu treats them: bad inf is nan, bad nan is nan,
bad normal is nan and bad subnormal/zero is minimal normal