tests: better float comparisons
update tests to use new floating point check
remove annoying output
use numdiff