tests: Rework the fuzzy comparison macros

We need a vec3 fuzzy comparison macro, but we also need to clean up the
mess that the existing macros have become.

The only bits that needs to be placed under a compiler check are the
typeof() declarations and the actual epsilon condition; the rest of the
macros are perfectly compiler-agnostic.

Once we moved the epsilon check out to its own macro, the other macros
fall in line fairly simply.
tests/triangle: Use the vec3 fuzzy comparison macro
tests/plane: Use the fuzzy vec3 comparison macro