rewrite comparison to avoid negative unsigned numbers

Note that GCC -O2 is happy to optimize away (x<0) when x is an
unsigned quantity.  This was actually occurring in CHECKLEN(0),
causing the compiler to issue a warning.
Mixing signed and unsigned quantities in MIN() upset GCC's tender soul.