Fix --enable-debug

The check for -lm generates a warning which makes configure fail if we
set -Werror before the check for it