Fixed some signed/unsigned warnings in rlm_expr

GCC 4.8 gave a warning when compiling. Since we are sure that outlen is
at least 3, len cannot be negative.

As a bonus, some indention with spaces has been replaced by indention
with tabs.