Make 'i' and 'len' the same type as 'digestlen'

Because they're being compared to it.  Fixes #561
Code indention fixes in rlm_expr