crypto: fix utf8/utf-8 encoding check

Normalize the encoding in getEncoding() before using it. Fixes a
"AssertionError: Cannot change encoding" exception when the caller
mixes "utf8" and "utf-8".

Fixes #5655.

Suggested reviewer: @indutny. This time for real.