Adjustments found by using python3 -bb

Hat tip @mitsuhiko: https://twitter.com/mitsuhiko/status/420215233123463168

Note that I suspect the removal some of this code might cause some
regression in some edge cases but that's okay: Bugs build community.

Also I'm aware that decoding a base64 bytestring from UTF-8 is
asinine; doing it that way for consistency and politics.