Oauthlib always gives us bytes.

Not only was this a waste of time, the logic was actually bad so it
reintroduced a bug. Which sucks.

This resolves the bug mentioned by @michaelhelmick in #36, and was actually found by @zvin in #35.

I'm putting this here for code review, I suspect I'll merge it myself once @sigmavirus and @ib-lundgren take a look.