Make the b64 list be all integers

This will be faster on PyPy.