WIP: Better py3k support

ping @warrd, feel free to send me PRs so we can work in parallel

Outstanding issues:

1. Travis integration

I have opened an issue on Numpy for the odd requirement of `rb+` instead of `rb` on some of our files:

https://github.com/numpy/numpy/issues/4118