Cleanups and minor optimizations

I tried improving the performance, readability and robustness of the codebase. After each commit, the code complied cleanly on my box, each change is isolated in a small commit, so if you disagree with something, it's easy to perform cherry-pick.