Transition away from Java serialization for storing state on disk or in Zookeeper

This is all @hausdorff's work -- it addresses #419 and #525, and supersedes #497

I've rebased his commits into the current master, and validated that the tests pass.