More refactoring and explanation.
Changes which never got pushed at the last data hackathon. They are on
the blog however so we had better push them.