Prepare for real data consumption

To get a better feel for this PR, I suggest reading through each commit. I also added some top-level documentation on the classes which will help to understand the motivation/logic a bit more.