Braindump on large dataset handling.

..mostly "this will be hard and I don't know how to do it."