Parallel loading of datasets

Datasets that consist of more than one file now load in parallel. In addition, within each file, the data is transformed in parallel.

Experiments with inserting into the DB in parallel were unsuccessful and resulted in deadlock. My best guess is that MongoDB is locking and holding up the agents.

This is now ready for merge.