Checkpoint of directory-based dataset aggregation
And the rest ...
Added unit test for directory-based aggregation
Fix for transaction left uncommitted.
Tidyups ... mostly cosmetic