massive refactoring and simplification of dataset construction