complete main dataset building, refactoring