heavy data munging refactoring ongoing - broken commit