refactoring, tests, old datafile compatibility.