lots of changes, reworking data io, separating functions out so that they are testable and more readable