refactored and improved data cleaning
back to square one

need a better identification strategy...