added more processing to data cleaning