added code to remove bad data