added the report, some data scrubbing, added a clustering metric