22 nov feature
account for corrupted data for the moment