Ids now deleted from set if not replied to