remove logs
added deduplicate filter