logstash/logstash — pull request #877 by fbird (2013-12-16 08:26:52+00:00)
Title: add limit_duplicate filter plugin

Hi, I wrote a plugin to limit the duplicated logs by dropping the duplicated ones in a period. The reason I wrote this plugin is that in our system, there is a lot of duplicated logs(exception stacktrace) appeared very frequently, they are important that we can not ignore it, but we don't want it to flood our logs, so this filter can only allow the first log passed, and drop the followed duplicated ones in a period time.