fail2ban/fail2ban — Issue #17: Ban after too few attempts due to erroneous log rotation detection
Author: cepheid666
Created: 2011-12-30 07:16:09+00:00
State: open

(Posted here based on discussion through the fail2ban-users mailing list)

I have fail2ban monitoring a couple of BB error logs that are causing some havok with its logfile tracking. Specifically, the YaBB error logs aren't arbitrary-length, constant-append logs like most error logs (e.g. apache error log), but are instead fixed-length, constantly-overwritten logs - they are capped at N lines (say N = 100) and every new (last) line written to the file causes the oldest (first) line to be deleted.

Fail2ban's log rotation detection is barfing on this kind of log - it thinks the log is being rotated every time it sees a new line being written. As a result, it basically "rescans" the file every time, but without resetting the hack-attempt counter, causing each line to be counted multiple times.

As an example:
Say the maxretry for this jail is set to 10 attempts. Fail2ban notices the first matching hack attempt in the error log, and sets the counter to 1. A new hack attempt is written to the file; upon the next scan, fail2ban thinks the logfile got rotated, and rescans the entire file - it now sees 2 entries, and adds BOTH entries to the counter, which is now at 3 attempts. The same thing happens on the 3rd attempt, whereupon the counter is now set to 6, and for the 4th, where the counter now hits 10 and fail2ban locks out the offending IP. In this way, fail2ban thinks it sees 10 matching lines even though it actually saw only 4, because it rescanned the logfile after each "rotation" without resetting the counter.

The one obvious workaround for this type of file is to set maxretry to the correct "n-sum" number, where if n is the number of attempts we actually want to catch, we set maxretry = n + (n-1) + (n-2) + ... + 1. This is rather a poor hack but it'd work.

One possible solution is to disable logrotation detection for this type of log, whereupon fail2ban would ignore logrotation and simply look at the latest lines being written. Unfortunately, since the file is of "constant" length, it would be difficult to keep track of how many new lines were added and which ones they are.

Another potential (possibly best) solution is to reset the attempt counter when log rotation is "detected" for this type of log; thus, when fail2ban rescans the entire logfile after "rotation," it will still see the correct number of matching errors instead of accumulating them the way it does now. The reset action would need to be a user-settable flag for only this type of logfile, since it would not be desirable to do such a reset on regular rotating logs; thus, one could add a flag such as "cyclinglog = true" to the jail configuration, and if this flag is true, the attempt counter is reset upon rotation detection, but if false (default), everything would continue as normal.

It's a rare use case but IMHO should be treated.

Thanks!