fail2ban/fail2ban — pull request #256 by yarikoptic (2013-06-14 03:27:11+00:00)
Title: ENH: better fail2ban-regex and heavydebug logging level

Primarily it was triggered by need to debug why anchoring of asterisk wasn't working for @grooverdan and then got swamped into it... but now it is quite neat -- output is as ugly as before (so noone should get surprised) but no junk dumped on the screen if not asked for. But if asked for -- now there would be even more junk ;)

Sample output of a clean run -- "Wrong" is added just as an example ignore regex. Mention also that now IPs do not have their own section (I think it is not needed) but are printed in --verbose mode after corresponding fail regexes:

```
$> python2.7 ./fail2ban-regex -v testcases/files/logs/asterisk config/filter.d/asterisk.conf Wrong

Running tests
=============

Use ignoreregex line : Wrong
Use failregex file : config/filter.d/asterisk.conf
Use log file : testcases/files/logs/asterisk

Results
=======

Failregex: 13 total
|- #) [# of hits] regular expression
| 1) [0] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - Wrong password$
| 2) [2] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - No matching peer found$
| 1.2.3.4 Mon Feb 13 17:18:22 2012
| 1.2.3.4 Tue Mar 26 15:47:54 2013
| 3) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - Username/auth name mismatch$
| 1.2.3.4 Mon Feb 13 17:21:21 2012
| 4) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - Device does not match ACL$
| 1.2.3.4 Mon Feb 13 17:32:01 2012
| 5) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - Peer is not supposed to register$
| 1.2.3.4 Mon Feb 13 17:34:10 2012
| 6) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - ACL error \(permit/deny\)$
| 1.2.3.4 Mon Feb 13 17:36:23 2012
| 7) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Registration from '[^']*' failed for '(:\d+)?' - Not a local domain$
| 1.2.3.4 Mon Jun 10 18:15:03 2013
| 8) [1] ^\[\]\s*NOTICE(?:\[\d+\])\[\S+\] \S+: Call from '[^']*' \(:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
| 1.2.3.4 Tue Feb 05 23:44:42 2013
| 9) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Host failed to authenticate as '[^']*'$
| 1.2.3.4 Mon Feb 13 17:53:59 2012
| 10) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: No registration for peer '[^']*' \(from \)$
| 1.2.3.4 Mon Feb 13 17:39:20 2012
| 11) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Host failed MD5 authentication for '[^']*' \([^)]+\)$
| 1.2.3.4 Mon Feb 13 17:44:26 2012
| 12) [1] ^\[\]\s*NOTICE(?:\[\d+\]) \S+: Failed to authenticate user [^@]+@\S*$
| 1.2.3.4 Mon Feb 13 17:37:07 2012
| 13) [1] ^\[\]\s*SECURITY(?:\[\d+\]) \S+: SecurityEvent="InvalidAccountID",EventTV="[\d-]+",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="\d+",SessionID="0x[\da-f]+",LocalAddress="IPV[46]/(UD|TC)P/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UD|TC)P//\d+"$
| 1.2.3.4 Mon May 13 07:10:53 2013
`-

Ignoreregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] Wrong
`-

Date template hits:
|- [# of hits] date format
| [0] MONTH Day Hour:Minute:Second
| [0] WEEKDAY MONTH Day Hour:Minute:Second Year
| [0] WEEKDAY MONTH Day Hour:Minute:Second
| [0] Year/Month/Day Hour:Minute:Second
| [0] Day/Month/Year Hour:Minute:Second
| [0] Day/Month/Year2 Hour:Minute:Second
| [0] Day/MONTH/Year:Hour:Minute:Second
| [0] Month/Day/Year:Hour:Minute:Second
| [195] Year-Month-Day Hour:Minute:Second
| [0] Year.Month.Day Hour:Minute:Second
| [0] Day-MONTH-Year Hour:Minute:Second[.Millisecond]
| [0] Day-Month-Year Hour:Minute:Second
| [0] Month-Day-Year Hour:Minute:Second[.Millisecond]
| [0] TAI64N
| [0] Epoch
| [0] ISO 8601
| [0] Hour:Minute:Second
| [0]
| [0] MonthDayYear Hour:Minute:Second
| [0] Month-Day-Year Hour:Minute:Second
`-

Lines: 14 lines, 1 ignored, 13 matched, 0 missed
|- Ignored line(s):
| [2012-02-13 17:21:54] NOTICE[1638] chan_sip.c: Registration from '' failed for '1.2.3.4' - Wrong password
`-
```

Ideally majority of this functionality should move somewhere (e.g. this code reuses cmdline + setting of logging from fail2ban-testcases) and also could serve as a good start to handle that meta information for sample log lines as we talked about in I forgotten which issue ;)