Repository: fail2ban/fail2ban

Issue:
apache-auth.conf no longer detects anything
I have installed fail2ban 0.8.10-1.el6 from EPEL on Centos 6.4 (32-bit) but I just can't make it detect Apache Basic Auth failures.

I have following test log:

```
[Thu Jul 11 01:21:41 2013] [error] [client 194.228.20.113] user  not found: /
[Thu Jul 11 01:21:43 2013] [error] [client 194.228.20.113] user dsfasdf not found: /
[Thu Jul 11 01:21:45 2013] [error] [client 194.228.20.113] user dsfasdf not found: /
[Thu Jul 11 01:21:45 2013] [error] [client 194.228.20.113] user dsfasdf not found: /
[Thu Jul 11 01:21:47 2013] [error] [client 194.228.20.113] user dsfasdf not found: /
[Thu Jul 11 01:21:49 2013] [error] [client 194.228.20.113] user dsfasdffeg not found: /
[Thu Jul 11 01:21:51 2013] [error] [client 194.228.20.113] user dfgdgf not found: /
[Thu Jul 11 01:21:52 2013] [error] [client 194.228.20.113] user dfg not found: /
[Thu Jul 11 01:21:53 2013] [error] [client 194.228.20.113] user dfg not found: /
[Thu Jul 11 01:21:55 2013] [error] [client 194.228.20.113] user gsdfgdsfg not found: /
[Thu Jul 11 01:21:56 2013] [error] [client 194.228.20.113] user dfgers not found: /
[Thu Jul 11 01:21:56 2013] [error] [client 194.228.20.113] user dgrferd not found: /
[Thu Jul 11 01:29:42 2013] [error] [client 194.228.20.113] user dgrferd not found: /
[Thu Jul 11 01:29:44 2013] [error] [client 194.228.20.113] user fds not found: /
[Thu Jul 11 01:29:45 2013] [error] [client 194.228.20.113] user  not found: /
[Thu Jul 11 01:29:46 2013] [error] [client 194.228.20.113] user fd not found: /
[Thu Jul 11 07:39:58 2013] [error] [client 194.228.20.113] user dsf not found: /
[Thu Jul 11 08:06:50 2013] [error] [client 194.228.20.113] user dsf not found: /
```

Running `fail2ban-regex -v "/root/apache.log" "/etc/fail2ban/filter.d/apache-auth.conf"` produces:

```
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second

Running tests
=============

Use regex file : /etc/fail2ban/filter.d/apache-auth.conf
Use log file   : /root/apache.log


Results
=======

Failregex: 0 total
|- #) [# of hits] regular expression
|  1) [0] ^\[[^]]+\] \[error\] \[client <HOST>\] user .* (authentication failure|not found|password mismatch)\s*$
`-

Ignoreregex: 0 total

Summary
=======

Sorry, no match

Look at the above section 'Running tests' which could contain important
information.
```

It works just fine with older `apache-auth.conf` from 495f2dd877d0f1db6260af46a1e162dbea956cf6.

```
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second
Got time using template MONTH Day Hour:Minute:Second
Matched time template MONTH Day Hour:Minute:Second

Running tests
=============

Use regex file : /etc/fail2ban/filter.d/apache-auth-old.conf
Use log file   : /root/apache.log


Results
=======

Failregex: 18 total
|- #) [# of hits] regular expression
|  1) [0] [[]client <HOST>[]] user .* authentication failure
|  2) [18] [[]client <HOST>[]] user .* not found
|  3) [0] [[]client <HOST>[]] user .* password mismatch
`-

Ignoreregex: 0 total

Summary
=======

Addresses found:
[1]
[2]
    194.228.20.113 (Thu Jul 11 01:21:41 2013)
    194.228.20.113 (Thu Jul 11 01:21:43 2013)
    194.228.20.113 (Thu Jul 11 01:21:45 2013)
    194.228.20.113 (Thu Jul 11 01:21:45 2013)
    194.228.20.113 (Thu Jul 11 01:21:47 2013)
    194.228.20.113 (Thu Jul 11 01:21:49 2013)
    194.228.20.113 (Thu Jul 11 01:21:51 2013)
    194.228.20.113 (Thu Jul 11 01:21:52 2013)
    194.228.20.113 (Thu Jul 11 01:21:53 2013)
    194.228.20.113 (Thu Jul 11 01:21:55 2013)
    194.228.20.113 (Thu Jul 11 01:21:56 2013)
    194.228.20.113 (Thu Jul 11 01:21:56 2013)
    194.228.20.113 (Thu Jul 11 01:29:42 2013)
    194.228.20.113 (Thu Jul 11 01:29:44 2013)
    194.228.20.113 (Thu Jul 11 01:29:45 2013)
    194.228.20.113 (Thu Jul 11 01:29:46 2013)
    194.228.20.113 (Thu Jul 11 07:39:58 2013)
    194.228.20.113 (Thu Jul 11 08:06:50 2013)
[3]

Date template hits:
72 hit(s): MONTH Day Hour:Minute:Second
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second Year
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second
0 hit(s): Year/Month/Day Hour:Minute:Second
0 hit(s): Day/Month/Year Hour:Minute:Second
0 hit(s): Day/Month/Year2 Hour:Minute:Second
0 hit(s): Day/MONTH/Year:Hour:Minute:Second
0 hit(s): Month/Day/Year:Hour:Minute:Second
0 hit(s): Year-Month-Day Hour:Minute:Second
0 hit(s): Year.Month.Day Hour:Minute:Second
0 hit(s): Day-MONTH-Year Hour:Minute:Second[.Millisecond]
0 hit(s): Day-Month-Year Hour:Minute:Second
0 hit(s): Month-Day-Year Hour:Minute:Second[.Millisecond]
0 hit(s): TAI64N
0 hit(s): Epoch
0 hit(s): ISO 8601
0 hit(s): Hour:Minute:Second
0 hit(s): <Month/Day/Year@Hour:Minute:Second>
0 hit(s): MonthDayYear Hour:Minute:Second
0 hit(s): Month-Day-Year Hour:Minute:Second

Success, the total number of match is 18

However, look at the above section 'Running tests' which could contain important
information.
```



Discussion:
quite right. http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/aaa/mod_auth_basic.c?view=markup line 359


Resolving patch:
diff --git a/testcases/files/config/apache-auth/digest.py b/testcases/files/config/apache-auth/digest.py
new file mode 100755
--- /dev/null
+++ b/testcases/files/config/apache-auth/digest.py
@@ -0,0 +1,159 @@
+#!/bin/env python
+import requests
+import md5
+
+
+def auth(v):
+
+    ha1 = md5.new(username + ':' + realm + ':' + password).hexdigest()
+    ha2 = md5.new("GET:" + url).hexdigest()
+    
+    #response = md5.new(ha1 + ':' + v['nonce'][1:-1] + ':' + v['nc'] + ':' + v['cnonce'][1:-1]
+    #                  + ':' + v['qop'][1:-1] + ':' + ha2).hexdigest()
+    
+    nonce = v['nonce'][1:-1]
+    nc=v.get('nc') or ''
+    cnonce = v.get('cnonce') or ''
+    opaque = v.get('opaque') or ''
+    qop = v['qop'][1:-1]
+    algorithm = v['algorithm']
+    response = md5.new(ha1 + ':' + nonce + ':' + nc + ':' + cnonce + ':' + qop + ':' + ha2).hexdigest()
+    
+    p = requests.Request('GET', host + url).prepare()
+    #p.headers['Authentication-Info'] = response 
+    p.headers['Authorization'] = """
+        Digest username="%s",
+        algorithm="%s",
+        realm="%s",
+        uri="%s",
+        nonce="%s",
+        cnonce="",
+        nc="",
+        qop=%s,
+        response="%s"
+    """ % ( username, algorithm, realm, url, nonce, qop, response )
+#        opaque="%s",
+    print p.method, p.url, p.headers
+    s =  requests.Session()
+    return s.send(p)
+
+def preauth():
+    r = requests.get(host + url)
+    print r
+    r.headers['www-authenticate'].split(', ')
+    return dict([ a.split('=',1) for a in r.headers['www-authenticate'].split(', ') ])
+
+
+url='/digest/'
+host = 'http://localhost:801'
+
+v = preauth()
+
+username="username"
+password = "password"
+print v
+
+realm = 'so far away'
+r = auth(v)
+
+realm = v['Digest realm'][1:-1]
+
+# [Sun Jul 28 21:27:56.549667 2013] [auth_digest:error] [pid 24835:tid 139895297222400] [client 127.0.0.1:57052] AH01788: realm mismatch - got `so far away' but expected `digest private area'
+
+
+algorithm = v['algorithm']
+v['algorithm'] = 'super funky chicken'
+r = auth(v)
+
+# [Sun Jul 28 21:41:20 2013] [error] [client 127.0.0.1] Digest: unknown algorithm `super funky chicken' received: /digest/
+
+print r.status_code,r.headers, r.text
+v['algorithm'] = algorithm
+
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+nonce = v['nonce']
+v['nonce']=v['nonce'][5:-5]
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+# [Sun Jul 28 21:05:31.178340 2013] [auth_digest:error] [pid 24224:tid 139895539455744] [client 127.0.0.1:56906] AH01793: invalid qop `auth' received: /digest/qop_none/
+
+
+v['nonce']=nonce[0:11] + 'ZZZ' + nonce[14:]
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+#[Sun Jul 28 21:18:11.769228 2013] [auth_digest:error] [pid 24752:tid 139895505884928] [client 127.0.0.1:56964] AH01776: invalid nonce b9YAiJDiBAZZZ1b1abe02d20063ea3b16b544ea1b0d981c1bafe received - hash is not d42d824dee7aaf50c3ba0a7c6290bd453e3dd35b
+
+
+url='/digest_time/'
+v=preauth()
+
+import time
+time.sleep(1)
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+# Obtained by putting the following code in modules/aaa/mod_auth_digest.c
+# in the function initialize_secret
+#    {
+#       const char *hex = "0123456789abcdef";
+#       char secbuff[SECRET_LEN * 4];
+#       char *hash = secbuff;
+#       int idx;
+
+#       for (idx=0; idx<sizeof(secret); idx++) {
+#       *hash++ = hex[secret[idx] >> 4];
+#       *hash++ = hex[secret[idx] & 0xF];
+#       }
+#       *hash = '\0';
+#       /* remove comment makings in below for apache-2.4+ */
+#       ap_log_error(APLOG_MARK, APLOG_NOTICE, 0, s, /*  APLOGNO(11759) */ "secret: %s", secbuff);
+#   }
+
+
+import sha
+import binascii
+import base64
+import struct
+
+apachesecret = binascii.unhexlify('497d8894adafa5ec7c8c981ddf9c8457da7a90ac')
+s = sha.sha(apachesecret)
+
+v=preauth()
+
+print v['nonce']
+realm = v['Digest realm'][1:-1]
+
+(t,) = struct.unpack('l',base64.b64decode(v['nonce'][1:13]))
+
+# whee, time travel
+t = t + 5540
+
+timepac = base64.b64encode(struct.pack('l',t))
+
+s.update(realm)
+s.update(timepac)
+
+v['nonce'] =  v['nonce'][0] + timepac + s.hexdigest() + v['nonce'][-1]
+
+print v
+
+r = auth(v)
+#[Mon Jul 29 02:12:55.539813 2013] [auth_digest:error] [pid 9647:tid 139895522670336] [client 127.0.0.1:58474] AH01777: invalid nonce 59QJppTiBAA=b08983fd166ade9840407df1b0f75b9e6e07d88d received - user attempted time travel
+print r.status_code,r.headers, r.text
+
+url='/digest_onetime/'
+v=preauth()
+
+# Need opaque header handling in auth
+r = auth(v)
+print r.status_code,r.headers, r.text
+r = auth(v)
+print r.status_code,r.headers, r.text


Test patch: