Repository: jazzband/django-axes

Issue:
Improvements to axes.decorators.query2str
Scanning through the code, I noticed that this method had the following note: `The length of the output is limited to max_length to avoid a DoS attack.`

If performance is a legitimate concern in this method due to potential DoS attack, I noticed a couple of issues with the implementation.

I ran a few tests using the Python `timeit` module. Using a dictionary containing 10,000 items and iterating over the call to query2str() 1,000 times I got an average over 3 runs of 5.70709357833645 seconds. By removing the `[:max_length]` limiter, I got an average over 3 runs of 5.64547695267053 seconds. This difference is somewhat negligible, and could be written off to me running this on a busy machine.

Now, for one first-pass optimization, I moved the `[:max_length]` limiter, changing the line from `return '\n'.join(kvs)[:max_length]` to `return '\n'.join(kvs[:max_length])`; limiting the number of items before the join. With this change, I got an average of 5.573779275335255 seconds over 3 runs. But, there is one more big optimization to be made here: list comprehensions.

Switching to a list comprehension, I was able to get an average of 5.097600340998421 seconds over 3 runs. Now, while 0.6 seconds doesn't seem like a whole lot, keep in mind that this is one thousand iterations with dictionaries containing ten thousand keys. Increasing the number of iterations or size of the dictionary, this gap would widen significantly. If this method is a true concern, I think this would be a worthwhile change.

The test code follows:

``` python
import timeit

setup = """
import six

PASSWORD_FORM_FIELD = 'password'

def query2str(items, max_length=1024):
    kvs = []
    for k, v in items:
        if k != PASSWORD_FORM_FIELD:
            kvs.append(six.u('%s=%s') % (k, v))

    return '\\n'.join(kvs)[:max_length]

dict = {x: x for x in range(0, 10000)}
"""

timeit.timeit('query2str(dict.items())', setup, number=1000)


setup = """
import six

PASSWORD_FORM_FIELD = 'password'

def query2str(items, max_length=1024):
    return '\\n'.join([six.u('%s=%s' % (k, v)) for k, v in six.iteritems(items) if k != PASSWORD_FORM_FIELD][:max_length])
dict = {x: x for x in range(0, 10000)}
"""
timeit.timeit('query2str(dict)', setup, number=1000)
```

Note that this also moves the iteration inside `query2str()` method and uses the Python 2/3 compatible `six.iteritems()` instead of just calling `request.POST.items()` and `request.GET.items()` outside.



Discussion:
@tarkatronic Looks great, send a PR

Reading over #73, it looks like this changes the proposed behavior ever so slightly. Where that is meant to limit the overall size of saved data, this limits the number of elements which can be saved.

I was able to handle both scenarios by changing the line to:

``` python
    return '\\n'.join([six.u('%s=%s' % (k, v)) for k, v in six.iteritems(items) if k != PASSWORD_FORM_FIELD][:int(max_length/2)])[:max_length]
```

This takes approximately the same amount of time, and achieves the improvement requested in #73. If that approach works for you, I'll send a PR right over.

Additionally of note, I just tested this in Python 2.7.9 (my previous tests were all in 3.4.3) and found that the difference in timing is even more pronounced, as 2.x is just that much slower.

I'd also like to add Python 3.4 and Django 1.8.5 to the testing matrix, if you've got no problem with that.


Resolving patch:
diff --git a/axes/decorators.py b/axes/decorators.py
--- a/axes/decorators.py
+++ b/axes/decorators.py
@@ -160,15 +160,11 @@ def query2str(items, max_length=1024):
 
     If there's a field called "password" it will be excluded from the output.
 
-    The length of the output is limited to max_length to avoid a DoS attack.
+    The length of the output is limited to max_length to avoid a DoS attack via excessively large payloads.
     """
 
-    kvs = []
-    for k, v in items:
-        if k != PASSWORD_FORM_FIELD:
-            kvs.append(six.u('%s=%s') % (k, v))
-
-    return '\n'.join(kvs)[:max_length]
+    return '\n'.join(['%s=%s' % (k, v) for k, v in six.iteritems(items)
+                      if k != PASSWORD_FORM_FIELD][:int(max_length/2)])[:max_length]
 
 
 def ip_in_whitelist(ip):
@@ -398,11 +394,11 @@ def check_request(request, login_unsuccessful):
             for attempt in attempts:
                 attempt.get_data = '%s\n---------\n%s' % (
                     attempt.get_data,
-                    query2str(request.GET.items()),
+                    query2str(request.GET),
                 )
                 attempt.post_data = '%s\n---------\n%s' % (
                     attempt.post_data,
-                    query2str(request.POST.items())
+                    query2str(request.POST)
                 )
                 attempt.http_accept = request.META.get('HTTP_ACCEPT', '<unknown>')
                 attempt.path_info = request.META.get('PATH_INFO', '<unknown>')
@@ -461,8 +457,8 @@ def create_new_failure_records(request, failures):
         'user_agent': ua,
         'ip_address': ip,
         'username': username,
-        'get_data': query2str(request.GET.items()),
-        'post_data': query2str(request.POST.items()),
+        'get_data': query2str(request.GET),
+        'post_data': query2str(request.POST),
         'http_accept': request.META.get('HTTP_ACCEPT', '<unknown>'),
         'path_info': request.META.get('PATH_INFO', '<unknown>'),
         'failures_since_start': failures,
@@ -485,8 +481,8 @@ def create_new_trusted_record(request):
         user_agent=ua,
         ip_address=ip,
         username=username,
-        get_data=query2str(request.GET.items()),
-        post_data=query2str(request.POST.items()),
+        get_data=query2str(request.GET),
+        post_data=query2str(request.POST),
         http_accept=request.META.get('HTTP_ACCEPT', '<unknown>'),
         path_info=request.META.get('PATH_INFO', '<unknown>'),
         failures_since_start=0,


Test patch:
diff --git a/axes/tests.py b/axes/tests.py
--- a/axes/tests.py
+++ b/axes/tests.py
@@ -10,7 +10,7 @@
 
 from axes.decorators import COOLOFF_TIME
 from axes.decorators import FAILURE_LIMIT
-from axes.models import AccessLog
+from axes.models import AccessAttempt, AccessLog
 from axes.signals import user_locked_out
 from axes.utils import reset
 
@@ -23,7 +23,7 @@ class AccessAttemptTest(TestCase):
     LOCKED_MESSAGE = 'Account locked: too many login attempts.'
     LOGIN_FORM_KEY = '<input type="submit" value="Log in" />'
 
-    def _login(self, is_valid_username=False, is_valid_password=False, user_agent='test-browser'):
+    def _login(self, is_valid_username=False, is_valid_password=False, user_agent='test-browser', **kwargs):
         """Login a user. A valid credential is used when is_valid_username is True,
         otherwise it will use a random string to make a failed login.
         """
@@ -45,11 +45,13 @@ def _login(self, is_valid_username=False, is_valid_password=False, user_agent='t
         else:
             password = 'invalid-password'
 
-        response = self.client.post(admin_login, {
+        post_data = {
             'username': username,
             'password': password,
             'this_is_the_login_form': 1,
-        }, HTTP_USER_AGENT=user_agent)
+        }
+        post_data.update(kwargs)
+        response = self.client.post(admin_login, post_data, HTTP_USER_AGENT=user_agent)
 
         return response
 
@@ -205,3 +207,10 @@ def test_lockout_by_combination_user_and_ip(self):
         # But we should get one now
         response = self._login(is_valid_username=True, is_valid_password=False)
         self.assertContains(response, self.LOCKED_MESSAGE)
+
+    def test_log_data_truncated(self):
+        """Tests that query2str properly truncates data to the max_length (default 1024)
+        """
+        extra_data = {string.ascii_letters * x: x for x in range(0, 1000)}  # An impossibly large post dict
+        self._login(**extra_data)
+        self.assertEquals(len(AccessAttempt.objects.latest('id').post_data), 1024)