Fuzzing Django Applications With the Atheris Fuzzing Engine
By: Tomasz Nowak
Tags: Python, Security, Testing, Django, Fuzzing

Atheris hispida. Photo by Bree Mc, soulsurvivor08 at flickr.com, CC BY 2.0, via Wikimedia Commons

Fuzzing (fuzz testing) is a technique of finding flaws in software by providing it unexpected input. It was successful to find numerous serious security issues. Some others could have been found a lot earlier if fuzz testing was conducted.

This week Google released the Atheris fuzzing engine which allows programs written in Python to be tested with libFuzzer, an actively developed library for coverage-guided fuzz testing. Atheris is simple to install and trivial to use.

I will show you how to test a web application based on the Django framework and what can be achieved with a little effort.

TL;DR

It doesn’t find all views automatically. Prepare a good test corpus to help it find routes in your application, and a dictionary with relevant phrases or strings to use when fuzzing. The smaller part of the framework you harness, the more expectations you fail to meet. You may not want to test the URL resolver or the middleware, but views may depend on them. Get ready for surprising results. The more work you put into preparation, the better results you get. Implement shortcuts. Mock up external dependencies. If your view resolves domain names or connects to an external database you won’t get far in a reasonable time.

Preparations

Follow the official README:

Take care to install the latest version of Clang. In my case switching from Clang 11 to 12 increased the testing speed (“exec/s”) twofold.

Approach 1: Fuzzing complete page URLs

In this approach we create a django.test.Client and ask Atheris to visit random pages of our project:

#!/usr/bin/env python3 import os, django

# I assume this variable is already set (e.g. in your shell)

# os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")

django.setup() from django.test import Client

client = Client() def TestOneInput(data):

url = '/' + data.decode('latin2') # convert random bytes to string

# surprisingly these chars result in ValueError: Invalid IPv6 URL

url = url.replace('[', '%5B').replace(']', '%5D')

response = client.get(url)

if response.status_code not in [200, 301, 302, 404]:

# suspicious! log the HTTP status and the request path

print(response.status_code, url)

raise RuntimeError("Badness!")

elif response.status_code not in [302, 404]:

# not suspicious, but let's see what pages it finds

print(response.status_code, url) import atheris

import sys atheris.Setup(sys.argv, TestOneInput)

atheris.Fuzz()

First, we initialize the Django framework and create the test client. Then we use it to obtain a response for the given path. To simulate accessing your website as a logged-in user, you can call “client.force_login” method described in the Client documentation.

The last step is to check the HTTP status code in the response. If it’s unexpected than we immediately stop executing the script to investigate the case. If it’s typical, but Atheris found a properly working page we log the page address.

What’s expected or not depends on your application. I decided to ignore responses with HTTP redirects (statuses 301, 302) as well as the “Page not found” response (status 404).

Let’s start our script:

$ ALLOWED_HOSTS='["testserver"]' ./fuzz1.py -max_len=150

INFO: Configured for Python tracing with opcodes.

INFO: Seed: 3353455860

INFO: Loaded 2 modules (1024 inline 8-bit counters): 512 [0x558092250230, 0x558092250430), 512 [0x558092234510, 0x558092234710),

INFO: Loaded 2 PC tables (1024 PCs): 512 [0x5580924b5c70,0x5580924b7c70), 512 [0x5580924b8490,0x5580924ba490),

INFO: A corpus is not provided, starting from an empty corpus

#2 INITED cov: 20202 ft: 20202 corp: 1/1b exec/s: 0 rss: 153Mb

NEW_FUNC[1/42]: 0x5580924b7e69

NEW_FUNC[2/42]: 0x5580924b7e81

#3 NEW cov: 22119 ft: 23051 corp: 2/5b lim: 4 exec/s: 0 rss: 154Mb L: 4/4 MS: 1 CrossOver-

#4 NEW cov: 22123 ft: 23055 corp: 3/9b lim: 4 exec/s: 0 rss: 154Mb L: 4/4 MS: 1 CopyPart-

#9 REDUCE cov: 22123 ft: 23055 corp: 3/8b lim: 4 exec/s: 0 rss: 154Mb L: 3/4 MS: 5 CopyPart-ChangeBit-ShuffleBytes-CrossOver-CrossOver-

NEW_FUNC[1/1]: 0x5580927a98fd

#11 NEW cov: 22137 ft: 23079 corp: 4/12b lim: 4 exec/s: 11 rss: 154Mb L: 4/4 MS: 2 ChangeByte-ChangeByte-

#12 NEW cov: 22137 ft: 23099 corp: 5/16b lim: 4 exec/s: 12 rss: 154Mb L: 4/4 MS: 1 ChangeBinInt-

#18 NEW cov: 22137 ft: 23178 corp: 6/20b lim: 4 exec/s: 18 rss: 154Mb L: 4/4 MS: 1 CopyPart-

#35 NEW cov: 22137 ft: 23186 corp: 7/24b lim: 4 exec/s: 35 rss: 154Mb L: 4/4 MS: 2 ChangeBit-ChangeByte-

#41 NEW cov: 22137 ft: 23199 corp: 8/28b lim: 4 exec/s: 41 rss: 154Mb L: 4/4 MS: 1 CrossOver-

NEW_FUNC[1/1]: 0x5580927a9901

#50 NEW cov: 22214 ft: 23276 corp: 9/31b lim: 4 exec/s: 25 rss: 154Mb L: 3/4 MS: 4 ShuffleBytes-ChangeByte-EraseBytes-ChangeBinInt-

#64 pulse cov: 22214 ft: 23276 corp: 9/31b lim: 4 exec/s: 32 rss: 154Mb

#82 REDUCE cov: 22214 ft: 23276 corp: 9/30b lim: 4 exec/s: 27 rss: 154Mb L: 3/4 MS: 2 EraseBytes-ChangeByte-

What we see it a standard output from libFuzzer. If you are new to libFuzzer, the official tutorial explains it in details. I used “-max_len=150” to limit the length of the random value; be sure to take a look at libFuzzer options.

(Edit: An update has been released and the output now contains a proper function name instead of its unreadable memory address.)

As the script runs, the word “NEW” gets rarer and rarer in its output, which means that it’s getting hard to find new code to test. We could help the fuzzer by providing a directory of interesting inputs (corpus). In our case, those inputs would be names of existing pages. LibFuzzer will use that directory to store interesting inputs it finds itself, so it won’t start from scratch next time:

$ mkdir CORPUS1

$ ALLOWED_HOSTS='["testserver"]' ./fuzz1.py -max_len=150 CORPUS1

INFO: Configured for Python tracing with opcodes.

INFO: Seed: 17267508

(...)

INFO: 0 files found in CORPUS1

INFO: A corpus is not provided, starting from an empty corpus

#2 INITED cov: 20202 ft: 20202 corp: 1/1b exec/s: 0 rss: 153Mb

#3 NEW cov: 20206 ft: 20263 corp: 2/2b lim: 4 exec/s: 0 rss: 153Mb L: 1/1 MS: 1 ShuffleBytes-

(...)

#286 NEW cov: 22718 ft: 24298 corp: 28/72b lim: 4 exec/s: 31 rss: 155Mb L: 2/4 MS: 1 InsertByte-

^C KeyboardInterrupt: stopping. $ ALLOWED_HOSTS='["testserver"]' ./fuzz1.py -max_len=150 CORPUS1

INFO: Configured for Python tracing with opcodes.

INFO: Seed: 96071814

(...)

INFO: 26 files found in CORPUS1

INFO: seed corpus: files: 26 min: 1b max: 4b total: 70b rss: 152Mb

#27 INITED cov: 22718 ft: 24286 corp: 19/50b exec/s: 27 rss: 154Mb

#31 NEW cov: 22718 ft: 24303 corp: 20/54b lim: 4 exec/s: 31 rss: 154Mb L: 4/4 MS: 4 EraseBytes-CrossOver-ChangeBit-CrossOver-

We are not limited to GET requests. A form handling functionality can be tested with the “client.post(path, data, content_type, …)” function. I recommend reading the function’s documentation as interpretation of “data” in Django depends on the “content_type” argument.

Approach 2: Fuzzing Django views

It takes a lot of time for the fuzzer to discover all views in your application, so let’s direct our test at specific ones. We can use the RequestFactory and call the view directly.

#!/usr/bin/env python3 import os, django

django.setup() from django.test.client import RequestFactory

rf = RequestFactory() from myapp.views import ProductListView as view_under_test def TestOneInput(data):

url = '/' + data.decode('latin2') # convert bytes to string

# surprisingly these chars result in ValueError: Invalid IPv6 URL

url = url.replace(']', '%5D').replace('[', '%5B')

request = rf.get(url)

response = view_under_test.as_view()(request)

if response.status_code not in [200]:

print(response.status_code, url)

raise RuntimeError("Badness!")

elif response.status_code not in [404]:

print(response.status_code, url) import atheris

import sys atheris.Setup(sys.argv, TestOneInput)

atheris.Fuzz()

Bypassing the resolver and middleware results in a big speed up. For my application a crash is reported in the very first test case:

$ ALLOWED_HOSTS='["testserver"]' ./fuzz2.py -max_len=150 -only_ascii=1 CORPUS2 INFO: Configured for Python tracing with opcodes.

INFO: Seed: 1307295576

INFO: Loaded 2 modules (1024 inline 8-bit counters): 512 [0x558c20687db0, 0x558c20687fb0), 512 [0x558c1faebf90, 0x558c1faec190),

INFO: Loaded 2 PC tables (1024 PCs): 512 [0x558c2077e880,0x558c20780880), 512 [0x558c207810a0,0x558c207830a0),

INFO: 159 files found in CORPUS2 === Uncaught Python exception: ===

AttributeError: 'Request' object has no attribute 'LANGUAGE_CODE'

Traceback (most recent call last):

File "./fuzz2.py", line 31, in TestOneInput

response = view_under_test.as_view()(request)

(...)

File "/usr/local/lib/python3.8/site-packages/rest_framework/request.py", line 414, in __getattr__

return self.__getattribute__(attr) ==1861== ERROR: libFuzzer: fuzz target exited

SUMMARY: libFuzzer: fuzz target exited

MS: 0 ; base unit: 0000000000000000000000000000000000000000 artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709

Base64:

However, this not a bug. The view under test needs the LANGUAGE_CODE value, which is normally provided by the middleware. We can provide it by setting a constant value as an attribute after creating the request:

request = rf.get(url)

setattr(request, 'LANGUAGE_CODE', 'pl')

But we can also fuzz the value together with the page address. To do that we need to switch to FuzzedDataProvider and ask it for two Unicode strings:

def TestOneInput(data):

fdp = atheris.FuzzedDataProvider(data)

url = '/' + fdp.ConsumeUnicode(40)

url = url.replace(']', '%5D').replace('[', '%5B')

request = rf.get(url)

setattr(request, 'LANGUAGE_CODE', fdp.ConsumeUnicode(2))

(...)

FuzzedDataProvider provides a range of methods that return values of different types which may be more suitable for your use case, so make sure to take a look at the documentation. For example, if we wanted to constrain the value of a variable to a list of predefined values we could use PickValueInList:

language_code = fdp.PickValueInList(['cz', 'sk', 'pl'])

Final thoughts

“This is way too slow.” Optimize. Checks the settings and disable the middleware classes you don’t use. Switch to SQLite. Look through articles about benchmarking and improving Python performance (for example, on the brilliant PythonSpeed blog).

“Reviewing the code will take less time than optimizing for fuzzing.” That’s possible, but preparing automated tests is a one-time investment. If you integrate fuzz testing in your CI pipeline you will benefit from testing your application for regressions after every change.

“It’s amazing how easy it is to set up.” I’m glad you liked it!

Please let me know in the comments if it worked for your project or you have ideas for improvement in the web applications fuzzing techniques. Or just say ‘Hello!’ :-)