Package: atheris
Atheris: A Coverage-Guided, Native Python FuzzerAtheris is a coverage-guided Python fuzzing engine. It supports fuzzing of Python code, but also native extensions written for CPython. Atheris is based off of libFuzzer. When fuzzing native code, Atheris can be used in combination with Address Sanitizer or Undefined Behavior Sanitizer to catch extra bugs.Installation InstructionsAtheris supports Linux (32- and 64-bit) and Mac OS X, Python versions 3.6-3.10.You can install prebuilt versions of Atheris with pip:pip3installatherisThese wheels come with a built-in libFuzzer, which is fine for fuzzing Python
code. If you plan to fuzz native extensions, you may need to build from source
to ensure the libFuzzer version in Atheris matches your Clang version.Building from SourceAtheris relies on libFuzzer, which is distributed with Clang. If you have a sufficiently new version ofclangon your path, installation from source is as simple as:# Build latest release from sourcepip3install--no-binaryatherisatheris# Build development code from sourcegitclonehttps://github.com/google/atheris.gitcdatheris
pip3install.If you don't haveclanginstalled or it's too old, you'll need to download and build the latest version of LLVM. Follow the instructions in Installing Against New LLVM below.MacApple Clang doesn't come with libFuzzer, so you'll need to install a new version of LLVM from head. Follow the instructions in Installing Against New LLVM below.Installing Against New LLVM# Building LLVMgitclonehttps://github.com/llvm/llvm-project.gitcdllvm-project
mkdirbuildcdbuild
cmake-DLLVM_ENABLE_PROJECTS='clang;compiler-rt'-G"Unix Makefiles"../llvm
make-j10# This step is very slow# Installing AtherisCLANG_BIN="$(pwd)/bin/clang"pip3install<whatever>Using AtherisExample#!/usr/bin/python3importatheriswithatheris.instrument_imports():importsome_libraryimportsysdefTestOneInput(data):some_library.parse(data)atheris.Setup(sys.argv,TestOneInput)atheris.Fuzz()When fuzzing Python, Atheris will report a failure if the Python code under test throws an uncaught exception.Python coverageAtheris collects Python coverage information by instrumenting bytecode.
There are 3 options for adding this instrumentation to the bytecode:You can instrument the libraries you import:withatheris.instrument_imports():importfoofrombarimportbazThis will cause instrumentation to be added tofooandbar, as well as
any libraries they import.Or, you can instrument individual functions:@atheris.instrument_funcdefmy_function(foo,bar):print("instrumented")Or finally, you can instrument everything:atheris.instrument_all()Put this right beforeatheris.Setup(). This will find every Python function
currently loaded in the interpreter, and instrument it.
This might take a while.Atheris can additionally instrument regular expression checks, e.g.re.search.
To enable this feature, you will need to add:atheris.enabled_hooks.add("RegEx")To your script before your code callsre.compile.
Internally this will import theremodule and instrument the necessary functions.
This is currently an experimental feature.Similarly, Atheris can instrument str methods; currently onlystr.startswithandstr.endswithare supported. To enable this feature, addatheris.enabled_hooks.add("str"). This is currently an experimental feature.Why am I getting "No interesting inputs were found"?You might see this error:ERROR: no interesting inputs were found. Is the code instrumented for coverage? Exiting.You'll get this error if the first 2 calls toTestOneInputdidn't produce any
coverage events. Even if you have instrumented some Python code,
this can happen if the instrumentation isn't reached in those first 2 calls.
(For example, because you have a nontrivialTestOneInput). You can resolve
this by adding anatheris.instrument_funcdecorator toTestOneInput,
usingatheris.instrument_all(), or moving yourTestOneInputfunction into an
instrumented module.Visualizing Python code coverageExamining which lines are executed is helpful for understanding the
effectiveness of your fuzzer. Atheris is compatible withcoverage.py: you can run your fuzzer using
thecoverage.pymodule as you would for any other Python program. Here's an
example:python3-mcoveragerunyour_fuzzer.py-atheris_runs=10000# Times to runpython3-mcoveragehtml(cdhtmlcov&&python3-mhttp.server8000)Coverage reports are only generated when your fuzzer exits gracefully. This
happens if:you specify-atheris_runs=<number>, and that many runs have elapsed.your fuzzer exits by Python exception.your fuzzer exits bysys.exit().No coverage report will be generated if your fuzzer exits due to a
crash in native code, or due to libFuzzer's-runsflag (use-atheris_runs).
If your fuzzer exits via other methods, such as SIGINT (Ctrl+C), Atheris will
attempt to generate a report but may be unable to (depending on your code).
For consistent reports, we recommend always using-atheris_runs=<number>.If you'd like to examine coverage when running with your corpus, you can do
that with the following command:python3 -m coverage run your_fuzzer.py corpus_dir/* -atheris_runs=$(( 1 + $(ls corpus_dir | wc -l) ))This will cause Atheris to run on each file in<corpus-dir>, then exit.
Note: atheris use empty data set as the first input even if there is no empty file in<corpus_dir>.
Importantly, if you leave off the-atheris_runs=$(ls corpus_dir | wc -l), no
coverage report will be generated.Using coverage.py will significantly slow down your fuzzer, so only use it for
visualizing coverage; don't use it all the time.Fuzzing Native ExtensionsIn order for fuzzing native extensions to be effective, your native extensions
must be instrumented. SeeNative Extension Fuzzingfor instructions.Structure-aware FuzzingAtheris is based on a coverage-guided mutation-based fuzzer (LibFuzzer). This
has the advantage of not requiring any grammar definition for generating inputs,
making its setup easier. The disadvantage is that it will be harder for the
fuzzer to generate inputs for code that parses complex data types. Often the
inputs will be rejected early, resulting in low coverage.Atheris supports custom mutators(as offered by LibFuzzer)to produce grammar-aware inputs.Example (Atheris-equivalent of theexample in the LibFuzzer docs):@atheris.instrument_funcdefTestOneInput(data):try:decompressed=zlib.decompress(data)exceptzlib.error:returniflen(decompressed)<2:returntry:ifdecompressed.decode()=='FU':raiseRuntimeError('Boom')exceptUnicodeDecodeError:passTo reach theRuntimeErrorcrash, the fuzzer needs to be able to produce inputs
that are valid compressed data and satisfy the checks after decompression.
It is very unlikely that Atheris will be able to produce such inputs: mutations
on the input data will most probably result in invalid data that will fail at
decompression-time.To overcome this issue, you can define a custom mutator function (equivalent toLLVMFuzzerCustomMutator).
This example produces valid compressed data. To enable Atheris to make use of
it, pass the custom mutator function to the invocation ofatheris.Setup.defCustomMutator(data,max_size,seed):try:decompressed=zlib.decompress(data)exceptzlib.error:decompressed=b'Hi'else:decompressed=atheris.Mutate(decompressed,len(decompressed))returnzlib.compress(decompressed)atheris.Setup(sys.argv,TestOneInput,custom_mutator=CustomMutator)atheris.Fuzz()As seen in the example, the custom mutator may request Atheris to mutate data
usingatheris.Mutate()(this is equivalent toLLVMFuzzerMutate).You can experiment withcustom_mutator_example.pyand see that without the mutator Atheris would not be able to find the crash,
while with the mutator this is achieved in a matter of seconds.$python3example_fuzzers/custom_mutator_example.py--no_mutator[...]#2      INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 37Mb#524288 pulse  cov: 2 ft: 2 corp: 1/1b lim: 4096 exec/s: 262144 rss: 37Mb#1048576        pulse  cov: 2 ft: 2 corp: 1/1b lim: 4096 exec/s: 349525 rss: 37Mb#2097152        pulse  cov: 2 ft: 2 corp: 1/1b lim: 4096 exec/s: 299593 rss: 37Mb#4194304        pulse  cov: 2 ft: 2 corp: 1/1b lim: 4096 exec/s: 279620 rss: 37Mb[...]$python3example_fuzzers/custom_mutator_example.py[...]INFO:foundLLVMFuzzerCustomMutator(0x7f9c989fb0d0).Disabling-len_controlbydefault.[...]#2      INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 37Mb#3      NEW    cov: 4 ft: 4 corp: 2/11b lim: 4096 exec/s: 0 rss: 37Mb L: 10/10 MS: 1 Custom-#12     NEW    cov: 5 ft: 5 corp: 3/21b lim: 4096 exec/s: 0 rss: 37Mb L: 10/10 MS: 7 Custom-CrossOver-Custom-CrossOver-Custom-ChangeBit-Custom-===UncaughtPythonexception:===RuntimeError:Boom
Traceback(mostrecentcalllast):File"example_fuzzers/custom_mutator_example.py",line62,inTestOneInputraiseRuntimeError('Boom')[...]Custom crossover functions (equivalent toLLVMFuzzerCustomCrossOver) are also
supported. You can pass the custom crossover function to the invocation ofatheris.Setup. See its usage incustom_crossover_fuzz_test.py.Structure-aware Fuzzing with Protocol Bufferslibprotobuf-mutatorhas
bindings to use it together with Atheris to perform structure-aware fuzzing
using protocol buffers.See the documentation foratheris_libprotobuf_mutator.Integration with OSS-FuzzAtheris is fully supported byOSS-Fuzz, Google's continuous fuzzing service for open source projects. For integrating with OSS-Fuzz, please seehttps://google.github.io/oss-fuzz/getting-started/new-project-guide/python-lang.APITheatherismodule provides three key functions:instrument_imports(),Setup()andFuzz().In your source file, import all libraries you wish to fuzz inside awith atheris.instrument_imports():-block, like this:# library_a will not get instrumentedimportlibrary_awithatheris.instrument_imports():# library_b will get instrumentedimportlibrary_bGenerally, it's best to importatherisfirst and then import all other libraries inside of awith atheris.instrument_imports()block.Next, define a fuzzer entry point function and pass it toatheris.Setup()along with the fuzzer's arguments (typicallysys.argv). Finally, callatheris.Fuzz()to start fuzzing. You must callatheris.Setup()beforeatheris.Fuzz().instrument_imports(include=[], exclude=[])include: A list of fully-qualified module names that shall be instrumented.exclude: A list of fully-qualified module names that shall NOT be instrumented.This should be used together with awith-statement. All modules imported in
said statement will be instrumented. However, because Python imports all modules
only once, this cannot be used to instrument any previously imported module,
including modules required by Atheris. To add coverage to those modules, useinstrument_all()instead.A full list of unsupported modules can be retrieved as follows:importsysimportatherisprint(sys.modules.keys())instrument_func(func)func: The function to instrument.This will instrument the specified Python function and then returnfunc. This
is typically used as a decorator, but can be used to instrument individual
functions too. Note that thefuncis instrumented in-place, so this will
affect all call points of the function.This cannot be called on a bound method - call it on the unbound version.instrument_all()This will scan over all objects in the interpreter and callinstrument_funcon
every Python function. This works even on core Python interpreter functions,
something whichinstrument_importscannot do.This function is experimental.Setup(args, test_one_input, internal_libfuzzer=None)args: A list of strings: the process arguments to pass to the fuzzer, typicallysys.argv. This argument list may be modified in-place, to remove arguments consumed by the fuzzer.
Seethe LibFuzzer docsfor a list of such options.test_one_input: your fuzzer's entry point. Must take a singlebytesargument. This will be repeatedly invoked with a single bytes container.internal_libfuzzer: Indicates whether libfuzzer will be provided by atheris or by an external library (seenative_extension_fuzzing.md). If unspecified, Atheris will determine this
automatically. If fuzzing pure Python, leave this asTrue.Fuzz()This starts the fuzzer. You must have calledSetup()before calling this function. This function does not return.In many casesSetup()andFuzz()could be combined into a single function, but they are
separated because you may want the fuzzer to consume the command-line arguments it handles
before passing any remaining arguments to another setup function.FuzzedDataProviderOften, abytesobject is not convenient input to your code being fuzzed. Similar to libFuzzer, we provide a FuzzedDataProvider to translate these bytes into other input forms.You can construct the FuzzedDataProvider with:fdp=atheris.FuzzedDataProvider(input_bytes)The FuzzedDataProvider then supports the following functions:defConsumeBytes(count:int)Consumecountbytes.defConsumeUnicode(count:int)Consume unicode characters. Might contain surrogate pair characters, which according to the specification are invalid in this situation. However, many core software tools (e.g. Windows file paths) support them, so other software often needs to too.defConsumeUnicodeNoSurrogates(count:int)Consume unicode characters, but never generate surrogate pair characters.defConsumeString(count:int)Alias forConsumeBytesin Python 2, orConsumeUnicodein Python 3.defConsumeInt(int:bytes)Consume a signed integer of the specified size (when written in two's complement notation).defConsumeUInt(int:bytes)Consume an unsigned integer of the specified size.defConsumeIntInRange(min:int,max:int)Consume an integer in the range [min,max].defConsumeIntList(count:int,bytes:int)Consume a list ofcountintegers ofsizebytes.defConsumeIntListInRange(count:int,min:int,max:int)Consume a list ofcountintegers in the range [min,max].defConsumeFloat()Consume an arbitrary floating-point value. Might produce weird values likeNaNandInf.defConsumeRegularFloat()Consume an arbitrary numeric floating-point value; never produces a special type likeNaNorInf.defConsumeProbability()Consume a floating-point value in the range [0, 1].defConsumeFloatInRange(min:float,max:float)Consume a floating-point value in the range [min,max].defConsumeFloatList(count:int)Consume a list ofcountarbitrary floating-point values. Might produce weird values likeNaNandInf.defConsumeRegularFloatList(count:int)Consume a list ofcountarbitrary numeric floating-point values; never produces special types likeNaNorInf.defConsumeProbabilityList(count:int)Consume a list ofcountfloats in the range [0, 1].defConsumeFloatListInRange(count:int,min:float,max:float)Consume a list ofcountfloats in the range [min,max]defPickValueInList(l:list)Given a list, pick a random valuedefConsumeBool()Consume eitherTrueorFalse.