Package: pythonfuzz
pythonfuzz: coverage-guided fuzz testing for pythonPythonFuzz is coverage-guidedfuzzerfor testing python packages.Fuzzing for safe languages like python is a powerful strategy for finding bugs like unhandled exceptions, logic bugs,
security bugs that arise from both logic bugs and Denial-of-Service caused by hangs and excessive memory usage.Fuzzing can be seen as a powerful and efficient strategy in real-world software in addition to classic unit-tests.UsageFuzz TargetThe first step is to implement the following function (also called a fuzz
target). Here is an example of a simple fuzz function for the built-inhtmlmodulefromhtml.parserimportHTMLParserfrompythonfuzz.mainimportPythonFuzz@PythonFuzzdeffuzz(buf):try:string=buf.decode("ascii")parser=HTMLParser()parser.feed(string)exceptUnicodeDecodeError:passif__name__=='__main__':fuzz()Features of the fuzz target:fuzz will call the fuzz target in an infinite loop with random data (according to the coverage guided algorithm) passed tobuf( in a separate process).The function must catch and ignore any expected exceptions that arise when passing invalid input to the tested package.The fuzz target must call the test function/library with with the passed buffer or a transformation on the test buffer
if the structure is different or from different type.Fuzz functions can also implement application level checks to catch application/logical bugs - For example:
decode the buffer with the testable library, encode it again, and check that both results are equal. To communicate the results
the result/bug the function should throw an exception.pythonfuzz will report any unhandled exceptions as crashes as well as inputs that hit the memory limit specified to pythonfuzz
or hangs/they run more the the specified timeout limit per testcase.RunningThe next step is to download pythonfuzz and then run your fuzzerpipinstallpythonfuzz
pythonexamples/htmlparser/fuzz.py#394378 NEW     cov: 608 corp: 24 exec/s: 1119 rss: 10.73828125 MBsubclassesofParserBasemustoverrideerror()Traceback(mostrecentcalllast):File"/Users/yevgenyp/fuzzitdev/pythonfuzz/pythonfuzz/fuzzer.py",line21,inworkertarget(buf)File"examples/htmlparser/fuzz.py",line12,infuzzpassFile"/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py",line111,infeedself.goahead(0)File"/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py",line179,ingoaheadk=self.parse_html_declaration(i)File"/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py",line264,inparse_html_declarationreturnself.parse_marked_section(i)File"/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_markupbase.py",line159,inparse_marked_sectionself.error('unknown status keyword %r in marked section'%rawdata[i+3:j])File"/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_markupbase.py",line34,inerror"subclasses of ParserBase must override error()")NotImplementedError:subclassesofParserBasemustoverrideerror()crashwaswrittentocrash-dbfa437e5956643645681fe6a3ac76997be0b29a7c7af82d88c8c390f379502dcrash=3c215b63612121This example quickly finds an an unhandled exception/flow in a few minutes.CorpusPythonFuzz will generate and test various inputs in an infinite loop.corpusis optional directory and will be used to
save the generated testcases so later runs can be started from the same point and provided as seed corpus.PythonFuzz can also start with an empty directory (i.e no seed corpus) though some valid test-cases in the seed corpus
may speed up the fuzzing substantially.PythonFuzz tries to mimic some of the arguments and output style fromlibFuzzer.More fuzz targets examples (for real and popular libraries) are located under the examples directory and
bugs that were found using those targets are listed in the trophies section.Credits & AcknowledgmentsPythonFuzz is a port offuzzitdev/jsfuzzwhich is in turn heavily based ongo-fuzzoriginally developed byDmitry Vyukov's.
Which is in turn heavily based onMichal ZalewskiAFL.For coverage PythonFuzz is usingcoverageinstrumentation and coverage library.ContributionsContributions are welcome!:) There are still a lot of things to improve, and tests and features to add. We will slowly post those in the
issues section. Before doing any major contribution please open an issue so we can discuss and help guide the process before
any unnecessary work is done.Trophiespython built-in HTMLParser - unhandled exceptionFeel free to add bugs that you found with pythonfuzz to this list via pull-request