Artifact of Fuzzing Machine Learning Paper
Creators: name: anon.
Publisher: name: Zenodo
Year: 2024
Subjects: ML
fuzzing

description: README 

This directory contains files related to fuzzing, coverage analysis, and setup, organized into two sections: Code/Tools/Environment and Results.

Code, Tools, and Environment

1. fuzzingML-main.zip     - Contains the source code for the tool, a fuzzer based on machine learning techniques.

2. afl_scriptsNseeds4CrashBaseTest.tar.gz   - Includes scripts and seed files required to start the fuzzing process.

3. m510_setup.tar.gz     - The complete fuzzing environment as it was tarred from the `m510` machine.

   - Setup all in /opt/ and run AFL++ in hot-storage for `m510` machine. 

user@node:/dev/shm$ dd if=/dev/zero of=testfile bs=1G count=1 oflag=dsync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.866424 s, 1.2 GB/s

4. Build_cov_env_cpython_scripts.tat.gz    - Scripts to build on your own machine 5-6 parts for coverage.5. cpython-wth-mlighter.tar.gz     - Instrumented version of CPython with `gcov` for coverage analysis, along with installed `sklearn` version 1.6.0. Seed test_916698062 was tested with `sklearn` version 1.5.2 due to a change of function interface.   - To restore the installation of Mlighter util, you will need to run:

cd /users/user42/cpython/install/usr/local/bin
./../lib/mlighter/mlighter/utils/install.sh

6. coverage_env_wt_gfauto.tar.gz     - The coverage environment prepared and tarred, containing the instrumented CPython setup.

   - Packages installed for Coverage machine:

gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

and in general on the machine:

Package                Version
---------------------- -------------
blinker                1.4
build                  1.2.2.post1
certifi                2020.6.20
command-not-found      0.3
coverage               7.6.10
cryptography           3.4.8
dbus-python            1.2.18
distlib                0.3.9
distro                 1.7.0
distro-info            1.1build1
filelock               3.16.1
httplib2               0.20.2
importlib-metadata     4.6.4
jeepney                0.7.1
joblib                 1.4.2
keyring                23.5.0
launchpadlib           1.10.16
lazr.restfulclient     0.14.4
lazr.uri               1.0.6
mlighter-utils         1.1.1
more-itertools         8.10.0
netifaces              0.11.0
numpy                  2.2.0
oauthlib               3.2.0
packaging              24.1
pandas                 2.2.3
pip                    24.3.1
pipenv                 2024.4.0
platformdirs           4.3.6
Pygments               2.11.2
PyGObject              3.42.1
PyJWT                  2.3.0
pyparsing              2.4.7
pyproject_hooks        1.2.0
python-apt             2.4.0+ubuntu1
python-dateutil        2.9.0.post0
pytz                   2024.2
PyYAML                 5.4.1
scikit-learn           1.6.0
scipy                  1.14.1
SecretStorage          3.3.1
setuptools             75.6.0
six                    1.17.0
ssh-import-id          5.11
threadpoolctl          3.5.0
tomli                  2.0.2
tzdata                 2024.2
ubuntu-advantage-tools 8001
ufw                    0.36.1
virtualenv             20.28.0
virtualenv-clone       0.3.0
wadllib                1.3.6
wheel                  0.37.1
zipp                   1.0.0

and in the instrumented Cpython version:

Package         Version
--------------- -----------
build           1.2.2.post1coverage        7.6.10
distlib         0.3.9
filelock        3.16.1
joblib          1.4.2
mlighter-utils  1.1.1
numpy           2.2.0
packaging       24.2
pandas          2.2.3
pip             24.3.1
pipenv          2024.4.0
platformdirs    4.3.6
pyproject_hooks 1.2.0
python-dateutil 2.9.0.post0
pytz            2024.2
scikit-learn    1.6.0
scipy           1.14.1
setuptools      75.6.0
six             1.17.0
threadpoolctl   3.5.0
tomli           2.2.1
tzdata          2024.2virtualenv      20.28.0xgboost         2.1.3

Except for seed 902236794 for which we used

numpy           2.2.1scikit-learn    1.5.2due to compatbility issues. For all seeds, we uesdsetuptools       65.5.0

when taking the coverage for the Python part (it seems like a minor change, so we left it as is).

Evaluation Setup

We generated 16 templates, marked in the paper from 1-16. The template numbers in the data here are as follows:

|-----|-----------------|------------------------------------------|| #   | Template ID.    | Models                                   ||-----|-----------------|------------------------------------------|| 1   | 137431083       | `KNeighborsClassifier`                   || 2   | 374268032       | `RadiusNeighborsClassifier`              || 3   | 375431503       | `DecisionTreeClassifier`                 || 4   | 475093897       | `LinearRegression`                       || 5   | 523744128       | `GradientBoostingClassifier`             || 6   | 523989622       | `MLPClassifier`                          || 7   | 649447714       | `GaussianNB and KNeighborsClassifier`    || 8   | 708003241       | `Ridge`                                  || 9   | 754757302       | `LogisticRegression`                     || 10  | 758989938       | `SVC`                                    || 11  | 770797067       | `DecisionTreeClassifier (duplicate)`     || 12  | 809881512       | `GaussianNB`                             || 13  | 86784155        | `RandomForestClassifier`                 || 14  | 88885636        | `LinearDiscriminantAnalysis`             || 15  | 902236794       | `XGBClassifier (from XGBClassifier)`     || 16  | 916698062       | `AdaBoostClassifier`                     ||-----|-----------------|------------------------------------------|

You can use the template ID above to locate the specific template in the Experiment folder.

Results

1. Results_AFL_crashTesting_24h_31sets_N_12days_1set.tar.gz     - Results from the fuzzing process, covering 24-hour runs with 31 sets across 12 days.

2. Results of Coverage:     - Coverage of the C/C++ part of the Cpython compiler:

       - Part 1: coverage_c_part_1.tar.gz (9 out of the 11 seeds)

       - Part 2: coverage_c_part_2.tar.gz (1 out of 11 seeds - repeat)     

       - Part 3: coverage_c_part_3.tar.gz (2 out of 11 seeds)     

   - Coverage summary and analysis of the C part is available here: function_coverage-16seeds_C.xlsx and line_coverage-16seeds_C.xlsx.

Usage Overview

- Fuzzing Setup:  - Use `fuzzingML-main.zip` for the fuzzer implementation.  - Extract `afl_scriptsNseeds4CrashBaseTest.tar.gz` to get the necessary scripts and seed files for fuzzing.

- Environment:  - The fuzzing environment is packaged in `m510_setup.tar.gz`. Extract this to replicate the setup on the `m510` machine.

- Coverage Analysis:  - Use `cpython-wth-mlighter.tar.gz` for the CPython setup instrumented with `gcov`.  - The coverage environment is stored in `coverage_env_wt_gfauto.tar.gz`.