# **karonte_stats**

It contains the necessary scripts to run karonte against our dataset and collect the results.  This folder contains mainly three scripts: `run.py`, `parse_results.py`, and  `parse_all_results.py`
### `run.py`
It runs karonte on all the firmware samples of a set vendors:
> **SYNOPSIS**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;python **run.py** [-n THREADS]  [-v VENDOR]
>
> **DESCRIPTION**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**\-n**:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;number of threads to spawn  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**\-v**, **--vendors**: vendors to consider.  Options are 'tenda', 'netgear', 'tp-link', 'd-link', 'lk', 'lk_unpatched', 'huawei',  and 'mediatek'.   
>
> **EXAMPLE**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;python run.py -n 10 -v tenda -v netgear  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;It runs 10 instances of karonte at the same time to analyze firmware samples from tenda and netgear.  A new firmware sample is considered as soon as one one of the 10 instances terminate. 

If no argument is specified, `run.py` runs karonte on each firmware on the dataset one at the time. 

The results are stored in **./results/<vendor_name>/**, and each file reports the following statitstics:
- **Network parsers found**
- **BDG graph** and **CPS matched**
- **Bugs found**
- **Time elapsed**

Note that, each report might contain duplicates. 
To have a clear view of the results, please refer to the next section. 

### `parse_results.py`
It consumes a karonte report and gives a clear view of its content:
> **SYNOPSIS**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;python **parse_results.py** RESULT_FILE
>
> **DESCRIPTION**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Parses the karonte report RESULT_FILE, and show its results. 
>
> **EXAMPLE**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;python parse_results.py ./results/d-link/dir_880.json  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;This command shows the information about the network-facing binaries, BDG, CPSs, and bugs found in the dir880 D-Link firmware sample. 

### `parse_all_results.py`
It consumes all karonte reports in **./results/** and gives a clear view of their statistics content:
> **SYNOPSIS**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;python **parse_all_results.py**
>
> **DESCRIPTION**  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Parses all of karonte reports. 

For each vendor, it shows: 
* Number of binaries within the firmware samples of the vendor
* Average network-facing binaries
* Number of firmware samples *
* Number of firmware samples where user requests are handled by one binary 
* Number of firmware samples where user requests are handled by more than one binary
* Number of alerts generated by karonte
* Minimum analysis time
* Maximum analysis time
* Number of matching environment CPS
* Number of semantic CPS matching
* Average BDG cardinality,

