DOI: 10.5281/zenodo.2630589
title: Task Scheduler Performance Survey Results

description: <strong>Task scheduler performance survey</strong>

This dataset contains results of task graph scheduler performance survey.<br>
The results are stored in the following files, which correspond to simulations performed on <br>
the `elementary`, `irw` and `pegasus` task graph datasets published at https://doi.org/10.5281/zenodo.2630384.


	elementary-result.zip
	irw-result.zip
	pegasus-result.zip


The files contain compressed pandas dataframes in CSV format, it can be read with the following Python code:<br>
```python<br>
import pandas as pd<br>
frame = pd.read_csv("elementary-result.zip")<br>
```

Each row in the frame corresponds to a single instance of a task graph that<br>
was simulated with a specific configuration (network model, scheduler etc.).<br>
The list below summarizes the meaning of the individual columns.


	<strong>graph_name</strong> - name of the benchmarked task graph
	<strong>graph_set</strong> - name of the task graph dataset from which the graph originates
	<strong>graph_id</strong> - unique ID of the graph
	<strong>cluster_name</strong> - type of cluster used in this instance the format is &lt;number-of-workers&gt;x&lt;number-of-cores&gt;; 32x16 means 32 workers, each with 16 cores
	<strong>bandwidth</strong> - network bandwidth [MiB]
	<strong>netmodel</strong> - network model (simple or maxmin)
	<strong>scheduler_name</strong> - name of the scheduler
	<strong>imode</strong> - information mode
	<strong>min_sched_interval</strong> - minimal scheduling delay [s]
	<strong>sched_time</strong> - duration of each scheduler invocation [s]
	<strong>time</strong> - simulated makespan of the task graph execution [s]
	<strong>execution_time </strong>- real duration of all scheduler invocations [s]
	<strong>total_transfer</strong> - amount of data transferred amongst workers [MiB]


The file `charts.zip` contains charts obtained by processing the datasets.<br>
On the X axis there is always bandwidth in [MiB/s].<br>
There are the following files:


	[DATASET]-schedulers-time - Absolute makespan produced by schedulers [seconds] 
	[DATASET]-schedulers-score - The same as above but normalized with respect to the best schedule (shortest makespan) for the given configuration.
	[DATASET]-schedulers-transfer - Sums of transfers between all workers for a given configuration [MiB]
	[DATASET]-[CLUSTER]-netmodel-time - Comparison of netmodels, absolute times [seconds]
	[DATASET]-[CLUSTER]-netmodel-score - Comparison of netmodels, normalized to the average of model "simple"
	[DATASET]-[CLUSTER]-netmodel-transfer - Comparison of netmodels, sum of transfered data between all workers [MiB]
	[DATASET]-[CLUSTER]-schedtime-time - Comparison of MSD, absolute times [seconds]
	[DATASET]-[CLUSTER]-schedtime-score - Comparison of MSD, normalized to the average of "MSD=0.0" case
	[DATASET]-[CLUSTER]-imode-time - Comparison of Imodes, absolute times [seconds]
	[DATASET]-[CLUSTER]-imode-score - Comparison of Imodes, normalized to the average of "exact" imode


<strong>Reproducing the results</strong>

<em>1. Download and install Estee (https://github.com/It4innovations/estee)</em>

$ git clone https://github.com/It4innovations/estee<br>
$ cd estee<br>
$ pip install .<br>
<br>
<em>2. Generate task graphs</em><br>
You can either use the provided script `benchmarks/generate.py` to generate graphs<br>
from three categories (elementary, irw and pegasus):

$ cd benchmarks<br>
$ python generate.py elementary.zip elementary<br>
$ python generate.py irw.zip irw<br>
$ python generate.py pegasus.zip pegasus<br>
<br>
or use our task graph dataset that is provided at https://doi.org/10.5281/zenodo.2630384.

<em>3. Run benchmarks</em><br>
To run a benchmark suite, you should prepare a JSON file describing the benchmark.<br>
The file that was used to run experiments from the paper is provided in<br>
`benchmark.json`. Then you can run the benchmark using this command:<br>
<br>
$ python pbs.py compute benchmark.json

The benchmark script can be interrupted at any time (for example using Ctrl+C).<br>
When interrupted, it will store the computed results to the result file and restore<br>
the computation when launched again.

<em>3. Visualizing results</em><br>
<br>
$ python view.py --all &lt;result-file&gt;<br>
<br>
The resulting plots will appear in a folder called `outputs`.