# Containerised Application Profiling and Classification Using Benchmarks
## 6 Experimentation  
Experiments were conducted on five heterogeneous hardware setups, three for training and testing and two for the creation of the external validation dataset. For the training and validation dataset, two of the configurations were physical machines and one an academic cloud service virtual machine (VM) ([[https://okeanos-knossos.grnet.gr/home/](https://okeanos-knossos.grnet.gr/home/)]([https://okeanos-knossos.grnet.gr/home/](https://okeanos-knossos.grnet.gr/home/)), accessed on 15 November 2022). For the external validation dataset, an on-premise physical machine and a VM were used. The hardware configurations and their operating system are listed in Table 2. On top of every configuration, we deployed the Docker platform which was utilised to run different benchmarks in a containerised environment. It is important to bring to attention that there is a high diversity in the hardware configuration chosen in order to test as broadly as possible the hardware agnostic capabilities of the algorithm. The chosen benchmarks are part of two of the most known and standard benchmarks suites, Sysbench ([[https://github.com/akopytov/sysbench](https://github.com/akopytov/sysbench)]([https://github.com/akopytov/sysbench](https://github.com/akopytov/sysbench)), accessed on 10 November 2022) and Phoronix ([[https://www.phoronix-test-suite.com/](https://www.phoronix-test-suite.com/)]([https://www.phoronix-test-suite.com/](https://www.phoronix-test-suite.com/)), accessed on 10 November 2022). The set of benchmarks was configured to be as diverse as possible; hence, approximately 63 different benchmarks were used, covering different types of application types in terms of hardware usage intensity and tasks performed. For the Sysbench benchmarks, three different workload levels were set for each, whereas for Phoronix, whenever options were available, the default ones was always chosen (Table 3). All executions were done in a simple Docker container comprised of the Ubuntu. For the training dataset on which the classification model was based, a little over 1000 iterations were performed. Approximately 70% of the results came from configuration A, 15% from configuration B, and the rest 15% from Configuration C. For the testing dataset, approximately 250 iterations were performed, with an approximate ratio 80% in configuration D and 20% in configuration E.  
Figure 7: Number of features that surpass the 5% threshold of \(p\)-value for every benchmark. Statistical collection is accomplished through the Docker stats command ([[https://docs.docker.com/engine/reference/commandline/stats/](https://docs.docker.com/engine/reference/commandline/stats/)]([https://docs.docker.com/engine/reference/commandline/stats/](https://docs.docker.com/engine/reference/commandline/stats/)), accessed on 11 November 2022), which makes use of the Docker API and retrieves container specific metrics. The same metrics were collected for all configurations in order to create the benchmark profiles. Each execution was configured to last at least 10 min and terminate as soon as possible after this duration. In practice, that means that after the completion of a benchmark run, an identical execution would take place immediately and this loop would continue until the first termination exceeded the 10-min limit. The metrics scraping interval was set to 10 s, which means that after the profiler was invoked, Docker stats was triggered every 10 s. Lastly, the profiler was actually called after the download and installation step that the benchmark suites perform in order to get ready for the benchmarks' execution, and the container would stop operating right after the completion of the execution; hence, metrics were gathered only during real benchmark execution time, to avoid any inconsistencies in the profiles. After creating the described datasets, a second version of them was formed, where same benchmarks with different workloads are homogenised, resulting in four (4) datasets, two (2) with workload distinction and two (2) without (Figure 8).