# Containerised Application Profiling and Classification Using Benchmarks
## 4 Profile Vector Creation
After the theoretical elimination process, a preliminary statistical analysis was conducted in order to identify the most stable features to represent the profile. The dataset used for this statistical analysis is the corresponding training dataset as shown later in the experimentation section, and the key performance indicator for the selection was the relative standard deviation (RSD). In order to compute the overall RSD for every feature, the RSD for every individual benchmark and workload combination is first calculated. Afterward, the mean value of all the different feature RSD is calculated. As depicted in Figure 4, 16 features with relatively small RSD were selected (most of them having RSD lower than 25%). Although the RSD for some of the features appeared to be high (i.e., geo_avg_net_tx_bytes, a metric describing the geometrical average bytes transited has RSD close to 40%) in reality it is adequate, given that the dataset contains data from three different hardware configurations. In addition, it is important to mention that some benchmarks have some features stable and another one with more significant deviation that affects the overall RSD. This does not raise any concerns because the used ML classification algorithm (random forest) discards features (binary trees) that do not aid the classification process. As depicted in Figure 4, features that concern the CPU and memory footprint of the average deltas produced the most promising results, because they represent the rate at which the application or benchmark request and use CPU and memory time, respectively. Lower deviations were found only in the task-(process threads) and network-based features, an expected outcome because there is a constant number of tasks and network packages that most of the benchmarks produce. After the preliminary statistical analysis, a more in-depth analysis was conducted, concerning the variances and the mean values of the different datasets of metrics that were chosen. Therefore, an ANOVA test [22] was performed for the already filtered features, for every combination of features and benchmarks. As far as the categories identified for the ANOVA test, because this is a statistical test to identify if there is a differentiation of values from different hardware configurations, two groups arose from the dataset. One group contains results from the training dataset hardware configurations, and the other group contains data from the external testing dataset hardware configurations. The null hypothesis (H\({}_{0}\)) is: The mean values of the two groups are equal, while the alternative hypothesis states that there is a significant difference in the mean values. The \(p\)-value computed from the results is compared with a threshold of 5% (as in most academic literature), in order to decide about the rejection or not of the null hypothesis. Starting with this reasoning, the ideal results should yield strongly in favour of the null hypothesis, because equal mean values would indicate that the proposed algorithm, would make the same decision as far as the profiling is concerned for a specific feature, regardless of the hardware it used. A visual example of the mean values for a specific benchmark in different hardware configuration can be seen in Figure 5. It is important to mention that this test was performed for all the combination of features and benchmarks. generally, the \(p\)-value of the selected features should be as high as possible, to give confidence on the results. Figure 4: Profile features relative standard deviation. Figure 5: Scatter chart of average memory consumed by the _sysbench_memory_ benchmark on different hardware configuration sets. Although there is a deviation between the \(p\)-values of the different feature-benchmark combinations, there is a significant number of features that surpasses the 5% threshold by a great deal, with some combinations reaching 90%.