# PHEFT: Pessimistic Image Processing Workflow Scheduling for DSP Clusters
## 6 Experimental Results
### Performance Profile  
In the previous section, we presented the average performance degradations of the strategies over three metrics and test cases. Now, we analyze results in more detail. Our sampling data were averaged over a large scale. However, the contribution of each experiment varied depending on its variability or uncertainty [30; 31; 32]. To analyze the probability of obtaining results with a certain quality and their contributors on average, we present the performance profiles of the strategies. Measures of result deviations provide useful information for strategies analysis and interpretation of the data generated by the benchmarking process.  
The performance profile \(\delta(\tau)\mathrm{p}\tau\) is a non-decreasing, piecewise constant function that presents the probability that a ratio \(\gamma\) is within a factor \(\tau\) of the best ratio [33]. The function \(\delta(\tau)\) is the cumulative distribution function. Strategies with larger probabilities \(\delta(\tau)\) for smaller \(\tau\) will be preferred.  
Figure 5 shows the performance profiles of the strategies according to total completion time, in the interval \(\tau=[1\ldots 1.2]\), to provide objective information for analysis of a test set.  
Figure 5(a) displays results for Montage workflows. PHEFT had the highest probability of being the better strategy. The probability that it was the winner on a given problem within factors of 1.02 of the best solution was close to 0.9. If we chose to be within a factor of 1.1 as the scope of our interest,  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & **Criteria** & \multicolumn{6}{c}{**Strategy**} \\ \cline{3-8}  & **Rand** & **BC** & **PESS** & **OPTI** & **PHEFT** & **OHEFT** \\ \hline \multirow{4}{*}{Montage} & \(C_{max}\) & 3.189 & 0.010 & 0.009 & 0.039 & 0.005 & 0.011 \\  & \(c\)_cpu_ & 0.209 & 0.173 & 0.100 & 0.196 & 0.050 & 0.093 \\  & \(c\)_cps_ & 0.001 & 0.305 & 0.320 & 0.348 & 0.302 & 0.306 \\  & Mean & 1.133 & 0.163 & 0.143 & 0.194 & 0.119 & 0.137 \\  & Rank & 6 & 4 & 3 & 5 & 1 & 2 \\ \hline \multirow{4}{*}{Ligo} & \(C_{max}\) & 0.044 & 0.043 & 0.012 & 0.012 & 0.001 & 0.002 \\  & \(c\)_cps_ & 0.580 & 0.542 & 0.059 & 0.059 & 0.002 & 0.013 \\  & \(c\)_cps_ & 0.040 & 0.040 & 0.012 & 0.013 & 0.002 & 0.002 \\  & Mean & 0.221 & 0.208 & 0.028 & 0.028 & 0.002 & 0.005 \\  & Rank & 6 & 5 & 3 & 4 & 1 & 2 \\ \hline \multirow{4}{*}{All test cases} & \(C_{max}\) & 1.616 & 0.027 & 0.011 & 0.025 & 0.003 & 0.006 \\  & \(c\)_cpu_ & 0.394 & 0.357 & 0.079 & 0.128 & 0.026 & 0.053 \\ \cline{1-1}  & \(c\)_cps_ & 0.020 & 0.173 & 0.166 & 0.180 & 0.152 & 0.154 \\ \cline{1-1}  & Mean & 0.677 & 0.186 & 0.085 & 0.111 & 0.060 & 0.071 \\ \cline{1-1}  & Rank & 6 & 5 & 3 & 4 & 1 & 2 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Rounded performance degradation and ranking.  
then strategies except Rand and OPTI would have sufficed with a probability of 1. Figure 5b displays results for Ligo workflows. Here, PHEFT and OHEFT were the best strategies, followed by OPTI and PES.  
Figure 6 shows \(cpw\) performance profiles of six strategies for Montage and Ligo workflows considering \(\tau=[1\ldots 1.2]\). In both cases, PHEFT had the highest probability of being the better strategy for \(cpw\) optimization. The probability that it was the winner on a given problem within factors of 1.1 of the best solution was close to 0.85 and 1 for Montage and Ligo, respectively.  
Figure 7 shows the mean performance profiles of all metrics, scenarios and test cases, considering \(\tau=[1\ldots 1.2]\). There were discrepancies in performance quality. If we want to obtain results within a factor of 1.02 of the best solution, then PHEFT generated them with probability 0.8, while Rand with a probability of 0.47. If we chose \(\tau=1.2\), then PHEFT produced results with a probability of 0.9, and Rand with a probability of 0.76.