# A Novel Multi-Task Performance Prediction Model for Spark
## 4 Experimentation
### Experimental Results  
We conducted prediction experiments on the datasets of four baseline application tasks and divided the training set and test set by experiments, which ensured the model was tested on the test set, and the test results are shown in Table 3. In addition, we compared the model with baseline regression prediction models such as kNN, LR, SVC, RF, GBDT, etc., and it can be seen that our proposed model outperforms most of the baseline models. As depicted in Table 3, the metrics MAPE, RMSE, MAE, and \(R^{2}\) for our proposed MHAC model outperform those of the baseline prediction models across four benchmark datasets. Specifically, on the WordCount dataset, the MAPE of our MHAC model shows a \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Task Name** & **Model Name** & **MAPE** & **RMSE** & **MAE** & \(\mathbf{R^{2}}\) \\ \hline \multirow{6}{*}{WordCount} & KNN & 0.208 & 5.410 & 4.059 & 0.503 \\  & LR & 0.250 & 6.166 & 4.735 & 0.355 \\  & SVC & 1.000 & 22.758 & 21.425 & \(-7.787\) \\  & RF & 0.180 & 5.493 & 3.671 & 0.488 \\  & GBDT & 0.304 & 6.701 & 5.251 & 0.238 \\  & MHAC & 0.148 & 3.917 & 2.834 & 0.740 \\ \hline \multirow{6}{*}{PageRank} & KNN & 0.283 & 1.715 & 1.361 & \(-0.132\) \\  & LR & 0.328 & 2.134 & 1.615 & \(-0.752\) \\  & SVC & 0.999 & 5.744 & 5.513 & \(-11.694\) \\  & RF & 0.284 & 1.921 & 1.389 & \(-0.420\) \\  & GBDT & 0.291 & 1.564 & 1.328 & 0.059 \\  & MHAC & 0.255 & 1.419 & 1.212 & 0.225 \\ \hline \multirow{6}{*}{Pi} & KNN & 0.113 & 0.636 & 0.341 & 0.511 \\  & LR & 0.118 & 0.720 & 0.384 & 0.375 \\ \cline{1-1}  & SVC & 1.000 & 2.624 & 2.461 & \(-7.307\) \\ \cline{1-1}  & RF & 0.121 & 0.745 & 0.374 & 0.330 \\ \cline{1-1}  & GBDT & 0.236 & 0.810 & 0.598 & 0.208 \\ \cline{1-1}  & MHAC & 0.089 & 0.537 & 0.278 & 0.651 \\ \hline \multirow{6}{*}{LR} & KNN & 0.164 & 5.964 & 4.135 & 0.872 \\ \cline{1-1}  & LR & 0.334 & 10.917 & 8.336 & 0.572 \\ \cline{1-1}  & SVC & 0.999 & 31.995 & 27.305 & \(-2.677\) \\ \cline{1-1}  & RF & 0.114 & 5.304 & 3.300 & 0.899 \\ \cline{1-1}  & GBDT & 0.583 & 13.999 & 10.617 & 0.296 \\ \cline{1-1}  & MHAC & 0.098 & 3.994 & 2.647 & 0.943 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: This is the table of experimental results on WordCount, PageRank, Pi, and LR datasets. 17.7% reduction compared to the RF model, and the RMSE witnesses a 27.6% decline against the kNN model. On additional datasets, our model consistently yielded the optimal metrics. Our model is a multi-task prediction model. Therefore, we merged two or more benchmark datasets and conducted experiments on this combined dataset. The results, when compared with six benchmark prediction models, are presented in Table 4.  
From Table 4, we can observe that our model essentially achieved optimal performance across all metrics in the mixed datasets, surpassing the majority of the benchmark models. In the dataset combined from PageRank and WordCount, the MAPE metric, when compared with other models, showed reductions of 87.9%, 11.6%, 66.4%, 88.0%, 83.7%, and 80.2%, respectively. Furthermore, when comparing the MAPE values of the models in Table 3, we find that the kNN, a simple model, performs exceptionally well in single-task predictions, achieving a low MAPE. However, under multi-task conditions, the predictive performance of kNN deteriorates significantly, with its MAPE exceeding 2. In contrast, the LR model exhibits better performance in multi-task scenarios compared to other models, indicating its stronger adaptability to complex data. Our model surpasses the majority of the models in multi-task metrics, suggesting that it is also well-suited for complex tasks and capable of handling Spark performance prediction tasks in multi-task settings. To demonstrate the convergence of our model during the training process, we plotted the loss function. As shown in Figure 4, Figure 4a, Figure 4b, Figure 4c, and Figure 4d represent the loss function values on the WordCount, PageRank, Pi, and LR datasets, respectively.