## 3 Materials and Methods
### Prediction Model Training  
Once the training and test datasets with memory and time information were built, a regression model could be trained with the goal of estimating peak memory occupancy and turnaround time of a task in a given workflow. As a preliminary step, we analyzed the correlation between the features of the training data and each target variable, using the Spearman index. We obtained the following positive correlations: a value of 0.30 between storage memory and the input dataset size, 0.46 between execution memory and the task class, and 0.21 between execution time and the number of columns. These results can be seen in detail in Figure 1. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline
**Task** & **Task** & **Task** & **Dataset** & **Dataset** & **Categorical** & **Dataset** & **Peak Storage** & **Peak Execution** & **Duration** \\
**Name** & **Type** & **Class** & **Rows** & **Columns** & **Columns** & **Size (MB)** & **Memory (MB)** & **Memory (MB)** & **(ms)** \\ \hline GMM & Estimator & Clustering & 1,474,971 & 28 & 0 & 87.00 & 433.37 & 1413.50 & 108,204.00 \\ K-Means & Estimator & Clustering & 5,000,000 & 104 & 0 & 1229.78 & 4624.52 & 4112.00 & 56,233.50 \\ Decision Tree & Estimator & Classification & 9606 & 1921 & 0 & 84.91 & 730.09 & 297.90 & 39,292.00 \\ Naive Bayes & Estimator & Classification & 260,924 & 4 & 0 & 13.50 & 340.92 & 6982.80 & 16,531.50 \\ SVM & Estimator & Classification & 5,000,000 & 129 & 0 & 1542.58 & 6199.11 & 106.60 & 2898.940.50 \\ FCFrowth & Estimator & Association Rules & 823,593 & 180 & 180 & 6970.0 & 9493.85 & 1371.03 & 96,071.50 \\ GMM & Transformer & Clustering & 165,474 & 14 & 1 & 6.37 & 2.34 & \(1\times 10^{-6}\) & 62.50 \\ K-Means & Transformer & Clustering & 4,898,431 & 42 & 3 & 648.89 & 3.23 & \(1\times 10^{-6}\) & 35.00 \\ Decision Tree & Transformer & Classification & 1,595,372 & 42 & 4 & 257.69 & 3.68 & \(1\times 10^{-6}\) & 65.50 \\ Naive Bayes & Transformer & Classification & 347,899 & 4 & 0 & 17.99 & 4.26 & \(1\times 10^{-6}\) & 92.50 \\ SVM & Transformer & Classification & 5,000,000 & 129 & 0 & 1542.58 & 2.36 & \(1\times 10^{-6}\) & 55.50 \\ FCFrowth & Transformer & Association Rules & 136,073 & 34 & 34 & 13.55 & 1229.95 & 633.50 & 52,429.00 \\... &... &... &... &... &... &... &... &... \\ \hline \end{tabular}
\end{table} 
Table 3: A sample of the training dataset. Afterwards, we moved to the training of the regression model. Due to its complexity, the regression problem cannot be faced with a simple linear regressor or its regularized variants (e.g., Ridge, Lasso, or ElasticNet), but a more robust model is necessary. We experimentally evaluated this aspect by testing the forecasting abilities of these linear models, achieving poor results. For this reason, an ensemble learning model was used in order to fit the nonlinear distribution of features. Specifically, the stacking technique (meta learning) [20] was used by developing a two-layer model in which a set of regressors was trained on the input dataset and a decision tree was fit on their predictions. The first layer consisted of three tree-based regressors, able to grasp different aspects of input data: a gradient boosting, an AdaBoost, and an extra trees regressor. The second layer exploits a single decision tree regressor, which predicts the final value starting from the concatenation of the outputs from the first layer. The described ensemble model was set with the hyperparameters shown in Table 4.  
Among 20 trained models, initialized with different random states, we selected the best one by maximizing the following objective function:  
\[\mathcal{O}=\hat{R}^{2}-MAE\]  
whose goal is to choose the model that best explains the variance of the data, while minimizing the forecasting error. This function jointly considers the adjusted determination coefficient (\(\bar{R}^{2}\)), which guarantees robustness with respect to the addition of useless variables to the model compared to the classical \(R^{2}\) score, and the Mean Absolute Error (MAE), normalized with respect to the maximum.