## 3 Materials and Methods
### Research Methodology
#### 3.3.3 Experiment Set-Up  
The set-up adopted for each of the four considered experimental conditions involved several steps. First, the amount of delivery delay or the advance of a specific component was considered the dependent variable to predict (label). In contrast, the macroeconomic variables and/or past historical delivery delay records of the same component were considered the independent variables adopted for the predictions (features). Second, both the features and the label dataset were split into three consecutive temporal portions. The first 60% were identified as the training set, the second 20% as the validation set, and the last 20% as the test set. Thereafter, different data management strategies were applied according to the experiment under consideration. On the one hand, only the data augmentation and scaling steps described in Section 3.2.1 were applied for the UNIVARIATE experimental conditions. On the other hand, all steps reported in Section 3.2.1 were performed for the experiments that considered a MULTIVARIATE experimental condition. Specifically, for the feature engineering phase, the lagged macroeconomic indicators were generated considering possible lag values expressed in days in the discrete subset [30; 60; 90; 120; 150; 180; 210; 240; 270; 300; 330; 360]. In the feature selection step, the limit of macroeconomic features to select was selected by testing, separately for each model, different K values in the discrete subset [1; 20; 30]. For each of the three considered possible values of K, 134 separate predictive models--one for each component--were thus trained on their respective training sets, and a hyperparameter tuning step based on a grid search procedure was executed to select the best values of the hyperparameters for each model, minimizing the \(RMSE\) in the validation dataset. For those experiments that considered an LSTM model in the model learning phase, the Keras library and Keras tuner package were adopted to build the model and tune its hyperparameters [44]. The research space adopted to perform the grid search and the threshold adopted for the early stopping procedure are reported in Table 4. The other parameters were left to the default values reported in the Keras library. The research spaces adopted for the hyperparameter tuning when an ARIMAX model was considered in the experiments are reported in Table 5.  
Lastly, the number of macroeconomic features reporting the lowest _RMSE_ in the validation test for the majority of components was considered the best number of macroeconomic features to adopt for the final computation of the response metrics in the test set. According to the preliminary results reported in Figure 2a,b, the ARIMAX model reported better results when only one macroeconomic variable was considered. On the other hand, 20 features were identified as the optimal number of macroeconomic features for the LSTM models. The selected features and hyperparameters computed in the previous step were thus implemented to retrain multiple models on the union of the training and validation sets, and the response metrics described in Section Response Variables were computed for each component separately over their respective test sets for each of the four identified experimental conditions. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameters** & **Research Space** \\ \hline Number of layers & 1–4 \\ Learning rate & 0.0001–0.01 \\ Max number of epochs & Max 1000 \\ Early stopping threshold (\(\sigma\)) & 15 epochs with no improvements of _RMSE_ of the validation dataset \\ Number of layers & 1–4 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Research space of the LSTM hyperparameters.