## 3 Materials and Methods
### Research Methodology
#### 3.3.2 Experimental Conditions: Factors and Levels  
Two factors were selected for the experiment: the input data adopted in the data management stage and the predictive model considered in the model learning stage.  
For the first factor, two levels were tested. No macroeconomic variables were considered in the first level, which was identified with the name UNIVARIATE. Here, the future supplier delivery delay predictions of a specific component were based only on that component's past historical delivery performance records. In contrast, for the second level identified with MULTIVARIATE, macroeconomic variables and past delivery delay performance were used as input for the data management stage.  
Two levels were also selected for the second factor. An ARIMAX model was used as a predictive model for the first level. An ARIMAX model is a generalization of the ARIMA model [43] whereby predictions of future values can be generated by considering both historical values and the values assumed by exogenous predictors. The ARIMAX model can be used for both stationary and nonstationary time series. When the time series is stationary, the ARIMAX model can be referred to as ARMAX. A general ARMAX model with exogenous predictors can be formulated as follows:  
\[y_{t}=\beta_{0}+\beta_{1}y_{t-1}+\ldots+\beta_{p}y_{t-p}+\mathcal{O}_{1} \varepsilon_{t-1}+...+\mathcal{O}_{q}\varepsilon_{t-q}+\varepsilon_{t}+ \theta_{1}X_{it} \tag{2}\]  
where \(\beta_{i}\) is the coefficient of the autoregressive part, \(y_{t}\) is the value of the label at time \(t\), \(p\) is the order of the autoregressive process, \(\mathcal{O}_{i}\) represents the coefficient of the moving average part, \(\varepsilon_{t}\) is the residual error at time \(t\), and \(q\) is the order of the moving average component. Lastly, \(\theta_{i}\) is the coefficient of the exogenous variable \(X_{i}\), and \(X_{it}\) is the value of the covariate \(X_{i}\) at the time instant t. When no exogenous variables are considered, the  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Supplier** & \begin{tabular}{c} **Number of** \\ **Components** \\ \end{tabular} & \begin{tabular}{c} **Minimum** \\ **Value** \\ \end{tabular} & \begin{tabular}{c} **Mean** \\ **Value** \\ \end{tabular} & \
\begin{tabular}{c} **Standard** \\ **Deviation** \\ \end{tabular} &
\begin{tabular}{c} **Maximum** \\ **Value** \\ \end{tabular} \\ \hline
7 & 1 & \(-5\) & 4.8 & 6.0 & 28 \\
8 & 1 & \(-24\) & 7.5 & 14.9 & 45 \\
9 & 2 & \(-17\) & 14.1 & 15.1 & 60 \\
10 & 1 & \(-24\) & \(-4.7\) & 8.8 & 25 \\
11 & 12 &  
ARIMAX model coincides with the traditional ARIMA model. An LSTM was adopted as the other learning technique.  
Overall, the experimental design thus consisted of \(2^{2}\) experimental conditions represented as (UNIVARIATE, ARIMAX), (MULTIVARIATE, ARIMAX), (UNIVARIATE, LSTM), and (MULTIVARIATE, LSTM). Specifically, the experimental condition (MULT IVARIATE, LSTM) represents the proposed approach, while the remaining three are considered as benchmarks.  
The response variables monitored for each experimental condition are represented by widely adopted accuracy metrics for regression problems. The considered metrics include the mean absolute error (\(MAE\)), the symmetric mean absolute percentage error (\(SMAPE\)), and \(RMSE\), which are defined as follows:  
\[MAE=\frac{1}{N}{\sum}_{t=1}^{N}\left|Y_{t}-\hat{Y}_{t}\right| \tag{3}\]  
\[SMAPE=\frac{1}{N}{\sum}_{t=1}^{N}\frac{\left|Y_{t}-\hat{Y}_{t}\right|}{\frac{Y _{t}+\hat{Y}_{t}}{2}} \tag{4}\]  
\[RMSE=\sqrt{\frac{1}{N}{\sum}_{t=1}^{N}\left(Y_{t}-\hat{Y}_{t}\right)^{2}} \tag{5}\]  
where \(Y_{t}\) is the true historical value of the delivery delay or the advance recorded at time \(t\) for a specific component, \(\hat{Y}_{t}\) is the value predicted, and \(N\) is the length of the test set.