# Predictive Scheduling with Markov Chains and ARIMA Models
## 4 Experimental Verification of the Proposed Solution
### Prediction of Machine Failure Parameters  
The failure prediction process is performed using appropriate scripts formulated in the RStudio computing environment [48] that enable the analysis in the range described in Section 3.3. Modelling the machine failure rates using the Markov chain (containing information on changes in production) is performed with the use of the _markovchain_ library. Initially, the collected empirical data is verified to check whether they fulfil the properties of the Markov process. The analyses confirm that the data from the analyzed machines meet the required properties, which is further evidenced by the \(p\)-value index (Table 3). The \(p\)-value is the probability of obtaining hypothesis test results as extreme as the observed results, assuming that the null hypothesis is correct (data chain has a Markov property). In the next stage of the machine failure prediction analysis, the transition rate matrices are generated from the collected data. From the obtained information, we determine, at a given probability level, the occurrence of subsequent chain elements, which in this study is the probability of machine failure occurrence at subsequent production shifts (Table A1). For clarity of presentation, the results from calculations are given in the form of transition diagrams (Figure 1), which additionally enable determining the probability of failure not only on subsequent shifts but also during the current shift (the arrow returning to the node). In Figure 1, knots represent shifts and the probability of machine failure during a given shift is given at the beginning of each arrow (next to the knot), e.g., for machine \(M_{6}\), the probability that the machine failure will occur after shift 1 during shift 2 is 0.593. \begin{table}
\begin{tabular}{c c} \hline \hline
**Machine \(M_{3}\)—**180 observations \\
**\(\bullet\)** & Machine \(M_{6}\)—157 observations \\
**\(\bullet\)** & Machine \(M_{7}\)—208 observations \\
**\(\bullet\)** & Machine \(M_{8}\)—97 observations \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Machine failure and repair time data implemented in the scheduling solution. \begin{table}
\begin{tabular}{c c} \hline \hline
**Machine No.** & _p_**-Value [–]** \\ \hline \(M_{1}\) & 0.8922 \\ \(M_{2}\) & 0.9051 \\ \(M_{3}\) & 0.9510 \\ \(M_{6}\) & 0.7361 \\ \(M_{7}\) & 0.9684 \\ \(M_{8}\) & 0.5618 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Markov process identification results. The second key parameter of failure is its time. To this end, the _forecast_ package is used, which enables identifying the elements of ARIMA time series. As a result, the predicted machine failure repair times are determined. Before forecasting, each machine is tested to verify whether the process is stationary, also the component models are established (autoregression, moving average and the integration). In the subsequent step, future repair times are predicted. Due to the fact that in ARIMA models, the forecasts may also take negative values, the collected observations have been first subjected to variance-stabilizing transformation, and after the prediction, the process is completed and their original sets of values are returned. The results from the model identifying exemplary predicted times for the 5 subsequent steps of the time series are presented in Table 4.  
Figure 1: Markov chains with transition diagrams for the considered machines: (**a**) Machine \(M_{1}\); (**b**) Machine \(M_{2}\); (**c**) Machine \(M_{3}\); (**d**) Machine \(M_{6}\); (**e**) Machine \(M_{7}\); (**f**) Machine \(M_{8}\). The data below display the disparity between individual machine repair times. Each machine is coupled with a different ARIMA model. From the set of possible ARIMA models, we select a model that has a smaller AIC (Akaike Information Criterion) value. The analytical process is carried out with the exclusive use of autoregression (Machine \(M_{1}\)), or the moving average (Machine \(M_{8}\)), or a combination thereof (Machine \(M_{2}\), \(M_{6}\), \(M_{7}\)).