## 6 Model- and Data-Driven Focused Analysis
### Validation and Verification of the Results  
The next step of the decision support workflow presented in Section 5 is the validation and verification of results. The main question of the former is whether after executing all the previous steps, could we meet the original engineering and managerial requirements? Could we devise a better, combined way of using both the models and data available in our cyber-physical production system? More specifically, could we make use of the DES model and the collected, historical data of orders in the interest of stabilizing CDD performance? The answer to these questions is affirmative, because the series of simulation experiments, combined with unsupervised learning over the dataset of historical orders, have shown the following:  
* Rough-cut sensitivity analysis can highlight potential factors--both structural and behavioral--for change. * Clustering can generate characteristic patterns of daily demand. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset ID** & **C5** & **C10** & **C25** & **C50** & **C75** & **C100** & **Base** \\ \hline MAE.B & 0.273 & 0.180 & 0.092 & 0.091 & 0.062 & 0.015 & 0.000 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Aggregated performance comparison of buffer sizes over different clusters of the annual demand. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset ID** & **C5** & **C10** & **C25** & **C50** & **C75** & **C100** & **Base** \\ \hline MAE.S & 0.166 & 0.149 & 0.106 & 0.057 & 0.054 & 0.036 & 0.000 \\ MS.S \% & 96.3 & 95.8 & 94.7 & 94.6 & 92.4 & 92.2 & 100.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance comparison of sequencing over different clusters of the annual demand. * Detailed simulation can help in finding the right structural settings (in our case, buffer sizes) and the context-dependent sequencing and lot-sizing rules which fit best to a given daily demand pattern. We must emphasize that lessons drawn in every phase of the decision support workflow complied well with the so far implicit domain knowledge of our production planners. Hence, the next question was how can we verify that the suggested method really improves CDD performance? To answer this question, a new series of simulation experiments was designed and run, this time departing from the C25 clustering which offered the best compromise between representative power and efficiency. Conditions of the experiments were set as follows:  
* The size of the most sensitive buffer was increased. * The best-performing sequencing and lot-sizing rules were assigned to each member of the C25 cluster. Hence, these important decisions could be performed in a context-dependent manner. * Each daily input mix was classified according to the C25 clustering, and the assigned sequencing and lot-sizing rules were applied. * For each day, performance evaluation was calculated as the ratio of the makespan according to the new and the original protocols. * Aggregated comparisons were performed in time windows of different lengths over a rolling horizon. Specifically, windows covering 1, 5, and 10 days were \"pushed\" through the whole 235-days-long horizon. Results of these verification studies are summarized in Table 5. Here, the columns present the aggregated results of experiments for each time window. Beyond the mean and the standard deviation of the ratios of the new and original makespans, Table 5 also shows the values for cases if 25%, 50%, and 75% of the data are around the mean. One can see that an absolute improvement cannot be warranted: for the one-day window, the value of the 75% neighborhood cell refers to this. However, improvement is consistent and significant as the length of the investigated time window is increasing, for every category (i.e., row in Table 5) of the comparison. Hence, one can deduce that the C25 clustering and the corresponding situation-dependent sequencing and lot-sizing rules reliably improved the delivery performance of our HMLV manufacturing facility.