## 5 Current Prediction Model
### Evaluation--Physical Interpretability  
As shown in Figure 1, our data displays two behavioral changes attributed to machine maintenance. Experts believe that these changes were caused by the replacement of machine parts with different conductivity or unintended changes in sensor-related connectivity. This means that these changes should be related to the machine resistance parameter \(R_{mach}\). To validate whether the interpretability of our model is correct, we can verify whether the grey-box model is able to adapt to unseen maintenance periods by updating only this single parameter.  
We first train the grey-box model using all welds from a single maintenance period \(P1\). Next, we randomly select a (limited) number of welds from a different period \(P2\), and use these to update only the \(R_{mach}\) parameter using the previously described training process. Finally, we evaluate the model on all remaining welds from \(P2\). Assuming the interpretability of the model is correct, we should obtain good predictions.  
We compare the performance against the average-based model. Because the average model would actually be at a disadvantage if it retained data from \(P1\) to predict \(P2\), we assume the average model only uses the available data from \(P2\). Additionally, we select the training data for this model in such a way that all welding programs are equally  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Grey-Box Model** & **Average-Based Model** \\ \hline Train MSE & 11,544.88 & 11,090.66 \\ Train MAP & 1.04 & 1.02 \\ Train Reject (\%) & 2.7 & 2.5 \\ \hline Test MSE & 10,844.92 & 10,800.81 \\ Test MAP & 1.01 & 1.01 \\ Test Reject (\%) & 2 & 2.3 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Prediction errors in mean squared error (MSE), mean average percentage error (MAP) and percentage of rejected welds for the first version of our model versus the reference average-based model.  
represented. While this does inflate data requirements disproportionately, it gives the best possible prediction capability to the reference model, which is the focus of this experiment.  
The experiment was performed for all maintenance period combinations, averaging the results of ten different runs. The results are shown in Figure 3, where we see how the grey-box model achieves good predictions using as few as 50 welds to update \(R_{mach}\), each time converging to the same error as the average-based model. This confirms that all other grey-box model parameters can be reused across the maintenance periods. The results seem to hint that the average-based model stabilizes using data from fewer welds, but actually, the opposite is true, as the actual weld count should be multiplied by the number of welding programs. It is difficult to determine the number of welds required for reliable predictions when using randomly sampled data due to the welding program imbalance in the data, as the effect on the prediction score will be larger if more commonly used welding programs are sampled. A comparison with a more realistic setting can be made using the experiment in the next section.  
We have shown that the grey-box model matches the predictive power of the statistical model while also being interpretable. Next, we demonstrate how the adaptability to changes truly sets the grey-box model apart from the statistical model.