# Steelmaking Predictive Analytics Based on Random Forest and Semantic Reasoning
## 4 An Application of the Framework: Use Case of Cold Rolling
### Validation  
The purpose and contribution of the framework is to provide improved assistance for the human in the loop. The classifications of the RF model produced good, bad, and best outputs, whereas the final status output provided new decision-making knowledge on whether to continue the rolling process, with an estimation of how much more rolling was recommended based on classification and expert knowledge. Because of this, there was no ground truth or direct labels to compare for validation. However, there are many well-studied validation techniques for situations where no ground truth is available, often deployed during unsupervised learning models, such as interval validation, external validation, domain expertise, twin-sample validation, and cross-validation [37]. In our use case, we adopt external validation and domain expertise validation methodologies to validate our framework. These validation methods have been applied in various domains [38; 39]. Similarly, we followed an iterative screening and refinement process with domain experts and stakeholders to share results and obtain valuable feedback, as highlighted in Figure 6. Within the iterative cycle, the results are displayed, discussed, and validated with the domain experts. The expert rules are refined with any newly obtained knowledge from these instances. Once the rule set is recompiled, it is passed through the semantic reasoner again, producing new results which are displayed to the experts once more, repeating the iterative cycle. By utilizing domain experts for validation, we can ensure the accuracy, relevance, and robustness of the framework, providing insights and tactic knowledge that are not apparent from data alone. With each iteration, we displayed the one hundred data entries. These entries contained the status, classification, expert rules, sensory data, and the actual stock reduction of the roll. Furthermore, we categorised the entries into two groups: (1) expected output consisting of 80% of the entries where the status of the entry matched the amount of stock reduction for that roll, e.g., \"stop rolling\" for entries where the stock reduction was greater than the expected average value, and (2) unexpected output where 20% of entries where the opposite interaction occurred. All entries were displayed to the experts; however, it would be revealed to be a time consuming task to iterate through all the entries, so we further refined and handpicked the most interesting results and compared the predictions with those of domain experts. For the expected output category where the majority of data points were, ten entries were displayed to the experts. All entries were studied and approved by the experts, stating that they would have made the same decision based on the data. Two out of the ten entries are displayed in Table 2. This confirmed that the support provided by the framework was accurate for the expected category. Similarly, ten entries from the unexpected output category were also displayed to the experts. In three of those cases, the status was'stop rolling' despite a low diameter reduction and an accurate 'good' classification. In these cases, an expert rule was triggered  
Figure 6: Methodology of iterative process with experts for validation. that prioritised stopping operation if there was a high tonnage produced on a pair of work rolls that were recently damaged. Having investigated these results with domain experts, we discovered that there was a 'pinch' on one of the rolls, which is required to be removed before further operation. This positive feedback confirmed that the framework was able to provide accurate results in these cases. Meanwhile, there were three instances where the status produced was'stop rolling' by a similar expert rule. However, the experts stated that although the expert rule was correct, some knowledge was missing, stating that work rolls in Stand 3 are expected to handle more work load and pressure.