## 4 Content Prediction of Free Calcium Oxide (f-CaO) in Cement Clinker Using Machine Learning
### Identification of Abnormal Operating Conditions in Cement Production  
The identification of abnormal operating conditions in this work is done by training a ResNet50 [17] deep learning model to classify real-time thermal images showing the material discharge positions in the grate cooler. The ResNet50 model incorporates an additional convolutional layer within each residual block inside of the network to increase its depth and thus model generality and typically performs well in image classification tasks. In the context of our problem, the ResNet50 model learns from the \"flying ash\" features in real-time images to determine their respective normalities of operating conditions. The loss function of the ResNet50 model is an enhanced version of the cross-entropy loss, known as the focal loss. This focal loss function is defined as follows:  
\[FL(p_{t})=-(1-p_{t})^{\gamma}\log(p_{t}),\]  
where \(\gamma\) is a constant, and its value being 0 makes the focal loss degenerate to the regular cross-entropy loss; \(p_{t}\) is defined as follows:  
\[p_{t}=\left\{ \begin{array}{ll}p,&\text{if }y=1\\ 1-p,&\text{otherwise}\end{array} \right.,\]  
with \(p\) being the model's estimated probability for the ground-truth class with label \(y=1\). Such a focal loss in ResNet50 [38] naturally solves the class imbalance problem, and therefore is particularly suitable to our problem since most of the real-time thermal images are collected under normal operating conditions in the cement production line, while the ones showing phenomenons that reflect abnormal operating conditions are the minority, considering the fact that most of the time the cement production line can operate and continuously produce cement clinkers without major problems. The trained ResNet50 model in this work compares and classifies the thermal images showing the material discharging positions inside of a cement grate cooler. It does so by assessing whether the central bright area of the real-time image is obstructed by flying smoke and sank, known as \"flying ash\". This assessment is used to determine the status of abnormal operating conditions. As shown in Figure 3, if there is substantial smoke and ash obstruction in the center area, the model examines whether there is smoke and ash on both sides of the image as well. If significant smoke and ash is detected, the \"flying ash\"  
Figure 2: The proposed approach in this work. phenomenon is determined, thus abnormal operating conditions are detected. On the other hand, as shown in Figure 4, if the central area of the thermal image is brightly lit and there is no obvious smoke or ash on either side, the condition is categorized as no \"flying ash\"; thus, normal operating conditions are detected. To study the reliability and have better interpretability of the adopted models in this work, we leverage the deep features learned by the ResNet neural networks and generate heatmaps based on the Class Activation Maps (CAM) [39] to highlight the regions in the thermal images that are crucial for determining a particular class. The results presented in Figure 5 show the parts of the thermal image that are \"looked at\" by the neural networks, where the heatmap results mark out the regions that are the most salient for making a classification decision. Contrasting Figure 5a and Figure 5b, one can clearly see that the trained ResNet50 models can identify the materials discharge positions from the thermal images of grate cooler. Over these discharge positions, the \"flying ash\" may appear when the cement clinker production is operating under abnormal conditions, which eventually would lead to poor clinker quality and impaired accuracy of f-CaO content prediction. Therefore, preprocessing the thermal image datasets using classification itself can also be viewed as an alarming mechanism, and based on which, ruling out the DCS control data under abnormal operating conditions help increase the accuracy of f-CaO content prediction (see Section 5 for such experimental results).