title: Addressing Concept Drift in Machine Learning-Based Monitoring of Manufacturing Processes
Creators: name: Jourdan, Nicolas
Publisher: name: UNSPECIFIED
Year: 2024

description: Machine learning (ML) has significantly contributed to the development of advanced process and condition monitoring systems in manufacturing, enabling real-time monitoring and analysis of equipment and processes to detect deviations from normal operations and predict potential failures. As ML applications transcend from academic research to real-world usage, questions regarding their continuous reliability on the shopfloor arise. The training dataset of an ML model only captures a snapshot of the manufacturing process in time. After model deployment, the environment will encounter changes such as wear, aging or defective sensors as well as changes in factory layout and machine placement that are not captured in the training dataset, a scenario referred to as concept drift, which is often neglected in academic studies. Concept drift leads to performance degradation of an ML model which is not obvious to machine or plant operators, potentially causing unnoticed downstream issues unless addressed properly. Recent process models for structuring industrial ML projects such as CRISP-ML(Q) have started to consider this issue, but do not offer guidance on implementing mechanisms to detect or counteract concept drift. Motivated by this gap, this thesis analyzes methods for the detection of concept drift in the context of ML applications for process and condition monitoring in manufacturing, aiming to improve the application's reliability and acceptance. First, a literature review and expert interviews are conducted to gain insights on concept drift handling in manufacturing research and practice. Consequently, a framework is derived that concretizes the monitoring phase of the CRISP-ML(Q) process model for the target domain by outlining active and passive concept drift detection strategies accompanied by decision criteria for their respective usage. Existing concept drift detection strategies often rely on two-sample tests assuming independent and identically distributed (i.i.d.) data, an assumption that proves invalid in the targeted use cases. Thus, a refinement method called Localized Reference Drift Detection (LRDD) is proposed as a preprocessing step for two-sample testing. The developed framework as well as the proposed LRDD method are validated in terms of applicability and performance through three case studies. In the first case study, a tool condition monitoring scenario is investigated. It is shown how variations in the operating conditions degrade the model performance and how the framework can reliably detect drifts, employing LRDD for active concept drift detection. In the second case study, the use case of process monitoring in milling is analyzed. It is shown that concept drift is present in the dataset due to the aging of the machine components between experiment runs. The drift is reliably detected through the configured active concept drift detection. In the third case study, a condition monitoring use case is implemented within a pigment production line at a company. Within the case study, passive concept drift detection is implemented as the data distributions vary strongly between production batches. It is shown that the passive concept drift detection paired with automatic retraining enables effective condition monitoring of a critical component within the production line. Overall, this thesis provides solution approaches to dealing with concept drift in the manufacturing domain. The proposed methods and decision criteria can either be directly applied to existing use cases or serve as inspiration and guidance for use cases beyond the scope of the case studies.