# Data-Driven Fault Detection and Diagnosis: Challenges and Opportunities in Real-World Scenarios
###### Abstract  
The pervasive digital innovation of the last decades has led to a remarkable transformation of maintenance strategies. The data collected from machinery and the extraction of valuable information through machine learning (ML) have assumed a crucial role. As a result, data-driven predictive maintenance (PdM) has received significant attention from academics and industries. However, practical issues are limiting the implementation of PdM in manufacturing plants. These issues are related to the availability, quantity, and completeness of the collected data, which do not contain all machinery health conditions, are often unprovided with the contextual information needed by ML models, and are huge in terms of gigabytes per minute. As an extension of previous work by the authors, this paper aims to validate the methodology for streaming fault and novelty detection that reduces the quantity of data to transfer and store, allows the automatic collection of contextual information, and recognizes novel system behaviors. Five distinct datasets are collected from the field, and results show that streaming and incremental clustering-based approaches are effective tools for obtaining labeled datasets and real-time feedback on the machinery's health condition. p 2022129912020 August 202020 August 20201 November 10 202  
more reliable [7]. On the one hand, knowing the system's health condition in a given instant leads to maximizing the production profits and minimizing all costs and losses by implementing efficient and just-in-time maintenance strategies [8]. On the other hand, data-driven PdM requires the collection of large volumes of historical data from multiple sources and proper big data processing technologies to build an integrated environment in which production and maintenance processes can be managed more efficiently [9]. The large body of existing literature on PdM mainly focuses on improving the accuracy of data processing through machine learning (ML) and deep learning (DL) algorithms. In particular, several data-driven fault detection and diagnosis approaches are developed and applied to different domains. Supervised models, such as shallow or deep artificial neural networks (ANNs) [10; 11], are largely adopted for rotating machinery fault diagnosis. In these applications, signals are collected from the component in one or more health conditions and the goal is to build a classification model that classifies future signals into one of those conditions with the highest accuracy. Therefore, all fault classes are known in advance. Unsupervised models, such as k-means [12] or autoencoders [13], are used for fault detection, whose aim is to automatically detect any fault in its early stage [14]. In these applications, only two classes, health or damaged, are often considered. Therefore, diagnosing the detected fault is not automatic but requires human intervention. In addition, industrial artificial intelligence deals with industrial big data, characterized by the so-called 3 Bs challenges, broken data, bad quality data, and background [15]. In the big challenge to fit the AI in industries and implement data-driven PdM, the more significant issues are related to the large amount of data generated by machines, the lack of contextual information, the unbalanced datasets, and the dynamic nature of industrial environments [16]. Consequently, latency, bandwidth, and network scalability limit the real-time monitoring of machinery. The absence of labels indicating the actual health condition and the lack of data in all faulty modes restrict the use of supervised ML models [17]. Finally, the accuracy of these models reduces with time because of their poor ability to learn from current data and adapt to multiple scenarios. Given these issues, the traditional offline and batch fault diagnosis, which relies on many labeled data, may be ineffective and impracticable in real-world applications.