# A Text Understandability Approach for Improving Reliability-Centered Maintenance in Manufacturing Enterprises

Theresa Madreiter, Linus Kohl, Fazel Ansari

► To cite this version:

Theresa Madreiter, Linus Kohl, Fazel Ansari. A Text Understandability Approach for Improving Reliability-Centered Maintenance in Manufacturing Enterprises. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2021, Nantes, France. pp.161-170, 10.1007/978-3-030-85874-2_17. hal-04030370

HAL Id: hal-04030370

https://inria.hal.science/hal-04030370

Submitted on 15 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# A Text Understandability Approach for Improving Reliability-Centered Maintenance in Manufacturing Enterprises

Theresa Madreiter¹, ² (✉) [0000-0002-8096-2060], Linus Kohl¹, ² [0000-0002-3019-4403] and Fazel Ansari¹, ² [0000-0002-2705-0396]

¹ Research Group of Smart and Knowledge-Based Maintenance, TU Wien

²Fraunhofer Austria Research GmbH

{theresa.madreiter, linus.kohl, fazel.ansari}@tuwien.ac.at

**Abstract.** Textual data majorly reflects objective and subjective human specific knowledge. Focusing on big data in industrial and operation management, the value of textual data is oftentimes undermined. Optimal use of data reinforces the integrative modeling and analysis of RAM (Reliability, Availability, Maintainability). Data-driven reliability engineering and maintenance management, gains benefit from textual data, especially for identifying unknown failure modes and causes, and solving problems. The scientific challenge is how to effectively discover knowledge from text data and convert it into automated processes for inferential reasoning, predicting and prescribing. This paper outlines how the reliability-centered maintenance in production systems can be improved by explicating and discovering human-specific knowledge from maintenance reports and related textual documents. Hence, a theoretical model for text understanding is proposed, which is demonstrated as a proof-of-concept demonstrator using real world manufacturing datasets. The text understanding model is represented by a three-dimensional matrix comprising three indexes, i.e. text readability, word associations within texts as well as sentiment. The implementation of the model as a software prototype involves using text mining techniques and machine learning algorithms. This paper emphasizes on the importance of knowledge extraction from text in the context of industrial maintenance, by demonstrating how an increased value of text understandability of maintenance reports correlates to an early stage detection of failure, the reduction of human failures and leads to an immense improvement of explication of human knowledge.

**Keywords:** Reliability, Maintenance, Human Failure, Text Mining, Industry 4.0.

## 1 Introduction: Unexhausted Potentials of Unstructured Data in Reliability Engineering and Maintenance

A high failure rate in an industrial system ultimately leads to high instability and low-efficiency in production, high operation costs, and poor process quality [1]. Failure rate is, therefore, a key indicator for evaluating a successful maintenance management
strategy in reliability-centered manufacturing. In the era of Industry 4.0, success in
terms of reliability engineering and maintenance management i) requires the reinforce-
ment of data-driven strategies focusing on reliability, availability and maintainability
(RAM), and thus ii) demands knowledge-based maintenance (KBM) strategies encom-
passing both predictive and prescriptive maintenance approaches [2]. KBM employs
artificial intelligence (AI) for integrative analysis, modeling, predicting and reducing
the likelihood of failures and thus increasing reliability and availability in production
systems. Gaining benefits from multi-channel, multi-structured data sources, is thereby
essential for thriving data-driven reliability engineering and KBM. Plausible imple-
mentation of data-driven reliability engineering and KBM across manufacturing enter-
prises, however, confront the following challenges, namely: i) inappropriate and inef-
ficient use of multiple data sources, ii) suboptimal use of multi-structured data, iii) mul-
timodality of data i.e. missing semantic correlation of information, iv) multiple and
overlapping reliability-centered and maintenance strategy approaches, v) multidimen-
sionality of maintenance organizations including actors and backend/frontend teams,
processes and IT-systems, and vi) lack of benchmarked use-cases and economically as
well as technically evaluated KBM approaches in industrial applications.

With the integration of IoT-based and embedded sensory systems in Cyber Physical
Production Systems (CPPS) [3] as well as constantly evolving AI technologies, new
possibilities for industrial application of data-driven reliability engineering and KBM
strategies arise. However, in the literature of production and operation management,
data-driven reliability engineering and KBM are mostly understood as umbrella terms
referring to Condition based Maintenance (CbM) and Predictive Maintenance (PdM),
i.e. applying statistical- and machine learning (ML) on structured data in order to opti-
mize RAM [2]. Often the value of unstructured data, in particular text, is undermined
or totally ignored [2]. This is mainly due to its unstructured data format, thus requiring
pre-processing for converting text into machine-readable formats. Additionally,
maintenance reports (e.g. log books, shift books, failure reports, repair protocols, etc.)
are often characterized by informal language, incomplete sentences, incorrect syntax
and technical terminologies, making knowledge extraction even more challenging.
Therefore, standard text mining (TM) approaches are not sufficient for this type of text.
In real-world manufacturing systems, however, the primary data type found in indus-
trial databases is textual data, majorly reflecting objective and subjective human spe-
cific knowledge [4]. When pursing data-driven reliability engineering and KBM ap-
proaches focusing strictly on structured data, the potential of a large portion of relevant
information is remained unexhausted. This reflects the fundamental problem of indus-
trial maintenance in today’s production systems, that relevant, up-to-date, and compre-
hensive knowledge is missing, which avoids achieving informed decisions [2]. In other
words, informed decision-making should be based on multiple data sources and multi-
modal data, including structured and unstructured formats. In addition, optimal use of
multiple data sources and multi-structured data may reinforce the integrative modeling
and analysis of RAM indicators. Data-driven reliability engineering and KBM, there-
fore, may gain benefit from textual data, especially for identifying unknown failure
modes and causes as well as improving troubleshooting processes for solving (non-
routine/routine) problems and therefore lowering the human failure rate. The scientific
challenge is how to effectively discover knowledge from text data and convert it into
automated processes for inferential reasoning and predicting the moment of failure, and
ultimately prescribing appropriate handling measures timely. In order to exploit the full
potential of text and therefore, overcome the quality obstacles of maintenance reports,
text understandability should be achieved, i.e. going beyond standard TM approaches
(cf. Section 2).

Considering the discussion above the following research question emerges, “How
can text understandability (TU) in industrial maintenance contribute to an increased
reliability and thus, a lowered failure rate?”. The objective of this paper is, therefore, to
i) propose a concept for TU in the context of industrial maintenance, focusing on crea-
tion of an objective function representing multidimensional TU and ii) provide a proof-
of-concept software prototype for demonstrating plausible industrial application of the
aforementioned concept.

The rest of the paper is structured as follows: Chapter 2 provides a literature review on the necessity of TU in maintenance. Chapter 3 introduces the concept of TU, its industrial proof-of-concept implementation and expected industrial impact. Finally, Chapter 4 concludes the discussion and identifies the pathways for future research.

## 2 State-of-the-Art and -Practice: Why Does Text Understandability in Maintenance Matter?

Knowledge extraction and discovery from text is used to discover hidden patterns and valuable information using AI algorithms [4]. Where knowledge discovery focuses on identifying and understanding valid, novel and useful patterns in data [5], TM applies this approach to machine supported analysis of text. TM involves information retrieval and extraction technique as well as natural language processing (NLP) combined with algorithms from machine learning and statistics to process and analyze unstructured data sources [6]. This enables (semi-)automated identification of relevant words in those texts using Named Entity Recognition (NER) and Part-of-Speech (POS) Tagging. Linking these words to quantitative information (e.g. costs, production quantities) can further enhance text understanding and therefore lead to increased knowledge gain [7]. These matched entities can then be used in subsequent steps to enrich databases and ML models further [8]. A typical TM pipeline for extracting relevant knowledge from one or more data sources includes the following steps: i) extracting the text from documents or web pages [9], ii) preprocessing and iii) extracting domain-specific information using advanced techniques like NER [8]. This entity relation extraction allows linking entities labelled by domain-specific tags and the information related to them.

However, a major challenge for the extraction of knowledge from maintenance reports is their unique form, often not meeting standard text quality measures in terms of syntax and semantics. Maintenance reports often feature informal language, special characters, individual abbreviations, domain (company) specific expressions and incomplete sentences [7, 10, 11]. Additionally, maintenance reports are often multilingual featuring English as well as non-English expressions and thus the quality of reports often depends on the qualification and competence level of maintenance operators [7].
Effective knowledge extraction requires handcrafted solutions for measuring associations, and the structured representation of domain-specific context [11]. In order to overcome the limitations of current approaches, the human (expert) ability to understand text should be reproduced by algorithms. Therefore, the knowledge extraction from maintenance reports requires text understandability methods, rather than a standard TM pipeline.

Despite foreseeable advantages of text understandability, the body of literature in maintenance mainly reflects knowledge extraction and discovery from textual data, using TM. The majority of the state-of-the-art approaches restrictively aims to extract specific information from text rather than fostering TU. Notable examples are summarized in Table 1, which consider textual maintenance data in various industries, trying to extract specific information, e.g. the time of failure or frequent occurring incidents.

<table><caption>**Table 1.** Research considering TM in maintenance</caption><thead><tr><td>Key statement</td><td>Extracted Information</td><td>Industry</td></tr></thead><tbody><tr><td>increased accuracy of failure times for reliability using TM [12]</td><td>time of failure</td><td>energy industry<br>food industry</td></tr><tr><td>detection of causalities and comprehension of incident progress patterns using NLP [13]</td><td>flows of events of accidents</td><td>space industry</td></tr><tr><td>extraction of information about component failure patterns using TM, which enables the identification of frequent warning and failure incidences [14]</td><td>components causing frequent warnings/failures</td><td>facility management</td></tr><tr><td>frequent incidents detection in building sectors based on characteristics using text analytics [15]</td><td>frequent incidents</td><td>facility management</td></tr></tbody></table>

In order to fully comprehend textual data in maintenance and exploit the comprised knowledge, a close to human-like TU model for the analysis of textual documents should be established. According to psycholinguistics, the human's TU is a complex and dynamic process, that takes place on various levels, including a syntactic and a semantic level [16]. In order to comprehend text, humans build and access a complex inner dictionary, storing syntactic and semantic information as well as generated inference, by linking previously gained knowledge with information from recent events [17, 18]. As a first approach a compositional framework for text understanding has been introduced by Ansari [7], focusing on text analysis in industrial maintenance.
# 3 Text Understandability in Maintenance: Conception and Industrial Proof-of-Concept Implementation

## 3.1 Text Understandability as an Objective Function

TU is defined as an objective function representing the multidimensionality of automated text comprehension, where multiple dimensions and related criteria identified mainly from language; syntax, semantics, and context as well as target objective function (e.g. cost, quality, time, productivity, etc.) contribute to a holistic modeling of TU. In this paper, the multidimensionality of TU is limited to three dimensions i) text readability, ii) extracting hidden sentiments and iii) existing associations, where each dimension increases the comprehension of text. Thereby, text readability, assures the interpretability of textual data, sentiment analysis reveals an author's opinion towards a certain topic or event, and association measurement extracts associable terms and expressions to enable further inference generation through linking newly extracted information with previously stored knowledge. TU can, therefore, be visualized as vector in a n-dimensional space, representing the relations between its features. Fig. 1 depicts the n-dimensionality while the proposed objective function is limited to three dimensions.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Representation of a 3-D objective function for TU [19]</figcaption></figure>

## 3.2 Procedural Model for Realizing Text Understandability

Based on the aforementioned concept, a procedural model for the realization of TU in the use case of industrial maintenance has been designed, as illustrated in Fig. 2. The concept is based on a standard maintenance process, where after the repair of failure, a maintenance report is written. Given an input maintenance report, the text readability is evaluated, since the extraction of relevant information requires the fulfillment of at least some basic readability criteria. However, readability standards applied to textual data depend on the information that needs to be extracted from text. Notably, generic quantitative measurements for text readability to evaluate the text readability from a human's perspective (e.g. Flesch-Kincaid GradeLevel Formula) assess text readability based on very limited factors (i.e. number of words and syllables) [20]. Therefore, the
text readability proposed in this paper enhances and adapts existing approaches for the evaluation of human text readability [21] and additionally incorporates indicators for machine readability (i.e. unusual punctuations or spelling mistakes), that prevent the usage of automatic syntax parsers [21, 22]. In particular, text readability is evaluated through i) text length, ii) detectable language, iii) usage of special characters as well as iv) correct spelling, enabling the exclusion of uninterpretable reports, and thus a higher accuracy of recommended reports.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Procedural model for TU in the use case of industrial maintenance [19]</figcaption></figure>

Further, psycholinguistics suggest that humans achieve text comprehension and further inference generation through linking newly extracted information with previously stored knowledge. On this bases, association measurement takes place within reports to identify associative features (i.e. machine, operator, etc.). For each single report and the selected feature, comparing to all reports belonging to the selected feature, the most associable expressions and bigrams are being extracted using term-frequency (TF) and bigram-frequency (BF). The applied approach is based on term frequency – inverse document frequency (TF-IDF), a method successfully applied in the field of law to measure similarities among legal documents [23]. Due to the short texts and the initial removal of stop words, TF and BF are used due to achieving better results than TF-IDF. Within a report, the context dependent associability of used words and bigrams to a specific feature, is reflected by an Association Measurement Index (AMI) calculated based on TF and BF.

Sentiment analysis is frequently used in social media to gain insights on how users feel about certain topics [24]. However, the application of sentiment analysis is not limited to social media and can also be applied to technical documents. Applying
sentiment analysis on maintenance reports reveals the opinion of maintenance employ-
ees towards certain maintenance actions and provides further information especially in
terms of very short reports [7]. Due to the short text length of maintenance reports and
their similarity to short social media posts, the proposed approach employs supervised
ML algorithms, which are often effectively applied to determine the sentiment of tweets
[24]. Certain classifiers such as Support Vector Machine (SVM), Logistic regression
(LR) and Bernoulli naïve bayes (BNB) have been trained on pre-annotated test datasets.
Due to achieving the highest accuracy (BNB: 78.1%, SVM: 75.0%, LR: 72.3%) on a
pre-annotated test dataset, the BNB classifier is used to determine the sentiment for
each processed maintenance report.

In a last step the TU is calculated based on text readability, the AMI as well as the
assessed sentiment. Then the TU, as well as the extracted understandability features,
are displayed in a dashboard for each report.

3.3 Development of Proof-of-Concept (PoC): TU-MARS Software

Based on the proposed procedural model, a software demonstrator for “Text Under-
standability by Measuring Associations, Readability and Sentiment (TU-MARS)” has
been implemented. TU-MARS analyzes maintenance reports from the semiconductor
industry in the format of unstructured free texts, written by maintenance operators after
the fix of an incident. The to-be analyzed reports are characterized by their i) short text
length, ii) incorrect syntax and iii) often-missing semantics due to being quickly written
in case of incident correction. Additionally, metadata such as the affected machine and
the classified down event are provided. TU-MARS analyzes previously written mainte-
nance reports and displays its TU measures as well as its multidimensional TU aspects,
including text readability statistics, the sentiment and associable terms and bigrams in
a dashboard, for a chosen equipment and an occurred down event (see Fig. 3).

<figure><img src="image_5.png" /><figcaption>Fig. 3. TU-MARS dashboard for TU [19]</figcaption></figure>
3.4 Potential Impact of TU in Industrial Maintenance

Through determining the TU of maintenance reports, knowledge from text can be in-
cluded in informed decision-making processes, positively affecting industrial mainte-
nance decisions and planning measures. A maintenance process usually starts with the
occurrence of an incident on a specific machine and the classification of the failure by
a machine operator. TU-MARS is able to identify highly interpretable and relevant pre-
viously written reports that have led to successful troubleshooting in the past, based on
TU. For a specific machine and the occurred down event, maintenance operators are
provided with the most fitting maintenance reports, based on their TU. Considering the
displayed TU-measures in Fig. 3, the selected recommended report shows a TU of
85.69% due to the high interpretability (text readability: 83.33%), high relevancy (AMI:
73.75%) as well as a successful outcome (sentiment: pos) of the report. Based on the
displayed associable maintenance actions and the domain specific knowledge of the
maintenance technician, efficient maintenance operations for the current failure event
can be derived and adopted. The proposed solution is based on the same dataset used
by Ansari et. al. [25], where the authors were able to increase the uptime by 6.7% and
the mean failure detection time by 97.3%, based on word recommendations for a clas-
sified failure event, and recommending the best-fitting maintenance operator. TU-
MARS does not only provide word recommendations, but also previously written re-
ports featuring a high interpretability, that lead to successful maintenance operations in
the past and are highly associable to the current failure. Therefore, it can be expected,
the mean failure detection time will increase by at least additional 10%, while also low-
ering the human failure rate up to 15%, by providing guidance during the maintenance
operation. Thereby, each TU dimension contributes valuable information, enabling an
increase of knowledge exploitation and potentially leading to a reduced equipment
downtime, lowered human failure rate and earlier failure detection (cf. Fig. 4).

<figure><img src="image_6.png" /><figcaption>Fig. 4. Impact of TU on industrial maintenance</figcaption></figure>
# 4 Conclusion and Future Research Agenda

This paper introduces a transferable and scalable concept and PoC for TU, demonstrat-
ing the potential of AI-enhanced text analytics in industrial maintenance. Compared to
already existing approaches, TU offers a novel approach of knowledge extraction from
text, by enabling multilevel text comprehension. TU-MARS displays beneficial mainte-
nance reports and relevant technical terms, specifically for affected machines and oc-
curred incidents, which the maintenance technician has to repair. The proposed concept
is capable of reducing the human failure rate and time to failure detection, thus achiev-
ing an increased reliability, by effectively using hidden knowledge stored in mainte-
nance reports. It ultimately enhances informed decision making in maintenance and
production planning. Although, the proposed concept addresses the opportunities pro-
vided by TU in maintenance, the proposed approach needs to be further developed and
verified dealing with the following industry-oriented and scientific challenges:

* Exploring economic significance of text analytics in maintenance, so that information from textual data are incorporated in production and maintenance planning and knowledge loss is prevented,

* Establishing context specificity for text analytics in technical environments, since standard TM solutions are not suitable to handle context-specific terminologies or abbreviations, and

* Implementation of feedback loops, so that the analytical results can be validated by domain experts in order to increase the value of proposed results of TU-MARS.

Finally, yet importantly, TU-MARS will be further developed by introducing further syntax-, semantic-, language- and context-specific dimensions focusing, but not limited to, industrial maintenance.

References

1. Zhang, D., Zhang, Y., Yu, M., Chen, Y.: Reliability evaluation and component importance measure for manufacturing systems based on failure losses. J Intell Manuf (2017) https://doi.org/10.1007/s10845-015-1073-1

2. Ansari, F., Glawar, R., Nemeth, T.: PriMa: a prescriptive maintenance model for cyber-physical production systems. International Journal of Computer Integrated Manufacturing (2019). https://doi.org/10.1080/0951192X.2019.1571236

3. Ansari, F., Hold, P., Sihn, W.: Human-Centered Cyber Physical Production System: How Does Industry 4.0 impact on Decision-Making Tasks? IEEE, Piscataway, NJ (2018)

4. Gandomi, A., Haider, M.: Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management (2015) https://doi.org/10.1016/j.ijinformatmgt.2014.10.007

5. Hotho, A., Nürnberger, A., Paass, G.: A Brief Survey of Text Mining. LDV Forum 20, 19–62 (2005)

6. Geierhos, M., Bäumer, F.S.: Text Mining. https://www.enzyklopaedie-der-wirtschaftsinformatik.de/lexikon/technologien-methoden/text-mining (2020). last accessed 2021/06/12
7. Ansari, F.: Cost-based text understanding to improve maintenance knowledge intelligence in manufacturing enterprises. Computers & Industrial Engineering (2020). https://doi.org/10.1016/j.cie.2020.106319

8. Sumithra, M.K., Sridhar, R.: Information Retrieval in Financial Documents. In: Evolving Technologies for Computing, Communication and Smart World, vol. 694. Lecture Notes in Electrical Engineering, pp. 265–274. Springer Singapore, Singapore (2021)

9. Ansari, F., Fathi, M., Chala, S.A.: Towards implementing context-aware dynamic text field for web-based data collection. IJHFE (2016) https://doi.org/10.1504/IJHFE.2016.10003149

10. Strack, B., Lenart, M., Frank, J., Kramer, N.: Ontology for maintenance of onshore wind turbines. Forsch Ingenieurwes (2021) https://doi.org/10.1007/s10010-021-00466-x

11. Alfeo, A.L., Cimino, M.G.C.A., Vaglini, G.: Technological troubleshooting based on sentence embedding with deep transformers. J Intell Manuf (2021) https://doi.org/10.1007/s10845-021-01797-w

12. Arif-Uz-Zaman, K., Cholette, M.E., Ma, L., Karim, A.: Extracting failure time data from industrial maintenance records using text mining. Advanced Engineering Informatics (2017) https://doi.org/10.1016/j.aei.2016.11.004

13. Nakata, T.: Text-mining on incident reports to find knowledge on industrial safety. In: 2017 Annual Reliability and Maintainability Symposium (RAMS). 2017, pp. 1–5. IEEE (2017)

14. Gunay, H.B., Shen, W., Yang, C.: Text-mining building maintenance work orders for component fault frequency. Building Research & Information (2019) https://doi.org/10.1080/09613218.2018.1459004

15. Bortolini, R., Forcada, N.: Analysis of building maintenance requests using a text mining approach: building services evaluation. Building Research & Information (2020) https://doi.org/10.1080/09613218.2019.1609291

16. Osterhout, L., Kim, A., Kuperberg, G.: The Neurobiology of Sentence Comprehension. The Cambridge Handbook of Psycholinguistics, pp. 365–389 (2008)

17. Aitchison, J.: Words in the mind. An introduction to the mental lexicon, 4th edn. Wiley-Blackwell, Chichester, West Sussex, Malden, MA (2012)

18. Graesser, A., Britton, B.: Five metaphors for text understanding Models of understanding text, pp. 341–351 (1996)

19. Madreiter, T.: Design and Development of a Prototype of a Text Understanding Tool for Maintenance 4.0 by Measuring Associations, Readability and Sentiment (TUMARS). Master Thesis, Vienna University of Technology (2020)

20. Kincaid, J.P., Fishburne Jr., R., Rogers, R., Chissom, B.: Derivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training (1975)

21. Collins-Thompson, K.: Recent Advances in Automatic Readability Assessment and Text Simplification. ITL (2014). https://doi.org/10.1075/itl.165.2.01col

22. Kiefer, C.: Assessing the Quality of Unstructured Data: An Initial Overview. LDWA 2016 (2016)

23. Mandal, A., Chaki, R., Saha, S., Ghosh, K., Pal, A., Ghosh, S.: Measuring Similarity among Legal Court Case Documents. In: Proceedings of the 10th Annual ACM India Compute Conference on (2017) https://doi.org/10.1145/3140107.3140119

24. Poornima, A., Priya, K.S.: A Comparative Sentiment Analysis Of Sentence Embedding Using Machine Learning Techniques. In: 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS). pp. 493–496. IEEE (2020)

25. Ansari, F., Kohl, L., Giner, J., Meier, H.: Text mining for AI enhanced failure detection and availability optimization in production systems. CIRP Annals (2021) https://doi.org/10.1016/j.cirp.2021.04.045