## 3 Results
### Text Classification for FMEA's Relations Consistency  
For the purpose of this research, we opted for an end-to-end deep learning classification model. The objective of the model is to predict the concept classes of FMEA cells' text. Based on this prediction, we can apply the same analysis presented in Section 3.1 for assessing the consistency of FMEA's relations consistency presented before. First, we established a data preprocessing pipeline that consists of abbreviation substitution, text cleaning, and text tokenization. Together with domain experts, we collected a set of abbreviations commonly used in the FMEA documents for abbreviation substitution. Additionally, we applied the basic cleaning function to remove punctuation and numbers for the text cleaning. Finally, tokenization based on spaces is applied to FMEA cells' text. Second, we trained a recurrent neural network-based classification model. The model consists of an embedding layer with an input size of the number of distinct words from the training set with a masking function to handle the different lengths, followed by a recurrent neural network layer with 100 hidden units. We test LSTM, Bidirectional LSTM (BiLSTM), GRU, and Bidirectional GRU (BiGRU). Finally, a fully connected layer of five neurons followed by sigmoid functions is established as an output layer. We propose two approaches to evaluate our classification model and method (automatically assessing the relations consistency in FMEA documents). First, we propose conducting a cell-based test training split on the full dataset to evaluate the classifier on unseen FMEA cells. Here, we collect all the FMEA cells from all the FMEA documents, apply the cleaning pipeline and remove duplicated cells. Second, we propose to do document-based splitting. Here, we divided the FMEA documents into training and testing in the training phase. The criteria are made under the assumption that if we have annotated  
Figure 4: Comparison of concepts class distribution over P_FME vs. UP FMEA cells. As expected, the Functional concept class is poorly represented in UP_FMEA. In contrast to the expectations, the most common concept class in both UP_FMEA and P_FMEA cells is the physical concept class. Figure 5: Comparison of the relation types distribution in P_FMEA and UP_FMEA. Relation inconsistencies are present in both types of FMEAs (i.e., P_FMEA and UP_FMEA). In UP_FMEA, the relations with missing information are more common. FMEA documents of a specific distribution, we evaluate the proposed method's performance in assessing a new FMEA document from the same distribution. Thus, we use one P_FMEA document and one UP_FMEA for testing. We use four-fold cross-validation to test the classification performance. This approach is used to test the effectiveness of the proposed method for assessing the relation consistency. The first evaluation strategy targets the classification performance removing any biases caused by a potential target leak. Similarly, we use four-fold cross-validation to test the classification performance. Based on the first evaluation approach, we noticed that all the classifiers' architecture performances drop with respect to minority classes (i.e., the lower three classes that are the least represented in the collected dataset). However, with respect to the Functional and the Physical concept classes, LSTMS and Bidirectional LSTMS surpass the other architecture by a small margin. The results of the comparison are depicted in Figure 6 and summarized in Table 1. Additionally, Table 1 (a full table of results) is added to Appendix A. Moreover, we used the second approach of splitting (documents-based splitting) to test the effectiveness of the proposed method for assessing the relation consistency. The results of each FMEA relation type detection are summarized in Table 2.