## 3 Results  
### FMEA Documents Dataset Description and FMEA Relation Consistency Assessment  
For this research, we opted for the strategy to initially generate a seed dataset, upon which consecutively a classifier is trained, which finally can automatically annotate the complete available documents. Thus, we collected a sample of FMEA documents that domain experts annotated for this research. The annotation process is conducted using an annotation tool. This annotation tool shows the complete cells' text and allows for multilabeled FMEA cells' text annotation. For a FMEA cell text that does not belong to the defined classes or is ambiguous, domain experts are given the option to annotate the text as out of the scheme. The experts annotated four P_FMEA and nine UP_FME. In P_FMEA, the number of distinctive cells is five times higher than the number of distinctive cells in UP_FMEA. Based on the annotation, 10% and 18% of the P_FMEA and UP_FMEA cells are annotated as out of scheme, respectively. These cells are excluded from further analysis. The comparison of concepts class distribution over P_FME vs. UP FMEA cells is depicted in Figure 4.  
Based on the annotation acquired from the experts, we can annotate the relations between the cells present in the FMEA documents. Here, we distinguish five types of annotated relations. The first type is considered to be consistent relation if the head cell and tail cell concept classes conform to the sequence in the classification scheme. The second type is considered to be a relation with missing information (Missing Information relation) if the head cell and tail cell concept classes do not conform to the classification scheme sequence, skipping concepts belonging to one or more of the concept classes (e.g., 5M to Functional). The third type is considered to be a relation with reversed direction (Reversed relation) if the head cell and tail cell concept classes do not conform to the sequence in the classification scheme and link concepts in the reversed direction (e.g., Physical to Functional). The fourth type is considered to be a relation within the same concept (Same Concept relation) if the head cell and tail cell concept belong to the same class. In this case, we cannot determine any information about the relation consistency. Moreover, a relation is considered out of scheme (Out of scheme relation) if it connects to a head cell or a tail cell that has annotation out of scheme. It should be noted that for the comparison of the head cell concept classes and the tail node concept classes, we compare the highest (closest to the functional concept class) concept classes from the tail cell to the lowest (closest to 5M concept class) concept class for the head cell.  
Based on the analysis of the relations present in P_FMEA and UP_FMEA, the percentages of relations that are out of scheme in P_FMEA and UP_FMEA are 37% and 22%, respectively. Additionally, the number of relations in P_FMEA is six times higher than UP_FMEAs. Moreover, we noticed that both types of documents have consistency impairments concerning the causal relation. Additionally, the percentages of the relations between concepts within the same concept class and those that indicate reversed direction are comparable between the UP_FMEA and P_FME documents. However, in UP_FMEA, the relations with missing information are more present. Thus, we think that the definition of the scope of UP_FMEA is insufficient. Consequently, an increasing number of relations with missing information are observed. The comparison of relation type distribution present in P_FMEA and UP_FMEA is depicted in Figure 5.