## 4 Experiments and Results  
In pursuit of our research objectives, The selected language models in this experiment are BERT [36] and MatBERT [37]. These models were chosen because BERT is a solid baseline architecture that is used for many general downstream domain tasks, and MatBERT was specifically trained to handle materials science terminologies, which is relevant to the semiconductor manufacturing industry. Also, we make use of the data sets and the in-domain pre-training pipeline using masked language modeling with two masking strategies (UM and PMI) established by [PERSON] [38] using a semiconductor manufacturing related data set. Specifically, we pre-train BERT and MatBERT for five epochs with a learning rat of 1 \(\times\) 10\({}^{-5}\). The resulted models are named BERT UM, MatBERT UM, BERT PMI, and MatBERT PMI. UM and PMI refer to the masking strategies used in pre-training the language model. We collected a data set containing two types of industrial documents from an actual semiconductor manufacturing company. The data set comprises a set of presentation slides and a set of FMEA documents. The text from the FMEA documents was extracted at the tabular cell level, similar to the work of [PERSON] et al. [4,13]. Mainly, cell texts belonging the failure mode effect and root cause columns were extracted. To extract the textual elements of the presentation slides in their intended order, an optical character recognition (OCR) based method was utilized. Although the OCR-based method produces occasional errors, it represents the best option for this research at this stage. Specifically, the db_resnet50 text recognition model and the crm_mobilenet_v3_large model from DocTR [39] were utilized. The collection of texts from various presentation slides and FMEA documents underwent annotation using the Brat annotation tool [40]. A team of two proficient natural language processing (NLP) experts, with a thorough understanding of the provided annotation guidelines, carried out the annotation process separately. The annotated data set comprises 495 texts from FMEA documents and 440 texts from presentation slides. [PERSON]'s \(\kappa\) and \(F1\) score were used to estimate the IAA, which was calculated on the token level of the text using a language model tokenizer. In addition to the annotated data set, an additional set of 481 texts extracted from presentation slides underwent annotation by only one NLP expert. The IAA is estimated for each label type (cause, effect, and trigger) and for each source of text represented in the data set (i.e., presentation slides and FMEA documents) separately. Table 1 summarizes the results of the IAA for different data sets and tokenization of various language models. Based on Table 1, the overall IAA for all three label types is 85% [PERSON]'s \(\kappa\). However, substantial differences in the IAA can be observed when comparing differentlabels. While there is high agreement in the annotation of triggers with an IAA of 94% [PERSON]'s \(\kappa\), the annotation of causes and especially the annotation of effects proves to be more challenging, with an IAA of 86% and 76% [PERSON]'s \(\kappa\), respectively. The IAA for texts from FMEA documents is generally higher than for texts from presentation slides. The annotated data set is aggregated from the two annotators on the text level, where each text coming from an FMEA cell or extracted from a presentation slide represents one data instance. The annotations of the two annotators are added to this instance. Where there is agreement between the annotators, the duplicated annotations are removed. When there is a disagreement between the annotators, the annotations are aggregated. For the SST method, the different annotations are added to the tokens. In the case of the MST method, the different relations with the same trigger are added as different relations. Table 1 provides an overview of the percentage of tokens associated with the trigger, the cause, and the effect classes in the resulting data set.