## 5 Discussion  
The suggested annotation guidelines have demonstrated their efficiency, as reflected by the high level of agreement between annotators on the annotated data set. The results show a higher IAA on the triggers compared to the causes and effects. This difference in agreement may be due to the fact that triggers usually consist of only a few tokens, while causes and effects tend to be longer and have less clear-cut and unambiguous boundaries. Additionally, the IAA for FMEA is higher than that for presentation slides. This difference in IAA can be attributed to the fact that the content in FMEA documents is typically more structured and organized than in presentation slides. FMEA documents often follow a standard format and contain specific sections related to failure modes, effects, and causes. As a result, annotators may find it easier to identify and annotate these sections consistently. In contrast, presentation slides often contain unstructured and varied content, with information presented in different formats and in no particular order. This variability can make it more challenging for annotators to identify and annotate the relevant content consistently.  
Two types of causal information extraction methods, based on SST and MST, are also presented in the study. The SST approach leverages BERT-based language models and supports the extraction of overlapping entities. This method is simple to implement as it only requires a multi-label token classifier. However, this approach is limited when it comes to extracting enchained and nested relations and disrupted entities. The MST addresses these limitations and achieves good performance, sufficient for practical applications, with \(F1\_score>90\%\) on FMEA documents and \(F1\_score>70\%\) on presentation slides. Both methods perform better on texts from FMEA documents than on texts from presentation slides. This could be attributed to the same finding from the IAA, as presentation slides are more difficult to annotate. Additionally, this could be attributed to the small data set size, as the training data set for presentation slides is less representative of its complexity compared to the FMEA documents.  
Choosing a language model that is more aligned with the domain has shown to improve the performance of both the SST and MST methods. However, the effect of in-domain pre-training on the model performance is dependent on the test data set (i.e., FMEA or presentation slides) and the initial language model. In-domain pre-training can be an effective method for further improving the performance of the SST and MST methods. Specifically, PMI has consistently improved the model performance across different base models by incorporating domain-specific language into the masking objective. Of course, this enhancement in model performance comes at the cost of reduced generalizability of the in-domain pre-trained model to other domains, which does not pose an issue as it is quite rare to share in-domain pre-trained models between domains. However, the performance improvement seems to differ for the different entity types, with the F1 score for effect increasing more than the F1 score for cause. This could indicate that effect entities are more domain-specific than the cause entities. However, the effectiveness of in-domain pre-training is highly dependent on the initial language model and the data set used for pre-training. Overall, our findings reflect the importance of careful consideration of the language model and pre-training approach for achieving high-quality causal information extraction from unstructured documents in the semiconductor industry.