# Structure-Aware and Format-Enhanced Transformer for Accident Report Modeling
## 7 Discussion  
This study systematically verifies the effectiveness of incorporating structure-aware and format-enhanced strategies into the Transformer architecture for modeling accident investigation reports, which are characterized by high structural complexity and varying document lengths. Experimental results demonstrate that SAFE-Transformer significantly outperforms representative baselines such as RoBERTa [65], Longformer [32], and Big-Bird [31] across overall metrics and in handling long-tail multi-label categories. Notably, its advantages in P@5 and Macro-F1 underscore its strong ability to capture critical semantic fragments and rare labels. In contrast to traditional long-text approaches based on truncation or random sparsity, SAFE-Transformer explicitly parses section numbers and headings to transform symbolic-level information into global semantic anchors, enabling it to maintain stable performance across different document lengths. This finding aligns with recent studies [62; 63; 64] highlighting the benefits of explicit structural modeling in enhancing text reasoning and generation quality, further confirming the synergistic value of combining structural priors with sparse attention mechanisms. The observed inverted U-shaped performance trends across short, medium, and long documents offer important insights into long-text modeling strategies. While dense attention remains irreplaceable for fine-grained precision in short texts, structurally guided sparse attention achieves a better trade-off between efficiency and performance as sequence length increases. This suggests that future accident text analysis systems could adopt adaptive encoding strategies based on document length to balance accuracy and responsiveness. Additionally, ablation results show that the structural parsing layer and hierarchical sparse attention contribute most to performance gains, while gated fusion and symbol-aware encoding serve as stabilizers. These findings highlight the importance of multi-module synergy and provide a clear blueprint for future modular expansions. Despite promising results, several limitations should be acknowledged. First, the rule-based parser covering 38 directory patterns may not generalize well across regions, industries, or multilingual settings. Future work could explore differentiable parsers or few-shot prompt learning to reduce template dependence. Second, although the SAFE-Transformer effectively extracts accident intelligence, it does not explicitly address privacy protection--a crucial issue in Europe and many other countries. Given that accident reports frequently contain sensitive information, future research should investigate privacy-preserving strategies such as anonymization mechanisms or differential privacy integration into the Transformer architecture. Third, the current evaluation is limited to multi-label classification. The model's generalizability has not yet been validated in more complex applications such as causal chain extraction, evidence tracking, and abstractive summarization, which demand more sophisticated cross-hierarchical reasoning and long-range alignment. Furthermore, regarding causative agents in the \"Accident profile\", the current annotation framework may not fully align with the European Statistics on Accidents at Work (ESAW) model, potentially limiting cross-national applicability. Consequently, additional comparative studies are required to evaluate and potentially enhance the data modeling approach in diverse international contexts. Fourth, although SAFE-Transformer reduces computational complexity relative to full-attention models, its inference latency on general-purpose GPUs remains higher than that of operator-optimized dense models. Further acceleration may require integration with dynamic sparse kernels, adaptive pruning, or hardware-friendly positional encoding.