# Structure-Aware and Format-Enhanced Transformer for Accident Report Modeling
## 6 Result
### Ablation Study  
To evaluate the contributions of each module within SAFE-Transformer, we conducted an ablation study using a controlled variable method. Specifically, we removed the structural parsing layer (Struct), symbol-aware positional encoding (SymEnc), hierarchical sparse attention (HierAttn), and gated semantic fusion (GateFuse) modules one at a time and compared performance changes. All experiments were conducted on accident reports with a sequence length of 2048. The results are presented in Table 6.  
Structural Parsing Layer: Removing the structural parsing layer (SAFE w/o Struct) led to the most significant performance degradation. Micro AUC dropped by 3.63% and P@5 decreased by 5.28%, indicating that explicitly encoding document-level hierarchical syntax (e.g., numbering structures like \"1.2 \(\rightarrow\) (a)\") is critical for effective long-text modeling. The performance drop primarily stems from the model's inability to distinguish the logical relationships between section headings and body paragraphs, resulting in semantic confusion. For instance, in predicting labels related to liability attribution, the model without structural parsing may mistakenly associate content under \"5.1 Direct Causes\" with the unrelated heading \"6. Corrective Measures\".  
Symbol-Aware Positional Encoding: Disabling symbol-aware encoding (SAFE w/o SymEnc) resulted in a 2.17% drop in Micro AUC and a 3.17% decrease in P@5. This supports the importance of capturing the semantic heterogeneity of symbolic cues (e.g., \"1.\" vs. \"1\"). For example, in a paragraph labeled \"Qualification Review\", the model may incorrectly classify \"2) Unlicensed Operation of Special Equipment\" as related to \"Construction Plans\", failing to recognize that \"2)\" corresponds to a secondary clause of liability.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model Variant** & **Micro AUC** & **Macro AUC** & **Micro F1** & **Macro F1** & **P@5** \\ \hline SAFE-Transformer & 73.45 & 70.12 & 70.23 & 68.82 & 76.33 \\ SAFE w/o Struct & 69.82 & 67.05 & 67.12 & 65.74 & 71.05 \\ SAFE w/o SymEnc & 71.28 & 68.93 & 68.35 & 66.82 & 73.16 \\ SAFE w/o HierAttn & 70.15 & 68.11 & 67.84 & 66.02 & 72.47 \\ SAFE w/o GateFuse & 71.03 & 69.24 & 69.17 & 67.35 & 74.02 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Ablation Study Results.  
Hierarchical Sparse Attention: Replacing hierarchical sparse attention with full attention (SAFE w/o HierAttn) led to a 3.30% drop in Micro AUC and a 21.7% increase in computation time. The full attention mechanism introduces cross-hierarchical noise--such as interference from non-direct parent nodes--reducing the model's sensitivity to deeply nested structures. For instance, in the case of the fourth-level heading \"1 Improper Scaffolding Setup\", the model may incorrectly associate it with the third-level heading \"3.1 Management Deficiencies\" rather than its immediate parent \"(1) Construction Management Issues\".  
Gated Semantic Fusion: Disabling the gated fusion module (SAFE w/o GateFuse) led to a 1.06% drop in Micro F1 and a 2.31% decline in P@5. This suggests that dynamically aggregating semantic signals from parent nodes, headings, and paragraph content improves the precision of key paragraph recognition. For example, under the heading \"4.2 Lack of Protective Measures\", the model without gating may overlook details in the body text such as \"failure to wear a safety harness\", relying too heavily on the general semantic of \"protective measures\" in the heading.  
SAFE-Transformer's performance improvements rely on the synergy of its individual modules. Among them, the structural parsing layer contributed the most (32.6%), followed by hierarchical attention (28.1%) and gated fusion (22.9%), with symbol-aware encoding accounting for 16.4%. These findings confirm that effective modeling of long texts in vertical domains requires the explicit representation of document structure, symbolic semantics, and hierarchical interactions.