# Structure-Aware and Format-Enhanced Transformer for Accident Report Modeling
## 6 Result  
### Overall Performance Analysis  
Table 4 presents the overall performance of each model on the multi-label accident intelligence classification task. The experimental results demonstrate that SAFE-Transformer achieved the best performance across all five metrics. Specifically, the micro-average AUC reached 73.45%, representing a 4.71 percentage point improvement over the traditional short-text model RoBERTa [65], and a 1.56 point gain over the best-performing long-text baseline, BigBird [31]. The simultaneous improvements in macro-average AUC and F1 (70.12% and 68.73%, respectively) indicate that SAFE-Transformer maintains superior performance on frequent labels while also significantly enhancing recognition of low-frequency labels. The micro-average F1 score reached 71.23%, improving upon Longformer [32] (66.52%) and BigBird [31] (70.18%) by 4.71 and 1.05 points, respectively--validating the effectiveness of the dual-stream encoding and dynamic gating mechanism in capturing global-local semantic interactions. Notably, SAFE-Transformer achieved the highest P@5 score of 76.33%, demonstrating its superior ability to accurately rank the top five predicted labels, which is critical for extracting actionable accident intelligence. Under comparable parameter scales and training configurations across all models, the consistent and cross-metric gains confirm that the structure-aware and format-enhanced strategies bring stable and substantial performance improvements in modeling long-form accident reports. In contrast to long-text optimized models, RoBERTa [65] adopts a bidirectional self-attention mechanism and processes long texts through truncation, which limits \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Micro-AUC** & **Macro-AUC** & **Micro-F1** & **Macro-F1** & **P@5** \\ \hline RoBERTa & 68.74 & 65.21 & 64.79 & 62.15 & 67.42 \\ Longformer & 69.35 & 66.83 & 66.52 & 64.78 & 71.15 \\ BigBird & 71.89 & 68.92 & 70.18 & 67.33 & 74.19 \\ ERNIE-SPARSE & 70.86 & 67.98 & 70.45 & 67.98 & 75.27 \\ SAFE-Transformer & 73.45 & 70.12 & 71.23 & 68.73 & 76.33 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Overall performance metrics. its capacity to capture long-range dependencies. As a result, it recorded the lowest macro-F1 (62.15%) and micro-F1 (64.79%) scores among all baselines, highlighting the limitations of general-purpose pretrained models in handling domain-specific structured texts. Among the long-text models, the performance differences among various sparse attention implementations were also notable, reflecting distinct sensitivities to task-specific indicators. BigBird [31] combines global attention (targeting special tokens like [CLS]), sliding window attention (capturing local context), and random attention (facilitating token interaction via random sampling). It outperformed other baselines in micro-AUC (71.89%) and macro-AUC (68.92%), indicating that global block sparse attention is well-suited for modeling long-range dependencies. ERNIE-SPARSE [30], utilizing Hierarchical Sparse Transformers and Self-Attention Regularization (SAR), excelled in micro-F1, macro-F1, and P@5, highlighting its strong sensitivity to local semantics. In comparison, SAFE-Transformer's proposed Struct-Driven Hierarchical Sparse Attention showed a unique advantage across metrics: by enforcing attention constraints via dynamic hierarchical masking, it significantly outperformed other sparse attention schemes in both P@5 (76.33%) and macro-F1 (68.82%). This indicates its superior ability to balance global dependency modeling with local semantic precision. Further analysis revealed that SAFE-Transformer's symbol-sensitive sparse attention strategy--focusing on structural cues such as \"Article X\" or list markers like \"(a)\"--led to a 4.2% increase in macro-F1 for low-frequency labels (e.g., L6: legal clauses), validating the critical role of structured domain-aware sparsity in handling long-tail label distributions.