# Structure-Aware and Format-Enhanced Transformer for Accident Report Modeling
## 1 Introduction  
Accident investigation reports serve as critical vehicles for analyzing accident causation mechanisms, understanding risk evolution processes, and formulating effective prevention strategies. Systematic analyses of these reports significantly contribute to the identification of safety risks and optimization of accident prevention management systems [1, 2]. Traditional methods for modeling accident reports primarily rely on rule-based templates or classic machine learning algorithms, such as decision trees and support vector machines, for shallow feature extraction [3, 4, 5, 6, 7, 8]. However, these methods are constrained by high modeling costs and limited semantic understanding capabilities. With advancements in Transformer architectures [9], deep learning approaches utilizing pretrained language models like BERT [10] have become mainstream in accident text analysis, demonstrating substantial advantages in tasks such as accident classification and risk entity recognition [11; 12; 13; 14; 15]. Nevertheless, existing studies largely concentrate on sentence-level or short-text scenarios [16; 17; 18], leaving effective solutions for more realistic document-level modeling relatively unexplored. Document-level analysis enables context-aware identification of semantic associations across paragraphs and comprehensive causal chains. Such an approach precisely identifies dispersed accident risks, incident causes, contributing factors, and their interrelationships across various document sections [15; 19]. Utilizing global context, this method extracts detailed accident intelligence--including event sequences, human factors, technical elements, environmental influences, and management factors [20; 21]. Consequently, this analysis assists safety management authorities in building complete chains of risks, events, and evidence, uncovering hidden patterns from extensive historical accident cases. It provides a scientific basis for explaining accident causation mechanisms, studying risk evolution processes, and formulating targeted accident prevention strategies. Accident investigation reports, typically drafted by domain experts based on on-site inspections and technical analyses, are characterized by extensive length, complex structure, and dense information. While the compilation of these reports strictly adheres to the \"four-part\" logical framework established by the \"Regulations on Production Safety Accident Reporting and Investigation Handling\" (State Council Order No. 493)--specifically, (1) accident occurrence and emergency response, (2) direct/indirect cause analysis, (3) responsibility attribution, and (4) recommended corrective measures--in practice, significant variations exist among different industries, regions, and accident types. Nonetheless, most reports consistently include sections on accident profile, event sequences, emergency response, causal analyses, responsibility attribution, and preventive measures. Similar frameworks exist in many other countries. For instance, the U.S. Occupational Safety and Health Administration (OSHA) has a set of guidelines for reporting workplace accidents, which, while differing in structure, also focus on key aspects such as causation analysis, responsible parties, and corrective actions [22]. Furthermore, the European Statistics on Accidents at Work (ESAW) methodology (Regulation n.349/2011) guides accident data practices in Europe, providing a standardized framework for occupational accident reporting [23]. Public datasets under this framework typically use alpha-numeric coding to protect privacy. In addition, the international standard ISO 45001 has significantly improved the utilization of accident datasets within occupational health and safety management systems globally, emphasizing systematic data-driven safety improvement and risk management [24; 25]. These international practices underscore the broader applicability and relevance of accident data modeling research. Modeling accident investigation reports faces three primary challenges.