## 3 Literature Review
### Safety  
Many transportation applications come with strong safety requirements. The ability of LLMs to handle massive and complex data efficiently has huge potential for risk assessment and accident prevention in order to identify potential hazards early and offer proactive suggestions to mitigate risks. Existing studies focus on safety applications concerning multiple modalities, mainly including aviation safety and road traffic safety; see Table 2 for a summary. \begin{table}
\begin{tabular}{l l} \hline \hline
**Reference** & **Focus** \\ \hline [PERSON] and [PERSON] [37] & Document classification based on the text in aviation safety reports \\ [PERSON] and [PERSON] [38] & Analyzing underreported crash factors based on traffic crash narratives \\ [PERSON] et al. [39] & Extracting safety concepts by aviation text-data mining \\ [PERSON] et al. [40] & Mining the key information from traffic accident texts \\ [PERSON] et al. [41] & Forecasting traffic accidents with the context of autonomous driving \\ [PERSON] et al. [42] & Identifying incident causal factors to improve aviation safety \\ [PERSON] et al. [43] & Applying multi-label classification on aviation safety reports \\ [PERSON] et al. [44] & Predicting anomalies based on the Aviation Safety Reporting System (ASRS) \\ [PERSON] et al. [45] & Comparing the performance of ChatGPT, BARD and GPT-4 in traffic crashes \\ [PERSON] et al. [46] & Analyzing ChatGPT in severity investigation of road crashes \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Studies about LLM applications for increasing transportation safety. Safety is of utmost importance in aviation, mainly due to the complex operational environment involving high-speed transportation, complex technological systems, and the important role of public perception when it comes to aviation-related accidents. Accordingly, it is crucial for aviation stakeholders to mitigate risks to prevent catastrophic events, adhering to strict regulatory standards. Existing studies mainly work with domain-specific text-based data from various accident reporting systems, e.g., targeting to enhance the accuracy of report classification and analysis. A custom aviation-domain-specific BERT model (Aviation-BERT) was implemented by [39] to perform text-mining tasks and was pre-trained based on text from the National Transportation Safety Board (NTSB) and Aviation Safety Reporting System (ASRS). Based on this aviation-specific BERT model, ref. [43] focused on multi-label classification problems and highlighted the significance of Aviation-BERT with superior performance on the ASRS data. In [50], accident report automation, traffic data augmentation, and multisensory safety analysis were performed through innovative LLM-based tools. Ref. [37] classified safety reports about weather and procedures by presenting a novel safety-informed SafeAeroBERT model. Serving as effective copilots or assistants in the aviation safety system, LLMs could identify safety issues and potential risks according to the historical data [42; 48]. Ref. [47] compared the performance of domain-specific experts and aeroBERT in the human factor issues about aviation safety and discussed the advantages of LLMs within the context of safety analysis. Analyzing three cases (Air France AF447, TransAsia Airways flight GE235, and Helios Airways Flight HCY552 accidents), ref. [51] showed that ChatGPT could not investigate the factors simultaneously from both high-level and low-level perspectives. Moreover, ref. [44] utilized a compact model to predict different types of anomalies (e.g., aircraft equipment problem, conflict, altitude deviation, and procedural deviation) from ASRS. The authors demonstrated that pre-training a compact transformer-based language model is beneficial for solving domain-specific tasks. Similar proposals have been made for other modalities, e.g., concerning road traffic safety. Ref.