## 6 Conclusions and Future Work  
This study significantly contributes to crash severity classification using ensemble learning with pre-trained transformers and traditional machine learning models. We evaluated the effectiveness of various word embedding techniques (TF-IDF, Word2 Vec, and BERT) combined with machine learning classifiers (XGBoost, random forest, and naive Bayes) and advanced models (BERT and RoBERTa fine-tuned with Bi-LSTM) through two comprehensive experiments.  
The findings demonstrate that data augmentation and ensemble learning notably improve model performance, particularly in addressing class imbalance issues. In the first experiment on the original dataset, the combination of TF-IDF with XGBoost achieved high F1 scores for \"Personal Injury\" (99%) but struggled with the \"Property Damage\" class. Similarly, BERT-LSTM performed well for \"Fatal\" (98%) but poorly for \"Property Damage\" (2%). The ensemble model, combining TF-IDF/XGB, BERT-LSTM, and RoBERTa-LSTM, improved the overall performance with an F1 score of 99% for \"Fatal\" and 100% for \"Personal Injury\" but still failed to classify \"Property Damage\" effectively.  
In the second experiment on the augmented dataset, the ensemble model significantly improved classification performance across all classes. It achieved F1 scores of 99% for \"Fatal\", 99% for \"Personal Injury\", and 62% for \"Property Damage\", showing substantial improvement for the minority class compared to individual models. For instance, theXGB-TFIDF model achieved F1 scores of 98% for \"Fatal\", 98% for \"Personal Injury\", and 51% for \"Property Damage\".  
Despite these improvements, the classification performance for the \"Property Damage\" class still needs further enhancement, highlighting the necessity for additional research. Future work will focus on expanding the dataset to include traffic data from various sources to further evaluate model robustness. Additionally, we aim to integrate hybrid features from both text narratives and tabular data and apply explainable A.I. techniques to interpret model results and identify key factors contributing to crash severity. Further research will also explore new data augmentation methods and advanced modeling approaches to improve performance for minority classes. Moreover, we plan to refine our approach by considering the potential benefits of employing a weighted voting system, as well as other ensembling techniques, to further enhance the robustness and accuracy of our classification models.  
Conceptualization, S.J. and M.E.; software, S.J.; formal analysis, S.J. and M.E.; investigation, S.J., and M.E.; data curation, A.F.; writing--original draft preparation, S.J.; writing--review and editing, S.J., M.E., and R.N.; visualization, S.J. and M.E.; supervision, M.E., R.N., and A.P. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
The original contributions presented in the study are included in the article, further inquiries or requests can be directed to the corresponding author.  
I would like to thank the Queensland University of Technology (Q.U.T.) for funding this research. This work is conducted as part of my PhD research, supported by a funded scholarship from QUT.  
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.