# RTACompensator: Leveraging AraBERT and XGBoost for Automated Road Accident Compensation
## 4 Research Methodology
### Results  
In this section, we present the results of our model, designed to determine the part of responsibility attributed to road users involved in accidents. We evaluated our model on two augmented datasets: one augmented using Gemini, and the other using multilingual  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Hyperparameter** & **Range** & **Optimal Value** \\ \hline learning rate & 0.1–0.3 & 0.3 \\ \hline max depth & 3–10 & 6 \\ \hline min child weight & 1–10 & 1 \\ \hline gamma & 0.0–0.4 & 0.0 \\ \hline alpha & 0.0–0.4 & 0.2 \\ \hline colsample bytree & 0.3–1 & 1 \\ \hline colsample bylevel & 0.3–1 & 1 \\ \hline n estimators & 200–400 & 200 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Optimal hyperparameters for the part of responsibility model.  
BERT. To assess its performance, we compared our model with various alternatives, including XGBoost models using AraBERT embeddings with head and tail truncation strategies, XGBoost combined with Doc2 vec and Term Frequency-Inverse Document Frequency (TF-IDF) embeddings, Distilled BERT (DistilBERT), A Lite BERT (ALBERT), Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), Support Vector Machine (SVM), and LR. Tables 5 and 6 summarize the performance of our model compared with these alternatives on the two datasets, using the following metrics: precision (P), recall (R), f1-score (F), and accuracy (Acc). Figure 17 provides detailed category-level results for each corpus.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Models** & **P (\%)** & **R (\%)** & **F (\%)** & **Acc (\%)** \\ \hline Part of responsibility Model & **97.3** & **97.2** & **97.3** & **97.0** \\ \hline XGBoost+AraBERT (Head) & 95.5 & 95.7 & 95.6 & 95.4 \\ \hline XGBoost+AraBERT (Tail) & 95.8 & 96.0 & 95.9 & 95.7 \\ \hline XGBoost+Doc2 vec & 84.9 & 84.4 & 84.7 & 84.5 \\ \hline XGBoost+TF-IDF & 93.5 & 93.3 & 93.4 & 93.0 \\ \hline DistilBERT & 95.9 & 95.7 & 95.8 & 95.5 \\ \hline ALBERT & 96.9 & 96.8 & 96.9 & 96.6 \\ \hline CNN & 93.3 & 93.1 & 93.2 & 93.0 \\ \hline BiLSTM & 94.8 & 94.6 & 94.7 & 94.4 \\ \hline SVM & 88.9 & 88.5 & 88.7 & 88.5 \\ \hline LR & 89.9 & 89.7 & 89.8 & 89.6 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance of the part of responsibility model against other models on the augmented dataset with multilingual BERT.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Models** & **P (\%)** & **R (\%)** & **F (\%)** & **Acc (\%)** \\ \hline Part of responsibility Model & **97.7** & **97.6** & **97.7** & **97.3** \\ \hline XGBoost+AraBERT (Head) & 95.9 & 96.1 & 96.0 & 95.8 \\ \hline XGBoost+AraBERT (Tail) & 96.1 & 96.3 & 96.2 & 96.0 \\ \hline XGBoost+Doc2 vec & 85.3 & 84.8 & 85.1 & 84.8 \\ \hline XGBoost+TF-IDF & 93.9 & 93.6 & 93.8 & 93.1 \\ \hline DistilBERT & 96.4 & 96.2 & 96.3 & 96.1 \\ \hline ALBERT & 97.2 & 97.1 & 97.2 & 96.9 \\ \hline CNN & 93.7 & 93.4 & 93.6 & 93.3 \\ \hline BiLSTM & 95.2 & 95.1 & 95.2 & 94.9 \\ \hline SVM & 89.5 & 89.0 & 89.2 & 88.8 \\ \hline LR & 90.3 & 90.1 & 90.2 & 90.0 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance of the part of responsibility model against other models on the augmented dataset with Gemini.  
We also conducted an additional experiment using the Khaleej dataset, which consists of 6500 documents spanning the categories of sports, culture, medicine, technology, politics, finance, and religion. The dataset is publicly available at [[https://data.mendeley.com/datasets/572](https://data.mendeley.com/datasets/572) px667y9/1]([https://data.mendeley.com/datasets/572](https://data.mendeley.com/datasets/572) px667y9/1) (accessed on 6 January 2025). The primary objective of this experiment is to evaluate the generalization of our model on a different corpus. The performance of our classifier alongside other models is detailed in Table 7.