# RTACompensator: Leveraging AraBERT and XGBoost for Automated Road Accident Compensation
## 4 Research Methodology
### Arabic Corpus of Responsibility
#### 4.1.2 Data Augmentation Process  
To achieve optimal performance during model training, it is essential to have a sufficient amount of training data, which underscores the importance of effective data augmentation. Some studies have used thesaurus-based substitution techniques, replacing words or phrases with synonyms based on their distribution in the text. Others have applied word similarity calculations using pretrained word embeddings, such as Word2 Vec [50], GloVe [51], and FastText [52], by replacing certain words with their nearest neighbors in the embedding space. In our case, we opted for more advanced, context-aware approaches to ensure high-quality augmented data. Specifically, we utilized a transformer model (multilingual BERT) and Gemini, an advanced AI model developed by Google DeepMind. Multilingual BERT was chosen for its ability to generate contextually consistent text variations using the masked language modeling (MLM) task. Unlike traditional augmentation techniques, such as thesaurus-based substitutions or static word embeddings, BERT leverages deep contextual understanding to predict masked tokens with high accuracy and coherence. This ensures that the augmented text maintains the style, meaning, and grammatical correctness of the original. For example, as shown in Figure 9, given the input sentence _\"Reckless driver causes traffic accident on AI-Nast [MASK]\"_, the model generates multiple predictions for the _[MASK]_ token, such as _\"Street\"_, _\"Alley\"_, and _\"Road\"_. The BERT-based data augmentation process further utilizes this capability to create diverse text variations. It begins by loading the multilingual BERT model (_bert_base_multilingual_cased_) and tokenizing the input text into smaller units (tokens), which are then converted into numerical IDs. A subset of these tokens is randomly replaced with the special _[MASK]_ token, according to a predefined masking probability. The modified sequence is passed through the BERT model, which predicts contextually appropriate replacements for the masked tokens. These predictions are used to reconstruct the sequence, resulting in an augmented version of the text. Repeating this process multiple times generates a diverse set of augmented texts. The step-by-step process for BERT-based text augmentation is outlined in Algorithm 1. Figure 8: The pipeline for the creation of the Arabic corpus of responsibility. Figure 9: An example of the data augmentation process using multilingual BERT. ```
0: Text \(text\), Number of augmentations \(num\_augmentations\), Mask probability \(mask\_probability\)
0: List of augmented texts \(augmented\_texts\)
1: Load BERT tokenizer with name \"bert-base-multilingual-cased\". 2: Load BERT model with name \"bert-base-multilingual-cased\". 3: Initialize an empty list \(augmented\_texts\). 4:for\(i=1\) to \(num\_augmentations\)do
5: Tokenize \(text\) into tokens and convert tokens to token IDs: \(token\_ids\). 6: Mask tokens:
7:for each token ID in \(token\_ids\)do
8: With probability \(mask\_probability\), replace the token ID with the [MASK] token ID. 9:endfor
10: Pass the masked \(token\_ids\) as input to the BERT model. 11: Obtain the predicted logits from the model's output. 12: Replace masked tokens:
13:for each token ID that was masked do
14: Replace the masked token ID with the token ID of the highest predicted probability. 15:endfor
16: Decode the modified \(token\_ids\) back to text: \(augmented\_text\). 17: Append \(augmented\_text\) to \(augmented\_texts\). 18:endfor
19:returnaugmented_texts
```  
**Algorithm 1** BERT-Based Text Augmentation  
Gemini, on the other hand, provides an even more sophisticated approach to data generation by learning patterns, structures, and relationships from input text. Its advanced DL techniques, including transformers, allow it to produce coherent and contextually appropriate text. Gemini was selected for its ability to generate complex human-like text that aligns with the original corpus in terms of semantics and structure.