## 5 Conclusions and Future Scope  
The proposed ARABERT4 TWC model in this paper is applied for classifying the Arabic tweet and achieved high results for three datasets, which dominate other well-known tweet or text classification methods. The Arabic BERT architecture is built by fine-tuning the pre-trained BERT case model provided by the Hugging Face Transformer library and integrating custom layers and a classification layer on top of the encoder structure. Tokens were extracted from the full-text corpus using a word piece tokenizer based on a sub-word tokenization method. The proposed model can learn context-aware information and learn the context information bi-directionally. In order to complement the previously done work, this study first conducts experimental analysis across five datasets to investigate fine-tuning \begin{table}
\begin{tabular}{c c c} \hline \hline \begin{tabular}{c} **C Criteria** \\ \end{tabular} & \begin{tabular}{c} **hULMonA** \\ \end{tabular} & \begin{tabular}{c} **ARABERT4 TWC** \\ \end{tabular} \\ \hline \begin{tabular}{c} Training data \\ \end{tabular} & \begin{tabular}{c} Less training data \\ \end{tabular} & \begin{tabular}{c} More training data \\ \end{tabular} \\ \hline \begin{tabular}{c} Model architecture \\ \end{tabular} & \begin{tabular}{c} Transfer learning on large \\ corpus of Arabic data \\ \end{tabular} & \begin{tabular}{c} Additional layer to capture \\ unique characteristics of Arabic \\ text \\ \end{tabular} \\ \hline \begin{tabular}{c} Task-specific fine-tuning \\ \end{tabular} & \begin{tabular}{c} Fine tuning of general domain \\ target task of data. \\ \end{tabular} & \
\begin{tabular}{c} Provide task specific tuning of \\ data to achieve better \\ performance on social media \\ tasks. \\ \end{tabular} \\ \hline
\begin{tabular}{c} Model size \\ \end{tabular} & 140-million parameters & 300-million parameters \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance evaluation of proposed ARABERT4 TWC. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \begin{tabular}{c} **Models** \\ \end{tabular} & \begin{tabular}{c} **SANAD** \\ \end{tabular} & \begin{tabular}{c} **HARD** \\ \end{tabular} & \begin{tabular}{c} **AJGT** \\ \end{tabular} & \begin{tabular}{c} **ASTD** \\ \end{tabular} & \begin{tabular}{c} **ArSenTD-LEV** \\ \end{tabular} \\ \hline \begin{tabular}{c} AraBERTv0.1 [28] \\ \end{tabular} & - & 0.9620 & 0.9310 & 0.9220 & 0.5356 \\ \hline \begin{tabular}{c} AraBERTv1 [28] \\ \end{tabular} & - & 0.9610 & 0.9380 & 0.9260 & - \\ \hline \begin{tabular}{c} AraBERTv0.2-base [28] \\ \end{tabular} & - & - & - & - & 0.5571 \\ \hline \
\begin{tabular}{c} AraBERTv0.2-large [28] \\ \end{tabular} & - & - & - & - & 0.5694 \\ \hline hULMonA [41] & - & 0.9570 & - & 0.7710 & 0.5240 \\ \hline CAMeLBERT [42] & - & - & - & 0.7690 & - \\ \hline
\begin{tabular}{c} ARABERT4 TWC \\ \end{tabular} & 0.9824 & 0.9749 & 0.9405 & 0.8610 & 0.5370 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance accuracy for the applied dataset with different models. strategy in terms of tokenized vector and learning rate. The results of the experiments demonstrate that for three datasets, our model ARABERT4 TWC not only outperformed deep learning models like CNN, LSTM, and BERT, but also conventional machine-learning techniques. Future research directions include expanding our understanding of effectively combining the domain and cross-domain pre-training with BERT to include domain and task-specific information. We would further apply knowledge distillation on the BERT model to make it suitable for edge devices. Data curation, M.A. ; formal analysis, A.M.M. and M.E. ; investigation, A.M.F., M.A and A.M.M. ; supervision, M.A. ; writing--original draft, A.M.F. and A.M.M. ; writing--review and editing, A.M.M., M.E. and M.A. All authors have read and agreed to the published version of the manuscript. This research was funded by the Deanship of Scientific Research--Jouf University, grant number DSR-2021-02-0358. Not applicable. **Data Availability Statement:** Furnished on request.