C onfli BERT - A rabic: A Pre-trained A rabic Language Model for Politics, Conflicts and Violence (2023)

Abstract: This study investigates the use of Natural Language Processing (NLP) methods to analyze politics, conflicts and violence in the Middle East using domain-specific pre-trained language models. We introduce Arabic text and present ConfliBERT-Arabic, a pre-trained language models that can efficiently analyze political, conflict and violence-related texts. Our technique hones a pre-trained model using a corpus of Arabic texts about regional politics and conflicts. Performance of our models is compared to baseline BERT models. Our findings show that the performance of NLP models for Middle Eastern politics and conflict analysis are enhanced by the use of domain-specific pre-trained local language models. This study offers political and conflict analysts, including policymakers, scholars, and practitioners new approaches and tools for deciphering the intricate dynamics of local politics and conflicts directly in Arabic.

## introduction in the middle east, political upheaval and carnage have long been issues @xcite . deep divisions, geopolitical rivalries, and foreign meddling have historically riven the area, from the israeli-palestinian conflict to the ongoing civil war in syria. even if the root causes of these conflicts are complex and multidimensional, the role that language and communication play in shaping the narratives that underlay them cannot be ignored. language is commonly used as a strategy to rally support, defend violence, and discredit opposing viewpoints. therefore, it is essential to develop effective methods for understanding and analyzing the role that language and texts plays in middle eastern politics and conflicts via news reports and other sources. natural language processing (nlp) approaches can evaluate large amounts of text and have shown great promise in identifying patterns and insights that would otherwise be difficult to spot. recent, pre-trained language models (plm), like bert @xcite , have improved in efficiency for a range of nlp tasks, including sentiment analysis, text categorization, and language synthesis. plms have received a lot of attention in the literature, but most of it has focused on english or other widely spoken languages; very few studies have examined how well they apply to arabic. the arabic language has a rich and complicated morphology, which has increased the requirement for highly advanced nlp tools that can meet the language's expanding needs across a variety of fields and applications @xcite . this research fills this vacuum in the literature by investigating the application of arabic-specific plms for politics, conflicts and violence in the middle east. we reference a variety of pertinent academic works, such as investigations into the nature of political violence @xcite , the function of language in conflicts @xcite , and the creation and use of plms @xcite such as conflib-ert @xcite . the performance of two plms, bert and conflibert-arabic, focuses on the analysis of arabic text about politics, conflict, and violence in the middle east. bert is a more general-purpose plm that has been used to tackle a variety of nlp problems, whereas conflibert-arabic is a domain-specific plm optimized on a corpus gathering texts relevant to regional politics, conflicts and violence. we contrast their effectiveness with a different plm, @xcite . this work has implications for multiple users such as policymakers, researchers, and conflict analysts. by providing cutting-edge tools and methods for investigating politics and conflicts in the middle east, our study develops data for more effective conflict prevention and resolution programs. by examining the role that language and communication play in affecting the politics and conflicts in the region, we can provide a more nuanced understanding and prediction of the underlying causes of these conflicts and cooperation in the middle east. our experiments show that domain-specific pretraining significantly improves model performance, particularly for identifying information about political conflict. we examine in detail each model's applications and their benefits and drawbacks. ## related work recent developments in pre-trained language models have significantly advanced the field of natural language processing. here, we review three of the most prominent models: bert, multilingual bert, arabert, and conflibert. ## approach to develop conflibert-arabic, we implemented a series of steps, namely corpora collection, pretraining strategies, and evaluation tasks. the first step involves the creation of a domain-specific corpus for pre-training. publicly available arabic datasets focusing on politics and conflict domains are limited, and thus we conducted our own data collecting to extract political text from arabic sources, thus enabling us to achieve better results on political tasks. after building the corpus, we developed our domain-specific model based on bert, a powerful language model that has been successfully validated in multiple domains in both english and arabic languages @xcite @xcite @xcite @xcite . our masked-language modeling (mlm)-based bert model shows improved performance compared to other transformer models that use different self-supervision tasks. the final step involves evaluating the performance of conflibert-arabic on downstream tasks related to political and conflict analysis to measure its effectiveness in real-world applications. ## results and analysis ## conclusion and future work in this paper, we introduce conflibert-arabic, a pre-trained language model for politics, conflict and violence in arabic-language. conflibert-arabic's development required the acquisition and curation of a sizable domain-specific corpus for the pre-training stage. we also thoroughly assessed the model's performance across a range of nlp tasks and datasets, showing that conflibert-arabic regularly outperforms bert in the politics, conflict and violence domain, especially when working with sparse training data. researchers and decisionmakers interested in tracking, analyzing, and predicting political violence and war in the middle east will find these findings to be of great interest. conflibert-arabic is an important advancement that will help a large community of political scientists and decision-makers as a whole. in future work, we are planning to expand on parameters such as vocabulary size and epochs to better optimize conflibert-arabic. additionally, applying conflibert-arabic to more challenging tasks such as understanding, inference, question answering, and uncertainty qualification is planned