Question: What is the NileULex used by authors?
Evidence: Since offensive words typically have a negative polarity, we wanted to test the effectiveness of using a polarity lexicon in detecting offensive tweets. For the lexicon, we used NileULex BIBREF26, which is an Arabic polarity lexicon containing 3,279 MSA and 2,674 Egyptian terms, out of which 4,256 are negative and 1,697 are positive. We used the counts of terms with positive polarity and terms with negative polarity in tweets as features.
We experimented with various static embeddings that were pre-trained on different corpora with different vector dimensionality. We compared pre-trained embeddings to embeddings that were trained on our dataset. For pre-trained embeddings, we used: fastText Egyptian Arabic pre-trained embeddings BIBREF27 with vector dimensionality of 300; AraVec skip-gram embeddings BIBREF28, trained on 66.9M Arabic tweets with 100-dimensional vectors; and Mazajak skip-gram embeddings BIBREF29, trained on 250M Arabic tweets with 300-dimensional vectors.
Sentence embeddings were calculated by taking the mean of the embeddings of their tokens. The importance of testing a character level n-gram model like fastText lies in the agglutinative nature of the Arabic language. We trained a new fastText text classification model BIBREF30 on our dataset with vectors of 40 dimensions, 0.5 learning rate, 2$-$10 character n-grams as features, for 30 epochs. These hyper-parameters were tuned using a 5-fold cross-validated grid-search.
We also experimented with pre-trained contextualized embeddings with fine-tuning for down-stream tasks. Recently, deep contextualized language models such as BERT (Bidirectional Encoder Representations from Transformers) BIBREF31, UMLFIT BIBREF32, and OpenAI GPT BIBREF33, to name but a few, have achieved ground-breaking results in many NLP classification and language understanding tasks. In this paper, we used BERTbase-multilingual (hereafter as simply BERT) fine-tuning method to classify Arabic offensive language on Twitter as it eliminates the need of heavily engineered task-specific architectures. Although Robustly Optimized BERT (RoBERTa) embeddings perform better than (BERTlarge) on GLUE BIBREF34, RACE BIBREF35, and SQuAD BIBREF36 tasks, pre-trained multilingual RoBERTa models are not available. BERT is pre-trained on Wikipedia text from 104 languages and comes with hundreds of millions of parameters. It contains an encoder with 12 Transformer blocks, hidden size of 768, and 12 self-attention heads. Though the training data for the BERT embeddings don't match our genre, these embedding use BP sub-word segments. Following devlin-2019-bert, the classification consists of introducing a dense layer over the final hidden state $h$ corresponding to first token of the sequence, [CLS], adding a softmax activation on the top of BERT to predict the probability of the $l$ label:


where $W$ is the task-specific weight matrix. During fine-tuning, all BERT parameters together with $W$ are optimized end-to-end to maximize the log-probability of the correct labels.
For the lexicon, we used NileULex (El-Beltagy, 2016), which is an Arabic polarity lexicon containing 3,279 MSA and 2,674 Egyptian terms, out of which 4,256 are negative and 1,697 are positive.
Answer: It is an Arabic polarity lexicon containing 3,279 MSA and 2,674 Egyptian terms, out of which 4,256 are negative and 1,697 are positive.