# Affect Analysis in Arabic Text: Further Pre-Training Language Models for Sentiment and Emotion
## 3 Methodology
### Task-Adaptation Approach
#### 3.1.2 Pre-Training Datasets Pre-Processing  
The datasets collected in the previous sections were already pre-processed by their authors. Therefore, we performed light pre-processing to prepare the datasets for the further pre-training task. Tables 1-4 show the statistics of the STD, SRD, STRD, and ETD datasets. The following pre-processing steps were performed:  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Dataset-Name** & **Size** & **\#Classes** & **\#Pos** & **\#Negg** & **\#Other Classes** & \(\mathbf{\
eq}\)**\#Tweets** & **Dup** & **Size-after-Pre Processing** & **Avg SeqLen** \\ \hline
**AJGT**[55] & 1800 & 2 & 813 & 727 & - & 260 & - & 1540 & 10 \\
**ArSAS**[56] & 19,897 & 4 & 4323 & 7325 & 8113 & 79 & 57 & 11,648 & 23 \\
**ArSENT-D-LEV**[57] & 4000 & 5 & 895 & 1292 & 885 & 0 & 928 & 2187 & 23 \\
**Arabic-Dialet**[58] & 52,210 & 3 & 5546 & 15,086 & 30,033 & 1118 & 427 & 20,632 & 21 \\
**BBN-Dataset**[59] & 1200 & 3 & 481 & 558 & 127 & 34 & - & 1039 & 10 \\
**Syrian-Dataset**[59] & 2000 & 3 & 447 & 1349 & 202 & 0 & 2 & 1796 & 19 \\
**ATSAD**[60] & 56,795 & 2 & 16,106 & 16,476 & - & 3980 & 20,233 & 32,582 & 13 \\
**ASTD**[61] & 9693 & 4 & 736 & 1592 & 7274 & 91 & - & 2328 & 17 \\
**ArSarcasan**[63] & 10,547 & 3 & 1634 & 3473 & 5340 & 71 & 29 & 5107 & 17 \\
**SemEval2017**[62] & 3352 & 3 & 728 & 1108 & 1470 & 6 & 40 & 1836 & 17 \\
**TEAD**[64] & 555,923 & 2 & 391,530 & 162,763 & - & 673 & 957 & 554,293 & 13 \\
**Affect in Tweets**[39] & 1800 & 7 & 720 & 808 & 262 & 10 & - & 1528 & 17 \\
**MARSA**[65] & 56,782 & 3 & 16,647 & 19,858 & 18,726 & 384 & 1167 & 36,505 & 15 \\
**AraSenti-Tweet**[66] & 17,573 & 4 & 4789 & 5940 & 6461 & 367 & 13 & 10,729 & 16 \\
**Total (STD)** & 683,750 & 2 & 445,395 & 238,355 & 78,893 & 7073 & 25,406 & 682,197 & 13 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Sentiment Tweets Dataset (STD) Statistics.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset-Name** & **Size** & **\#Positive** & **\#Negative** & **\#Neutral** & **\#Tweets \(\mathbf{\geq}\) 3** & **Dup** & **Size-after-Preprocessing** & **Avg Len of Seq** \\ \hline
**OCLAR**[67] & 3916 & 1595 & 275 & 418 & 1587 & 41 & 1870 & 13 \\
**LABR**[68] & 63,257 & 39,069 & 7568 & 12,201 & 2328 & 2091 & 46,637 & 66 \\
**HARD**[69] & 409,562 & 263,453 & 51,864 & 80,326 & 8570 & 5349 & 315,317 & 22 \\
**BRAD**[70] & 510,599 & 312,583 & 75,066 & 106,785 & 14,429 & 1736 & 387,649 & 80 \\
**Total (SRD)** & 751,473 & 616,700 & 134,773 & 199,730 & 26,914 & 9217 & 751,473 & 54 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Sentiment Reviews Dataset (SRD) Statistics.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset-Name** & **Emotional-Tone**[71] & **LAMA-DINA**[72] & **Affect in Tweets**[39] & **AraEmoTw**[73] & **SemEval-2018** & **ATT DISC**[39] & **Total (ETD)** \\ \hline
**Size** & 10,065 & 8502 & 5600 & 226,774 & 1,019,435 & 1,262,210 \\
**\#Ioy** & 1169 & 1265 & 1389 & 38,591 & 185,104 & 227,518 \\
**\#Sad** & 1222 & 964 & 792 & 23,871 & 319,847 & 366,996 \\
**\#Fear** & 1140 & 1376 & 797 & 44,036 & 29* The URLs, user mentions, and hashtags present in any of the collected sentences were replaced with the tokens [1],[1],[1] and [3];
* Sentences with three words or fewer were removed;
* Null and duplicated were eliminated.