## 3 Methodology
### Data Augmentation  
In multi-class and multi-label classification, a balanced dataset has target classes that are evenly distributed. If one or more classes have overwhelmingly more samples than another (i.e., there is a skewness towards these classes), this dataset can be considered as an imbalanced dataset. One of the well-known methods for handling an imbalanced dataset is to perform _oversampling_ for the minority classes (classes with low samples) or _undersampling_ for the majority classes (classes with a large number of samples). Although either of these two approaches balances out the dataset, they do not directly tackle this problem, rather they may introduce new issues. Oversampling entails duplication of samples associated with the minority classes. Thus, it could lead to overfitting and slow down the training process. On the other hand, undersampling the majority classes removes a certain number of samples, which could lead to the model to be disadvantaged when learning important concepts that could be learned from the removed samples. In this work, we used the SenWave dataset [9] to train and evaluate the proposed models. To our knowledge, the SenWave dataset is the largest available fine-grained labeled dataset related to COVID-19. The dataset consists of 10,000 tweets available in 2 languages Arabic and English. In this work, we consider only the Arabic tweets. Each tweet in this dataset is labeled by 1 or more emotions, from a total of 11 emotions: _annoyed_, _anxious_, _denial_, _empathetic_, _joking_, _official_, _optimistic_, _pessimistic_, _sad_, _surprise_, and _thankful_. The main issue of SenWave dataset is that it is an imbalanced dataset, as shown in Figure 5. Each of the _anxious_, _denial_, _empathetic_, _optimistic_, and _sad_ classes (emotions) have few tweets compared with other classes. To address this issue, we proposed two  
Figure 4: Framework of the proposed BERT-based model. approaches. The first approach depends on creating new dataset, SenAIT, by merging the common classes of SemWave and AIT [63] datasets. The second approach is to use a dynamically weighted loss function (DWLF) that gives more weights for undersampled classes in the loss function during model training. * _SenAIT_: This dataset is created by merging the shared classes between SemWave and AIT datasets. Affect in Tweets (AIT) dataset was created as part of SemEval-2018 Task 1 [63] and it consists of 4380 Arabic tweets not related to COVID-19. Each tweet is labeled by 1 or more emotions, from a total of 11 emotions: _anger, anticipation, disgust, fear, joy, love, optimistic, pessimistic, sad, surprise_, and _trust_. SenWave and AIT datasets have four emotions as the common classes: _sad, surprise, optimistic_, and _pessimistic_. Thus, to enrich the SenWave dataset and for a comparative study, a new dataset, _SenAIT_, is created. This dataset contains all SenWave's tweets enriched by tweets from AIT dataset that are labeled by one or more of the four common classes. The resulting dataset contains 13,019 tweets with 11 classes and the distribution of these classes are shown in Figure 5.
* _Dynamically weighted loss function (DWLF)_: Another technique is proposed in this work to address the issue of imbalanced dataset through a weighted loss function. This technique involves having different weights for each class in the loss function based on the number of class's samples [64]. We assign a higher weight to the loss of the samples that belong to minor classes. For our multi-label classification task, we applied the proposed technique on the binary Cross-Entropy loss function that can be expressed mathematically as: \[L(x,y)=\{l_{1},\dots,l_{N}\}^{T}\] (1) \[l_{i}=-w_{i}[y_{i}.logx_{i}+(1-y_{i}).log(1-x_{i})]\] (2) where \(x_{i}\) is the input, \(y_{i}\) is the ground truth label, \(N\) is the batch size, and \(w_{i}\) is the sample weight that will optimize the contribution of the sample towards the overall loss. The sample weight is computed using the inverse of number of samples.