# TAWC: Text Augmentation with Word Contributions for Imbalance Aspect-Based Sentiment Classification
## 3 Word Selection for Augmentation
### Word Contributions Extraction
For example, if _horrible_ is commonly used in negative reviews, the PMI score between _horrible_ and the negative class would be high. High PMI scores indicate words strongly associated with the label. We utilized the word embedding Word2 vec based on the skip-gram model [31] to measure the semantic similarity between a word and the meaning of the label polarity. Due to the high inference costs of transformer-based models, such as BERT, we did not use them for similarity measurements. Some studies have also demonstrated that static word embeddings can outperform BERT-like models in context-free similarity measurement tasks, particularly at the word level [32]. We computed the cosine similarity to determine the semantic distance between a word and a class. The cosine similarity evaluates the semantic similarity between words and sentiment labels by calculating the angle between their vector representations. It is particularly useful in sentiment analysis when using word embeddings because it captures the strength of the association between words and different sentiment categories. A high cosine similarity indicates that the word is closely aligned with a particular sentiment, which makes it a useful measure for identifying sentiment words. Given two vectors of attributes, \(w\) and \(c\), the cosine similarity is represented by the dot product and magnitude as \[similarity(w,c)=cos(\theta)=\frac{v_{w}.v_{c}}{\parallel v_{w}\parallel\parallel v_{c} \parallel}. \tag{2}\]  
Taking the illustration into account, we calculated the cosine similarity between a word _horrible_ and the negative sentiment. Here, we identified the vectors of _horrible_ = [0.9, 0.8, 0.7] and negative sentiment = [1.0, 0.9, 0.8] from Word2 vec embedding. Calculating the dot product \(v_{w}\cdot v_{c}\) = (0.9 \(\times\) 1.0) + (0.8 \(\times\) 0.9) + (0.7 \(\times\) 0.8) = 0.9 + 0.72 + 0.56 = 2.18, the score magnitudes of _horrible_ and negative classes were 1.39 and 1.57, respectively. The score of cosine similarity was calculated using Equation (2) to reach 1.0, indicating perfect alignment (strong association with negative sentiment). We computed each word's correlation and similarity scores and defined a threshold to divide high and low degrees. Let words with high (low) correlation scores be \(C_{h}\) (\(C_{l}\)) and words with low (high) similarity scores be \(S_{l}\) (\(S_{h}\)). We chose the first (third) quartile as the low (high) degree threshold, as it provided better results than the median. In our initial experiment, we used the second quartile (median) as the threshold. However, we observed that the word extraction results did not adequately represent their role in the sentence in relation to the label. Upon further experimentation, we discovered that the first (or third) quartile yielded better performance during our second attempt, and we opted not to fine-tune the thresholds further. Nevertheless, adjusting these hyperparameters may still lead to additional improvements in the proposed method. In order to reduce the variance, we transformed the correlation and similarity scores using the min-max normalization technique, as shown in Equation (3) and Equation (4), respectively. In addition, we extracted the words into four categories according to the rules presented in Table 1.  
\[normalized(C)=\frac{C-min(C)}{max(C)-min(C)} \tag{3}\]  
\[normalized(S)=\frac{S-min(S)}{max(S)-min(S)} \tag{4}\]  
The pseudocode for word contribution extraction is given in Algorithm 1. We implemented the word contribution procedure in the SemEval 2016 dataset of negative and neutral labels, and some examples are presented in Table 2.