# TAWC: Text Augmentation with Word Contributions for Imbalance Aspect-Based Sentiment Classification
## 3 Word Selection for Augmentation
### Word Contributions Extraction  
In a sentiment classification task, the model is trained to assign one label from \(C=\{c_{1},c_{2},...,c_{k}\}\) to a sentence. Here, the training set is \(D\), and its lexicon is \(L\). For words \(w_{i}\in L\) and a class \(c_{j}\in C\), we can determine their relationship from two perspectives:  
* Analytical Correlation. This metric assesses how frequently word \(w_{i}\) co-occurs with class \(c_{j}\) but not other words in the training dataset;
* Semantic Similarity. This measures a word's semantic similarity with class \(c_{j}\). Thus, each metric's impact on word selection for augmentation can be determined. Analytical correlation can guide the preservation of important relationships in data. If two words are highly correlated, we want to ensure that substitutions do not break the relationship. In addition, semantic similarity ensures that word replacements are meaningful and retain the original intent of the given text. When both analytical correlation and semantic similarity are used together, the augmentation process becomes more refined. For example, when augmenting sentiment analysis data, we used correlation to ensure that  
Figure 3: Data preprocessing steps: (1) noise removal and (2) word correction to clean text. key sentiment-bearing words like _unfriendly_ and _bad_ are kept in sync. Furthermore, we used semantic similarity to replace _appetizer_ with _pasta_ (semantically similar) rather than an unrelated word like _table._  
According to the two measurements, all words in the training dataset can be categorized into four groups:  
1. **Meaningful** words: Words in this group are useful class-indicating words with high correlation and high semantic similarity with the corresponding label. 2. **Necessity** words. Words usually co-occur frequently with classes but have low semantic similarity. These words can provide extra information about the class. However, their low semantic similarity may cause noise. 3. **Reward** words: These words have low correlation but high semantic similarity, which is useful for model generalization. 4. **Irrelevant** words: Words with less contribution to sentiment classification because they have low correlation and low semantic similarity. We had to use appropriate metrics to recognize the contribution of words in the sentiment classification task. To measure analytical correlation with the label, we applied pointwise mutual information (PMI) to extract label-correlated words from the sentence. PMI in sentiment analysis helps to identify words that are strongly correlated with specific sentiment labels by calculating how much more likely they are to appear in relation to certain sentiments than by random chance. It efficiently extracts sentiment-related features from text data. This was inspired by [29], which applied PMI to build a system to automatically create a sentiment lexicon. [PERSON] and [PERSON] [30] used PMI to define an essential sentence based on the semantic similarity between a sentence and a document for the summarization task. The PMI score is computed as follows:  
\[PMI(w,c)=log\frac{P(w,c)}{P(w)P(c)} \tag{1}\]  
where \(w\) is a word, \(c\) is a class or label of a sentence, \(P(w,c)\) is the probability of a word and class co-occurring in the corpus, \(P(w)\) is the probability of a word occurring in the corpus, and \(P(c)\) is the probability of a class occurring in the corpus. We use the frequency of a word occurring in a certain class to estimate the probability of a word. Take the following example of a sentence with negative sentiment: _The product quality was disappointing and the customer service was horrible._ Words like _disappointing_ and _horrible_ are often associated with negative sentiment in a larger dataset. The PMI computes how frequently each word, such as _disappointing_ or _horrible_, co-occurs with negative sentiment compared to how often they appear independently.