# Enhancing Imbalanced Sentiment Analysis: A GPT-3-Based Sentence-by-Sentence Generation Approach
## 4 Experimental Details
### Sentiment Classification  
Three important stages are involved in this part of our study: preprocessing, sentiment modeling, and evaluation. Stage 1: Preprocessing  
Preprocessing at this stage is a preparation for the sentiment classification phase, thereby continuing the preprocessing that was conducted before and after the synthetic data generation process. This process encompasses several key steps. The first step is text normalization, which involves lowercasing all text for consistency and removing any special characters, numbers, and excessive spaces to streamline the content. Next is content reduction, which entails eliminating elements such as hashtags, URLs, and user handles that do not contribute to the core analysis, along with removing overly short words, specifically those just one character in length. The final step is noise removal, focusing on extracting and discarding stop words to reduce redundancy and erasing emojis, which often add unnecessary complexity to the text analysis. Together, these steps effectively refine the data, setting a strong foundation for accurate sentiment analysis. Stage 2: Sentiment Classification  
In the sentiment classification stage, we address both the original (imbalanced) data and the balanced dataset augmented with good-quality synthetic data. The methodologies for splitting the data and the sentiment classification process are still the same as the approaches used in our previous studies [11]. This involves dividing the dataset into training and testing subsets at an 80:20 ratio. The training data are further split into training and validation sets, also adhering to an 80:20 ratio. For both the original and balanced data, after modeling using the training and validation datasets, the models are then tested on the same testing dataset, which retains the imbalanced proportion characteristic of the original dataset. For sentiment modeling, five deep-learning models are utilized, namely, RNN, CNN, LSTM, BiLSTM, and GRU, which incorporate GloVe embeddings. The detailed architectures of the models are presented in Table 2. The fully connected layer configuration across all models is consistent, featuring 256 units with ReLU activation and a dropout of 0.5. This design choice aims to provide a robust learning capability while mitigating the risk of overfitting. The output layer in each model is a dense layer with three units employing a softmax activation function, which is well suited for multiclass classification tasks like sentiment analysis. For hyperparameters, all models share the same settings: the Adam optimizer is used with a learning rate of 0.0005, and each model is trained for 50 epochs with a batch size of 32. This uniformity in hyperparameters ensures a level playing field for comparing the models' performance. The primary distinction among the models lies in their main layer structures:  
* RNN uses two layers of SimpleRNN with 256 units each, implementing a dropout rate of 0.5. * CNN consists of three Conv1D layers with 128, 128, and 256 filters, each followed by a dropout of 0.5. * LSTM features two LSTM layers, each with 256 units, and a dropout rate of 0.5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Main Layer.** & **Fully Connected Layer.** & \multirow{2}{*}{**Output Layer**} & **Hyperparameters: Optimizer,** \\  & **Unit/Filter, Dropout** & & **Unit, Activation, Dropout** & **Learning Rate, Batch Size** \\ \hline RNN & SimpleRNN: 256 (2 layers), 0.5 & 256, ReLU, 0.5 & Dense(3, softmax) & Adam, 0.0005, 32 \\ CNN & Conv1D: 128, 128, 256, 0.5 & 256, ReLU, 0.5 & Dense(3, softmax) & Adam, 0.0005, 32 \\ LSTM & LSTM: 256 (2 layers), 0.5 & 256, ReLU, 0.5 & Dense(3, softmax) & Adam, 0.0005, 32 \\ BiLSTM & BiLSTM: 256 (2 layers), 0.5 & 256, ReLU, 0.5 & Dense(3, softmax) & Adam, 0.0005, 32 \\ GRU & GRU: 256 (2 layers), 0.5 & 256, ReLU, 0.5 & Dense(3, softmax) & Adam, 0.0005, 32 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The architectures of the deep-learning models used for sentiment classification.