# Interpretability in Sentiment Analysis: A Self-Supervised Approach to Sentiment Cue Extraction
## 5 Experiments
### Results and Analysis
#### 5.4.3 Model Generalization Tests  
To ascertain the adaptability and generalizability of our proposed method, we conduct cross-testing on three English datasets. Specifically, this involves using the model trained on each dataset to test the other two datasets. Additionally, we combine the datasets generated by the SS-SCE method from all three datasets to train a single sentiment cue extraction model, which is then tested on all three datasets. Additionally, we merge the datasets sampled from the three English datasets to train collectively and conduct tests on each dataset individually. For the amalgamated dataset, we use the term \"combined\" to denote it. In the cross-testing, we continue to use the same evaluation metrics as those presented in Table 3. It is noted that we use subscripts to denote the training dataset of the sentiment \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**SST-2**} & \multicolumn{3}{c}{**Yelp**} & \multicolumn{3}{c}{**IMdb**} & \multicolumn{3}{c}{**ChnSentiCorp**} \\ \cline{2-13}
**Metric** & \(X\) & \(X^{Y}\) & **Loss** & \(X\) & \(X^{Y}\) & **Loss** & \(X\) & \(X^{Y}\) & **Loss** & \(X\) & \(X^{Y}\) & **Loss** \\ \hline Accuracy & 0.9300 & 0.8719 & 0.0585 & 0.9885 & 0.9748 & 0.0138 & 0.9328 & 0.8798 & 0.0531 & 0.9369 & 0.8367 & 0.1002 \\ Precision & 0.9379 & 0.9072 & 0.0307 & 0.9876 & 0.9723 & 0.0153 & 0.9305 & 0.8333 & 0.0971 & 0.9387 & 0.7940 & 0.1448 \\ Recall & 0.9182 & 0.8224 & 0.0958 & 0.9895 & 0.9776 & 0.0120 & 0.9359 & 0.9501 & \(-\)0.014 & 0.9372 & 0.9174 & 0.0198 \\ F1 & 0.9280 & 0.8627 & 0.0652 & 0.9886 & 0.9749 & 0.0136 & 0.9332 & 0.8879 & 0.0453 & 0.9380 & 0.8512 & 0.0867 \\ RCT & - & 0.1682 & - & - & 0.3795 & - & - & 0.2858 & - & - & 0.3148 & - \\ \hline \(p_{t\text{-test}}\) & \multicolumn{3}{c}{\textgreater{}0.01} & \multicolumn{3}{c}{\textgreater{}0.01} & \multicolumn{3}{c}{\textless{}0.01} & \multicolumn{3}{c}{\textless{}0.01} & \multicolumn{3}{c}{\textless{}0.01} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: This table displays accuracy, precision, recall, F1 scores, and performance loss for original (\(X\)) versus cue-extracted (\(X^{Y}\)) inputs across SST-2, Yelp, IMDb, and ChnSentiCorp datasets. RCT values and \(t\)-test results are also included to assess the extraction’s effectiveness. extraction model. For example, \(X_{SST-2}^{Y}\) represents the \(X^{Y}\) generated by the sentiment cue extraction model trained on the SST-2 dataset. As shown in Table 4, when models trained on Yelp and IMDb datasets are tested on SST-2, they show a notable performance decline, particularly in accuracy and recall. The most pronounced drop is observed in the model trained on IMDb, with a 16.97% decrease in accuracy. This can be attributed to the disparity in text length and complexity between IMDb and SST-2 datasets. Although the precision of the IMDb-trained model remained relatively stable, indicating a consistent ability to identify true positives, the substantial decrease in recall, especially for this model, suggests challenges in capturing the full range of sentiment cues in shorter SST-2 texts. Moreover, the recall of \(X_{combined}^{Y}\) shows an improvement, indicating that the incorporation of Yelp and IMDb enhances the ability to extract sentiment cues. However, this integration also introduces additional information, which adversely affects the accuracy and precision of the model. In Table 5, the adaptability of the models to the Yelp dataset is more promising. The decrease in accuracy and the F1 score is less severe compared to their performance in the SST-2 dataset. This implies that the models are better equipped to handle the moderate text lengths and complexity of Yelp reviews. However, the performance of the model trained on the IMDb dataset is significantly poorer, especially in terms of recall. Similarly, the model trained on the combined dataset also experiences some degree of performance degradation, which may be attributed to the influence of the IMDb dataset.