# Interpretability in Sentiment Analysis: A Self-Supervised Approach to Sentiment Cue Extraction
## 4 Methodology
### Generating Dataset for Sentiment Cue Extraction
#### 4.2.3 Mask Sequence Interpretation Score  
In the SCE task, for a given input \(X\) with labels \(Y\), there exists an inverse sequence \(\bar{Y}=\{1-y_{1},1-y_{2},\cdots,1-y_{n}\}\). As defined by the task, if the token \(x_{i}\) in \(X\) is identified as an SC within \(X\), then the corresponding label \(y_{i}\) is assigned a value of 1; if not, \(y_{i}\) is set to 0. This principle suggests that masking all tokens \(x_{i}\) in \(X\) for which \(y_{i}=1\), resulting in a masked input \(X^{\bar{Y}}\), would hinder the sentiment classification model's ability to accurately determine the sentiment inclination of \(X^{\bar{Y}}\). Conversely, retaining only the tokens in \(X\), where \(y_{i}=1\), and masking those with \(y_{i}=0\), to create a new input \(X^{Y}\), should enable the sentiment classification model to predict its sentiment inclination effectively. In typical scenarios, to obtain \(X^{Y}\), it is necessary to replace tokens \(x_{i}\) in \(X\) corresponding to \(y_{i}=0\) in \(Y\) with a meaningless symbol like [MASK]. However, BERT provides a more straightforward solution for us. By using \(Y\) as the attention mask directly input into BERT, it automatically disregards tokens \(x_{i}\) in \(X\) corresponding to \(y_{i}=0\) in \(Y\). Therefore, for \(X\), when using \(Y\) as the mask sequence, we obtain:  
\[(p_{c0}^{Y},p_{c1}^{Y})=f_{\text{sc}}(X,Y), \tag{7}\]  
yielding the sentiment category prediction:  
\[C^{Y}=\arg\max(p_{c0}^{Y},p_{c1}^{Y}), \tag{8}\]  
and calculating the Probability Discrepancy as:  
\[\Delta P^{Y}=|p_{c0}^{Y}-p_{c1}^{Y}|. \tag{9}\]  
Similarly, when using the inverse sequence \(\bar{Y}\) as the mask sequence, we can determine:  
\[(p_{c0}^{\bar{Y}},p_{c1}^{\bar{Y}})=f_{\text{sc}}(X,\bar{Y}), \tag{10}\]  
with the corresponding sentiment category determined by:  
\[C^{\bar{Y}}=\arg\max(p_{c0}^{\bar{Y}},p_{c1}^{\bar{Y}}), \tag{11}\]  
and the Probability Discrepancy for \(X^{\bar{Y}}\) calculated as:  
\[\Delta P^{\bar{Y}}=|p_{c0}^{\bar{Y}}-p_{c1}^{\bar{Y}}|. \tag{12}\]  
When selecting a candidate sequence \(Y\) as the pseudo label, the ideal scenario aims to maximize \(\Delta P^{Y}\) while ensuring that \(C^{Y}=C\), and simultaneously minimize \(\Delta P^{\bar{Y}}\). However, this approach might lead to an extreme case where all elements of \(Y\) are set to 1 and all elements of \(\bar{Y}\) are set to 0. In such a scenario, \(X^{Y}\) would be identical to \(X\), and \(X^{\bar{Y}}\) would contain no informative content, which, while adhering to the principle, is not desirable for effective sentiment cue extraction. To circumvent this issue, it is preferable to have as few elements labeled as 1 in \(Y\) as possible. To achieve this balance, we introduce the Ratio of Cue Tokens (RCT), calculated as follows:  
\[RCT=\frac{\sum(Y)}{n} \tag{13}\]  
where \(\sum(Y)\) represents the number of elements valued at 1 in \(Y\), and \(n\) denotes the total number of elements in \(Y\). Moreover, within \(X\), there may be tokens that inversely affect the prediction of \(X\)'s sentiment inclination. Such tokens might cause \(f_{\text{sc}}\) to predict the sentiment category of \(X^{\bar{Y}}\) as being entirely opposite to that of \(X\). In these situations, it is desirable for \(\Delta P^{\bar{Y}}\) to be as large as possible to reflect a clear differentiation in sentiment inclination. Taking into consideration the principles mentioned above, we propose an evaluation metric named the Mask Sequence Interpretation Score (MSIS) as follows:\[MSIS=\left\{ \begin{aligned} &\frac{\Delta P^{Y}}{\Delta P^{\bar{Y}}\cdot RCT ^{2}},\qquad C=C^{Y}\cap\Delta P^{\bar{Y}}<\Delta P^{Y}\\ &\frac{\Delta P^{Y}\cdot\Delta P^{\bar{Y}}}{RCT^{2}},\quad C=C^{Y} \
eq C^{\bar{Y}}\cap\Delta P^{\bar{Y}}\geq\Delta P^{Y}\\ & 0,\qquad\qquad\qquad\qquad\text{\emph{Others}}\end{aligned} \right. \tag{14}\]  
Algorithm 2 outlines the procedure for evaluating the MSIS for a given candidate sequence \(Y^{c}\) associated with an instance \(X\).