# An End-to-End Mutually Interactive Emotion-Cause Pair Extractor via Soft Sharing
## 3 Materials and Methods
### Architecture  
The whole model contains three layers as illustrated in Figure 2: word-level encoder layer, clause-level encoder layer, and pairing layer. We take the vector representation \(\mathbf{v}_{ij}\) of the \(j\)-th word in the \(i\)-th clause as input. For each clause, the word vector sequence \(\mathbf{v}_{i,1},\mathbf{v}_{i,2},\ldots,\mathbf{v}_{i,m}\) is passed through a word-level encoder, implemented via a Bi-LSTM with attention [47]. The word-level encoder outputs a clause representation \(s_{i}\) for each clause. The higher level contains two clause-level encoders implemented for emotion clause detection and cause clause detection, respectively. Any of the current mainstream encoders, such as the stacked Bi-LSTM [48] or BERT [49], can be used. The two encoders take the clause representation sequence \(\mathbf{s}_{1},\mathbf{s}_{2},\ldots,\mathbf{s}_{d}\) as input and generate the emotion and cause representation of the clauses \(\mathbf{r}_{i}^{e},\mathbf{r}_{i}^{e}\). In order to mutually transfer the information obtained by the encoders, we use a soft-sharing strategy between the two encoders. The representations are then fed into detectors (logistic regression layers) to obtain the probability distribution \(\mathbf{a}_{i}^{e}\), \(\mathbf{a}_{i}^{e}\) of the clause being an emotion clause and a cause clause, respectively, formed as  
\[ \begin{split}\mathbf{a}_{i}^{e}&=\text{softmax}(\mathbf{W}^ {e}\mathbf{r}_{i}^{e}+\mathbf{b}^{e}),\\ \mathbf{a}_{i}^{e}&=\text{softmax}(\mathbf{W}^{e}\mathbf{r}_{i}^ {e}+\mathbf{b}^{e}),\end{split}  \tag{2}\]  
where \(\mathbf{W}^{e}\), \(\mathbf{W}^{e}\), \(\mathbf{b}^{e}\), \(\mathbf{b}^{e}\) are the parameters for emotion and cause detection layers. It should be noted that \(\mathbf{a}_{i}^{e}\) is a \(1\times 2\) vector in which one element \(p_{i}^{e}\) represents the probability that it is indeed an emotion clause and the other element represents the probability that it is not. \(\mathbf{a}_{i}^{e}\) and \(p_{i}^{e}\) are related as above. It is worth noting that these two outputs \(\mathbf{a}_{i}^{e}\) and \(\mathbf{a}_{i}^{e}\) in our approach also need to participate in pair extraction later, instead of having only one function of obtaining supervisory signals, as in the two auxiliary task outputs in the work of [13]. Thus, in contrast to the cascade of the hierarchical framework of [13], our model achieves parallelism. These probabilities can be regarded as the attention of emotion and cause encoders. Thus, we multiply the clause representation by the probabilities to obtain the emotion-weighted and cause-weighted clause representation \(\mathbf{\tilde{r}}_{i}^{e}\), \(\mathbf{\tilde{r}}_{i}^{e}\) as  
\[ \begin{split}\mathbf{\tilde{r}}_{i}^{e}&=p_{i}^{e}\mathbf{r }_{i}^{e},\\ \mathbf{\tilde{r}}_{i}^{e}&=p_{i}^{e}\mathbf{r}_{i}^{e},\end{split}  \tag{3}\]  
Once we collect all the weighted representation of clauses into two sets \(\mathcal{E}=\tilde{r}_{1}^{e},\tilde{r}_{2}^{e},\ldots,\tilde{r}_{d}^{e}\) and \(\mathcal{C}=\tilde{r}_{1}^{e},\tilde{r}_{2}^{e},\ldots,\tilde{r}_{d}^{e}\), the Cartesian product is applied on the two sets to generate all the potential emotion-cause pairs \((\mathbf{\tilde{r}}_{i}^{e},\mathbf{\tilde{r}}_{i}^{e})\). In Figure 3, we use \(\mathbf{r}_{ij}^{p}=\mathbf{\tilde{r}}_{i}^{e}\oplus\tilde{r}_{i}^{e}\oplus\mathbf{d}_{ij}\) as the representation of a pair, where \(\oplus\) denotes the concatenation operator and \(\mathbf{d}_{ij}\) is the positional embedding vector that indicates the relative position relation between clause \(i\) and \(j\)[50]. The pairs are fed into the pairing layer one at a time to obtain the predicted label. The pairing layer is a fully connected layer as  
\[ \begin{split}\mathbf{h}_{ij}&=\text{ReLU}(\mathbf{W}^{h}\mathbf{r} _{ij}^{p}+\mathbf{b}^{h}),\\ \mathbf{\hat{g}}_{ij}^{p}&=\text{softmax}(\mathbf{W}^{y}\mathbf{h} _{ij}+\mathbf{b}^{y}),\end{split}  \tag{4}\]  
where \(\mathbf{\hat{g}}_{ij}^{p}\) gives the Bernoulli distribution probabilities of \((c_{i},c_{j})\) to be an emotion-cause pair.