# A Multitask Learning Framework for Abuse Detection and Emotion Classification
## 3 Proposed Method
### Decoder  
For different tasks, we used BiLSTM and TextCNN as two main components of different decoders because of their different focus. In addition, in order to better facilitate abuse detection by emotion classification, we propose a cross-attention enhancement component.  
**Decoder for Emotion Classification**. A two-layered bidirectional long short-term memory (LSTM) network was applied to BERT's output to obtain word contextualized representations. For a LSTM output \(H^{e}=\{h_{0},h_{1},\dots,h_{n}\}\), the BiLSTM further represents each token as:  
\[h^{e}_{t}=[\overrightarrow{h_{t}};\overleftarrow{h_{t}}] \tag{1}\]  
where \(\overrightarrow{h_{t}};\overleftarrow{h_{t}}\in\mathbf{R}^{d}\) and \(d\) denotes the hidden size of the BiLSTM. We used a maximum pooling to get a representation of the entire sentence:  
\[H^{e}_{max}=max(h_{1};h_{2};\dots,h_{n}) \tag{2}\]  
Then, the probability distribution representation of the emotional label was generated by a full connection layer with a softmax activation function:  
\[P^{e}=softmax(H^{e}_{max}W^{e}+b^{e}) \tag{3}\]  
where \(P^{e}\in\mathbf{R}^{m\
eq 2}\).  
**Decoder for Abuse Detection**. TextCNN [16] was used to obtain finer local context features, and a representation similar to n-gram is obtained by adjusting the size of the convolution kernel. We then employed dynamic maximum pooling to capture sentence level representations of varying granularity. The output of textCNN is represented as:  
\[H^{a}_{k}=TextCNN_{k}(X) \tag{4}\]  
For different convolution kernels \(k\), we obtained different sentence-level representations \(\{H^{a}_{0},\dots,H^{a}_{k}\}\). In the actual operation, we chose three different convolution kernels of size \(\{2,4,6\}\) for StormW and \(\{2,3,4\}\) for the others. The final expression of the sentence was the concatenation of the output of the three kernels:  
\[H^{a}=[H^{a}_{2};H^{a}_{3};H^{a}_{4}] \tag{5}\]  
where \(H^{a}\in\mathbf{R}^{d^{\prime}\
eq 3}\) and \(d^{\prime}\) is the number of output channels of the CNN. As with the emotion classification decoder, we used a full connection layer with softmax to obtain the probability distribution:  
\[P^{a}=softmax(H^{a}W^{a}+b^{a}) \tag{6}\]  
**Cross Attention (CA)**. Although the input of BiLSTM and TextCNN \(X\) are derived from BERT, the results should be more focused on different subtasks after the decoder. CA is designed to capture the actual impact of the emotion classification subtask on the main task. Like self-attention [40; 41], cross attention can essentially be described as a mappingfrom a query to a set of key-value pairs, except that the key is derived from the BiLSTM's output \(H^{e}\):  
\[\begin{split} K&=XW^{K}\\ Q&=H^{e}W^{K}\\ V&=XW^{V}\end{split} \tag{7}\]  
On this basis, CA was calculated by:  
\[CA(K,Q,V)=softmax(\frac{QK^{T}}{\sqrt{d_{ca}}}V) \tag{8}\]  
To ensure consistency of dimensions, we set \(d_{ca}=d=768\), \(K,Q,V\in\mathbf{R}^{768\times 768}\). The output of the CA was fed to the normalization layer:  
\[H^{ca}=norm(CA(K,Q,V)) \tag{9}\]  
Finally, we replaced \(X\) in (4) with the emotional classification augmented results \(H^{ca}\). The overall flow of cross attention is shown in Figure 2. CA can effectively control the effect of the results of affective classification on abuse detection, and we further explore this in the ablation study Section 4.5.2.