# Sarcasm Detection Base on Adaptive Incongruity Extraction Network and Incongruity Cross-Attention
## 3 Methodology
### Incongruity Cross-Attention Module  
The external commonsense knowledge may contain a lot of noise information, which may be detrimental to our sarcasm detection if we do not screen the knowledge. In order to make better use of external knowledge, we propose an incongruity cross-attention module between text and external commonsense knowledge to fully extract useful commonsense knowledge. In this part, we will introduce the module in detail. Similarly, in order to make knowledge extraction learnable, we perform a linear transformation on \(T_{inco}\) and two linear transformations on \(K_{inco}\), which are calculated as follows:  
\[\left\{\begin{array}{l}T_{inco}^{Q}=W_{Q}^{\prime}T_{inco}+b_{Q}^{\prime}=W_ {Q}^{\prime}\Big{[}i_{1}^{inco},i_{2}^{inco},\cdots,i_{n}^{inco}\Big{]}+b_{Q}^ {\prime}=\begin{bmatrix}i_{1}^{incoq},i_{2}^{incoq},\cdots,i_{n}^{incoq}\\ \vdots\\ i_{1}^{incoq},i_{2}^{incoq},\cdots,i_{n}^{incoq}\Big{]}\\ K_{inco}^{V}=W_{Q}^{\prime}K_{inco}+b_{Q}^{\prime}=W_{Q}^{\prime}\Big{[}k_{1} ^{  
where \(W^{\prime}_{Q}\in R^{d*d},\ W^{\prime}_{K}\in R^{d*d},\ W^{\prime}_{V}\in R^{d*d},\ b^{ \prime}_{Q}\in R^{n},\ b^{\prime}_{K}\in R^{n},\ b^{\prime}_{V}\in R^{n}\) are the trainable parameters in these three linear transformations. \(T^{Q}_{inco}\in R^{d*n}\), \(K^{K}_{inco}\in R^{d*m},\ K^{V}_{inco}\in R^{d*m}\) are the new feature representations after linear transformation.  
To extract the effective information from commonsense knowledge, we have to compute the Euclidean distance for each word vector in \(T^{Q}_{inco}\) and \(K^{K}_{inco}\). Specifically, for word pairs \(\left(t^{incoq}_{i},k^{incok}_{j}\right)\), we calculate it as follows:  
\[d^{\prime}_{i,j}=\parallel t^{incoq}_{i}-k^{incok}_{j}\parallel_{2} \tag{8}\]  
where \(d^{\prime}_{i,j}\in R\) represents the Euclidean distance between the two vectors \(t^{incoq}_{i}\) and \(k^{incok}_{j}\), \(t^{incoq}_{i}\) the \(i\)-th word in the text sequence and \(k^{incok}_{j}\) the \(j\)-th word in the corresponding knowledge sequence. Similar to the method of the adaptive incongruity extraction module, we can also obtain the cross-incongruity matrix \(D^{\prime}\) and the score matrix \(\mathit{Scores}^{\prime}\) as follows:  
\[D^{\prime}=\begin{pmatrix}d^{\prime}_{1,1}&d^{\prime}_{1,2}&\cdots&d^{\prime} _{1,m}\\ d^{\prime}_{2,1}&d^{\prime}_{2,2}&\cdots&d^{\prime}_{2,m}\\ \vdots&\vdots&\ddots&\cdots\\ d^{\prime}_{n,1}&d^{\prime}_{n,2}&\cdots&d^{\prime}_{n,m}\end{pmatrix} \tag{9}\]  
\[\mathit{Scores}^{\prime}=\begin{pmatrix}s^{\prime}_{1,1}&s^{\prime}_{1,2}& \cdots&s^{\prime}_{1,m}\\ s^{\prime}_{2,1}&s^{\prime}_{2,2}&\cdots&s^{\prime}_{2,m}\\ \vdots&\vdots&\ddots&\cdots\\ s^{\prime}_{n,1}&s^{\prime}_{n,2}&\cdots&s^{\prime}_{n,m}\end{pmatrix} \tag{10}\]  
where \(D^{\prime}\in R^{n*m}\), \(\mathit{Scores}^{\prime}\in R^{n*m}\), \(s^{\prime}_{i,j}\in R\) represents the distance score of two words, one word in the text and each word and the other one in the corresponding commonsense knowledge.  
Then, the interaction score matrix \(\mathit{Scores}^{\prime}\) and \(K^{V}_{inco}\) were used for matrix multiplication so that useful information in the knowledge could be extracted. We also utilize the residual structure to better retain a part of the original information. The calculation process is as follows:  
\[TK_{inco}=LN\Big{(}K^{V}_{inco}\mathit{Scores}^{\prime T}+T_{inco}\Big{)} \tag{11}\]  
where \(LN\) is the operation of layer normalization [51], and \(TK_{inco}\in R^{d*n}\) represents the new features extracted by the incongruity cross-attention module between text and knowledge, which serves as a feature new representation of commonsense knowledge.