# Sarcasm Detection Base on Adaptive Incongruity Extraction Network and Incongruity Cross-Attention
## 3 Methodology
### Adaptive Incongruity Extraction Network  
Incongruous sentiment information is crucial for sarcasm detection. In this part, we will describe the adaptive incongruity extraction module in detail, which captures the sentiment incongruity information inside the sequence. The key point of this part is the learnable adaptive incongruity matrix, which is used to calculate the distance between each word in the input sequence. In previous works [2; 5], the basic method calculates the inner product, which mainly calculates the similarity. Therefore, [2; 5] cannot effectively extract incongruous information. At the same time, after calculating the score matrix, they performed the max operation on each row, which could only obtain the incongruent information between two words in a text sequence, thus losing a lot of information. Because of the incongruuous information, not just between two words, but possibly between some phrases, in order to make the adaptive incongruity matrix learnable, we first apply three different linear transformations to the input text sequence \(T_{emb}\in R^{dsl}\), which is encoded by BERT. Thus, we obtain the new feature representations as follows:  
\[ \begin{cases}T_{emb}^{Q}=W_{Q}T_{emb}+b_{Q}=W_{Q}[t_{1},t_{2},\cdots,t_{n}]+b_ {Q}=\left[t_{1}^{q},t_{2}^{q},\cdots,t_{n}^{q}\right]\\ T_{emb}^{K}=W_{K}T_{emb}+b_{K}=W_{K}[t_{1},t_{2},\cdots,t_{n}]+b_{K}=\left[t_{ 1}^{k},t_{2}^{k},\cdots,t_{n}^{k}\right]\\ T_{emb}^{V}=W_{V}T_{emb}+b_{V}=W_{V}[t_{1},t_{2},\cdots,t_{n}]+b_{V}=\left[t_{ 1}^{p},t_{2}^{p},\cdots,t_{n}^{p}\right]\end{cases}  \tag{1}\]  
where \(W_{Q}\in R^{dsl}\), \(W_{K}\in R^{dsl}\), \(W_{V}\in R^{dsl}\), \(b_{Q}\in R^{n}\), \(b_{K}\in R^{n}\), \(b_{V}\in R^{n}\) are the trainable parameters in these three linear transformations. \(T_{emb}^{Q}\in R^{dsl}\), \(T_{emb}^{K}\in R^{dsl}\), \(T_{emb}^{V}\in R^{dsl}\) are the new feature representations after linear transformation. Next, we compute the Euclidean distance between different words in \(T_{emb}^{Q}\) and \(T_{emb}^{K}\). To be specific, for word pairs \(\left(t_{i}^{q},t_{j}^{k}\right)\), we compute their Euclidean distance \(d_{i,j}\) as follows:  
\[d_{i,j}=\parallel t_{i}^{q}-t_{j}^{k}\parallel_{2} \tag{2}\]  
where \(d_{i,j}\in R\) denotes the Euclidean distance between the two vectors \(e_{i}^{q}\) and \(e_{j}^{k}\). To a certain extent, after [PERSON] embeds a word, the farther the semantic distance between twowords is, the larger the value of their Euclidean distance is, which can better capture the semantic incongruity information in the text. We get an adaptive incongruity matrix \(D\) based on all Euclidean distances \(d_{i,j},\ i,j\in(1,2,\cdots,n)\). \[D= \begin{pmatrix}d_{1,1}&d_{1,2}&\cdots&d_{1,n}\\ d_{2,1}&d_{2,2}&\cdots&d_{2,n}\\ \vdots&\vdots&\ddots&\cdots\\ d_{n,1}&d_{n,2}&\cdots&d_{n,n}\end{pmatrix}  \tag{3}\]  
where \(D\in R^{n*n}\) consists of the distance between individual words in the input text. Since each element in the matrix \(D\) is quite large, we need to mask it first and then normalize each row. For example, we normalize the row \(i\) as follows:  
\[m_{i}=(s_{i,1},s_{i,2},\cdots,s_{i,n})=softmax(d_{i,1},d_{i,2},\cdots,d_{i,n}) \tag{4}\]  
where \(m_{i}\in R^{n}\) represents the semantic gap between the \(i\)-th word and the other words. The larger the value is, the farther the semantic gap is for the corresponding word pairs. At the same time, we do not use the max function as in papers [2,5]; only the largest incongruity between word \(i\) and other words is obtained after using the Max function in papers [2,5]; their method would lose a lot of information because the semantic incongruity within a sentence should occur between two phrases, not just between two words. The incongruity is computed using the softmax operator between a word and multiple other words. A score matrix _Scores_ measuring semantic incongruity is obtained as follows:  
\[Scores=