Venue: NAACL 2021

# Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection

 Xu Guo, Boyang Li*, Han Yu and Chunyan Miao\({}^{*}\)

School of Computer Science and Engineering,

Nanyang Technological University, Singapore

{xu008, han.yu, boyang.li, ascymiaoo}@ntu.edu.sg

###### Abstract

The existence of multiple datasets for sarcasm detection prompts us to apply transfer learning to exploit their commonality. The adversarial neural transfer (ANT) framework utilizes multiple loss terms that encourage the source-domain and the target-domain feature distributions to be similar while optimizing for domain-specific performance. However, these objectives may be in conflict, which can lead to optimization difficulties and sometimes diminished transfer. We propose a generalized latent optimization strategy that allows different losses to accommodate each other and improves training dynamics. The proposed method outperforms transfer learning and meta-learning baselines. In particular, we achieve 10.02% absolute performance gain over the previous state of the art on the iSarcasm dataset.

+
Footnote †: Corresponding authors

_Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5394–5407

June 6-11, 2021. ©2021 Association for Computational Linguistics

## 1 Introduction

Sarcastic language is commonly found in social media posts Gonzalez-Ibanez et al. (2011); Maynard and Greenwood (2014), forum discussions Khodak et al. (2018), product reviews Davidov et al. (2010); Filatova (2012) and everyday conversations Gibbs (2000). Detecting sarcasm is an integral part of creative language understanding Veale et al. (2019) and online opinion mining Kannangara (2018). Due to highly contextualized expressions, detecting sarcasm is a challenging task, even for humans Fox Tree et al. (2020).

A challenge specific to sarcasm detection is the difficulty in acquiring ground-truth annotations. Human-annotated datasets Filatova (2012); Riloff et al. (2013); Van Hee et al. (2018); Oprea and Magdy (2020) usually contain only a few thousand texts, resulting in many small datasets. In comparison, automatic data collection using distant supervision signals like hashtags Ptacek et al. (2014); Bamman and Smith (2015); Joshi et al. (2015) yielded substantially larger datasets. Nevertheless, the automatic approach also led to label noise. For example, Oprea and Magdy (2020) found nearly half of the tweets with sarcasm hashtags in one dataset are not sarcastic.

The existence of diverse datasets and data collection methods prompts us to exploit their commonality through transfer learning. Specifically, we transfer knowledge learned from large and noisy datasets to improve sarcasm detection on small human-annotated datasets that serve as effective performance benchmarks.

Adversarial neural transfer (ANT) Ganin and Lempitsky (2015); Liu et al. (2017); Kim et al. (2017); Kamath et al. (2019) employs an adversarial setup where the network learns to make the shared feature distributions of the source domain and the target domain as similar as possible, while simultaneously optimizing for domain-specific performance. However, as the domain-specific losses promote the use of domain-specific features, these training objectives may compete with each other implicitly. This leads to optimization difficulties and potentially degenerate cases where the domain-specific classifiers ignore the shared features and no meaningful transfer occurs between domains.

To cope with this issue, we propose Latent-Optimized Adversarial Neural Transfer (LOANT). The latent optimization strategy can be understood with analogies to to one-step look-ahead during gradient descent and Model-Agnostic Meta Learning Finn et al. (2017). By forcing domain-specific losses to accommodate the negative domain discrimination loss, it improves training dynamics Balduzzi et al. (2018).

With LOANT, we achieve 10.02% absolute improvement over the previous state of the art on the iSarcasm dataset Oprea and Magdy (2020) and 3.08% improvement on SemEval-18 dataset Van Hee et al. (2018). Over four sets of transfer learning experiments, latent optimization on aver

age brings 3.42% improvement in F-score over traditional adversarial neural transfer and 4.83% over a similar training strategy from Model-Agnostic Meta Learning (MAML) Finn et al. (2017). In contrast, traditional ANT brings an average of only 0.9% F-score improvement over non-adversarial multi-task learning. The results demonstrates that LOANT can effectively perform knowledge transfer for the task of sarcasm detection and suggests that the proposed latent optimization strategy enables the collaboration among the ANT losses during optimization.

Our contributions can be summarized as follows:

1. Inspired by the existence of multiple small sarcasm datasets, we propose to use transfer learning to bridge dataset differences. To the best of our knowledge, this is the first study of transfer learning between different sarcasm detection datasets.
2. We propose LOANT, a novel latent-optimized adversarial neural transfer model for cross-domain sarcasm detection. By conducting stochastic gradient descent (SGD) with one-step look-ahead, LOANT outperforms traditional adversarial neural transfer, multi-task learning, and meta-learning baselines, and establishes a new state-of-the-art F-score of 46.41%. The code and datasets are available at [https://github.com/guoxuxu/LOANT](https://github.com/guoxuxu/LOANT).

## 2 Related Work

### Sarcasm Detection

Acquiring large and reliable datasets has been a persistent challenge for computational detection of sarcasm. Due to the cost of annotation, manually labeled datasets Walker et al. (2012); Riloff et al. (2013); Wallace et al. (2014); Abercrombie and Hovy (2016); Oraby et al. (2016); Van Hee et al. (2018); Oprea and Magdy (2020) typically contain only a few thousand texts. Automatic crawling Ptacek et al. (2014); Bamman and Smith (2015); Joshi et al. (2015); Khodak et al. (2018) using hashtags or markers yields substantially more texts, but the results are understandably more noisy. As a case study, after examining the dataset of Riloff et al. (2013), Oprea and Magdy (2020) found that nearly half of tweets with sarcasm hashtags are not sarcastic. In this paper, we evaluate performance on the manually labeled datasets, which are relatively clean and can serve as good benchmarks, and transfer the knowledge learned from automatically collected datasets.

Traditional sarcasm detection includes methods based on rules Tepperman et al. (2006) and lexical Kreuz and Caucci (2007) and pragmatic patterns Gonzalez-Ibanez et al. (2011). Context-aware methods Rajadesingan et al. (2015); Bamman and Smith (2015) make use of contexts, such as the author, the audience, and the environment, to enrich feature representations.

Deep learning techniques for sarcasm detection employ convolutional networks Ghosh and Veale (2016), recurrent neural networks Zhang et al. (2016); Felbo et al. (2017); Wu et al. (2018), attention Tay et al. (2018), and pooling Xiong et al. (2019) operations. Amir et al. (2016) incorporate historic information for each Twitter user. Cai et al. (2019) consider the images that accompany tweets and Mishra et al. (2017) utilize readers' gaze patterns. To the best of our knowledge, no prior work has explored transfer learning between different sarcasm datasets.

### Adversarial Transfer Learning

As a transfer learning technique, multi-task learning (MTL) allows related tasks or similar domains to inform each other and has been a powerful technique for NLP Collobert et al. (2011); Yang et al. (2017); Aharoni et al. (2019); Guo et al. (2019); Raffel et al. (2020). However, MTL does not always lead to performance improvements Alonso and Plank (2017); Bingel and Sogaard (2017); Changpinyo et al. (2018); Clark et al. (2019).

Theoretical analysis Ben-David et al. (2010) indicates that a key factor for the success of transfer is to reduce the divergence between the feature spaces of the domains. Ganin and Lempitsky (2015) propose to minimize domain differences via a GAN-like setup, where a domain discriminator network learns to distinguish between features from two domains and a feature extraction network learns to produce indistinguishable features, which are conducive to transfer learning.

Similar adversarial setups Liu et al. (2017); Kim et al. (2017) have been adopted for many NLP tasks, such as sentiment analysis Chen et al. (2018); Liu et al. (2018), satirical news detection McHardy et al. (2019), detection of duplicate questions Kamath et al. (2019), named entity recognition Zhou 

et al., 2019), and QA Yu et al. (2018).

However, as shown in our experiments, adding the domain discriminator to MTL does not always result in improved performance. We attribute this to the implicit competition between the negative domain discrimination loss and the domain-specific losses, which causes difficulties in optimization. In this paper, we improve the training dynamics of adversarial transfer learning using latent optimization on BERT features.

### Meta-Learning and Latent Optimization

The idea of coordinating gradient updates of different and competing losses using gradient descent with look-ahead has been explored in Latent-optimized Generative Adversarial Network (LOGAN) Wu et al. (2019, 2019), Symplectic Gradient Adjustment Balduzzi et al. (2018); Gemp and Mahadevan (2019), Unrolled GAN Metz et al. (2016), Model-Agnostic Meta Learning Finn et al. (2017) and extragradient Azizian et al. (2020). The difference between LOGAN and other techniques is that the LOGAN computes the derivative of the randomly sampled latent input, whereas other methods compute the second-order derivative in the model parameter space.

In this paper, we generalize latent optimization from GANs to multi-task learning, where the adversarial loss is complemented by domain-specific task losses. In addition, we apply latent optimization on the output of the BERT module, which differs from the optimization of the random latent variable in LOGAN. As large pretrained masked language models (PMLMs) gain prominence in NLP, latent optimization avoids gradient computation on the parameters of enormous PMLMs, providing reduction in running time and memory usage.

## 3 The LOANT Method

In supervised transfer learning, we assume labeled data for both the source domain and the target domain are available. The source domain dataset \(D_{s}\) comprises of data points in the format of \((x_{s},y_{s})\) and the target domain dataset \(D_{t}\) comprises of data points in the format of \((x_{t},y_{t})\). The labels \(y_{s}\) and \(y_{t}\) are one-hot vectors. The task of supervised cross-domain sarcasm detection can be formulated as learning a target-domain function \(f_{t}(x_{t})\) that predict correct labels for unseen \(x_{t}\).

### Model Architecture

Fig. 1 shows the model architecture for adversarial neural transfer (ANT) Liu et al. (2017); Kamath et al. (2019); Kim et al. (2017). We use a large pre-trained neural network, BERT Devlin et al. (2019), as the sentence encoder, though the architecture is not tied to BERT and can use other pretrained encoders. We denote the parameters of the BERT encoder as \(w_{b}\), and its output for data in the source domain and the target domain as \(z_{s}\in\mathbb{R}^{D}\) and \(z_{t}\in\mathbb{R}^{D}\) respectively. We denote this encoder operation as

\[z_{s}=E(x_{s},w_{b}),\:z_{t}=E(x_{t},w_{b}) \tag{1}\]

On top of these outputs, we apply domain-specific dense layers to create domain-specific features \(v_{s},v_{t}\) and shared dense layers to create shared features \(u_{s},u_{t}\). We use \(w_{s}\), \(w_{t}\), and \(w_{sh}\) to denote the parameters for the source dense layers, the target dense layers, and the shared dense layers.

The concatenation of features \([v_{s},u_{s}]\) is fed to the source-domain classifier, parameterized by \(\theta_{s}\); \([v_{t},u_{t}]\) is fed to the target-domain classifier, parameterized by \(\theta_{t}\). The two classifiers categorize the tweets into sarcastic and non-saracastic and are trained using cross-entropy. For reasons that will become apparent later, we make explicit the reliance on \(z_{s}\) and \(z_{t}\):

\[\begin{split}&\mathcal{L}_{s}(z_{s})=-\sum_{i}y_{s,i}\log p(\hat{ y}_{s,i}|z_{s}),\\ &\mathcal{L}_{t}(z_{t})=-\sum_{i}y_{t,i}\log p(\hat{y}_{t,i}|z_{ t}),\end{split} \tag{2}\]

Figure 1: Network architecture of the Adversarial Neural Transfer model.

 

where \(\hat{y}_{s}\) and \(\hat{y}_{t}\) are the predicted labels and \(i\) is the index of the vector components.

Simultaneously, the domain discriminator learns to distinguish the features \(u_{s}\) and \(u_{t}\) as coming from different domains. The domain discriminator is parameterized by \(\theta_{d}\). It is trained to minimize the domain classification loss,

\[\mathcal{L}_{d}(z_{t},z_{s})=-\log p(0|u_{s})-\log p(1|u_{t}). \tag{3}\]

Through the use of the gradient reversal layer, the shared dense layers and the feature encoder maximizes the domain classification loss, so that the shared features \(u_{s}\) and \(u_{t}\) become indistinguishable and conducive to transfer learning. In summary, the network weights \(w_{b},w_{s},w_{t},w_{sh},\theta_{s},\theta_{t}\) are trained to minimize the following joint loss,

\[\mathcal{L^{\text{ANT}}}=\mathcal{L}_{s}(z_{s})+\mathcal{L}_{t}(z_{t})- \mathcal{L}_{d}(z_{t},z_{s}), \tag{4}\]

whereas \(\theta_{d}\) is trained to minimize \(\mathcal{L}_{d}(z_{t},z_{s})\).

It is worth noting that the effects of three loss terms in Eq. 4 on the shared parameters \(w_{sh}\) and \(w_{b}\) may be competing with each other. This is because optimizing sarcasm detection in one domain will encourage the network to extract domain-specific features, whereas the domain discrimination loss constrains the network to avoid such features. It is possible for the competition to result in degenerate scenarios. For example, the shared features \(u_{s}\) and \(u_{t}\) may become indistinguishable but also do not correlate with the labels \(y_{s}\) and \(y_{t}\). The domain classifiers may ignore the shared features \(u_{s}\) and \(u_{t}\) and hence no transfer happens. To cope with this issue, we introduce a latent optimization strategy that forces domain-specific losses to accommodate the domain discrimination loss.

### Latent Representation Optimization

We now introduce the latent representation optimization strategy. First, we perform one step of stochastic gradient descent on \(-\mathcal{L}_{d}\) on the encoded features \(z_{s}\) and \(z_{t}\) with learning rate \(\gamma\),

\[z_{s}^{\prime} =z_{s}+\gamma\frac{\partial\mathcal{L}_{d}(z_{s},z_{t})}{\partial z _{s}}, \tag{5}\] \[z_{t}^{\prime} =z_{t}+\gamma\frac{\partial\mathcal{L}_{d}(z_{s},z_{t})}{\partial z _{t}}. \tag{6}\]

We emphasize that this is a _descent_ step because we are minimizing \(-\mathcal{L}_{d}\).

After that, we use the updated \(z_{s}^{\prime}\) and \(z_{t}^{\prime}\) in the computation of the losses

\[\mathcal{L}_{s}^{\text{LO}}(z_{s},z_{s}^{\prime}) =\mathcal{L}_{s}(z_{s})+\mathcal{L}_{s}(z_{s}^{\prime}), \tag{7}\] \[\mathcal{L}_{t}^{\text{LO}}(z_{t},z_{t}^{\prime}) =\mathcal{L}_{t}(z_{t})+\mathcal{L}_{t}(z_{t}^{\prime}). \tag{8}\]

The new joint objective hence becomes

\[\begin{split}\mathcal{L}^{\text{LO}}&=\mathcal{ L}_{s}^{\text{LO}}(z_{s},z_{s}^{\prime})+\mathcal{L}_{t}^{\text{LO}}(z_{ t},z_{t}^{\prime})\\ &\quad-\mathcal{L}_{d}(z_{s},z_{t}),\end{split} \tag{9}\]

which is optimized using regular stochastic gradient descent (SGD) on \(w_{b},w_{s},w_{t},w_{sh},\theta_{s}\), and \(\theta_{t}\).

Here we show the general case of gradient computation. Consider any weight vector \(w\) in the neural network. Equations 5 and 6 introduce two intermediate variables \(z_{s}^{\prime}\) and \(z_{t}^{\prime}\), which are a function of the model parameter \(w\). Therefore, we perform SGD using the following total derivative

\[\begin{split}\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{ d}w}&=\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w}+\frac{ \partial\mathcal{L}_{s}^{\text{LO}}(z_{s}^{\prime})}{\partial z_{s}^{\prime}} \frac{\partial z_{s}^{\prime}}{\partial w}\\ &\quad+\frac{\partial\mathcal{L}_{t}^{\text{LO}}(z_{t}^{\prime})} {\partial z_{t}^{\prime}}\frac{\partial z_{t}^{\prime}}{\partial w}.\end{split} \tag{10}\]

where

\[\begin{split}\frac{\partial z_{s}^{\prime}}{\partial w}& =\frac{\partial z_{s}}{\partial w}+\gamma\frac{\partial^{2} \mathcal{L}_{d}(z_{s})}{\partial z_{s}\,\partial w}\\ \frac{\partial z_{t}^{\prime}}{\partial w}&=\frac{ \partial z_{t}}{\partial w}+\gamma\frac{\partial^{2}\mathcal{L}_{d}(z_{t})}{ \partial z_{t}\,\partial w}\end{split} \tag{11}\]

For every network parameter other than the encoder weight \(w_{b}\), \(\partial z/\partial w\) is zero. The second-order derivative \(\partial^{2}\mathcal{L}_{d}(z)/\partial z\,\partial w\) is difficult to compute due to the high dimensionality of \(w\). Since \(\gamma\) is usually very small, we adopt a first-order approximation and directly set the second-order derivative to zero. Letting \(\phi_{s}=[w_{s},\theta_{s}]\) and \(\phi_{t}=[w_{t},\theta_{t}]\), we now show the total derivatives for all network

Figure 2: Schematic of the latent optimization strategy. The solid black arrows indicate the forward pass and the dotted red arrows indicate the backward pass.

 

parameters:

\[\begin{split}&\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{ \mathrm{d}w_{b}}=\frac{\partial\mathcal{L}^{\text{ANT}}}{\partial w_{b}}+\frac{ \partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial w_{b}}+\frac{\partial \mathcal{L}_{t}(z_{t}^{\prime})}{\partial w_{b}}\\ &+\frac{\partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial z_{s}^{ \prime}}\frac{\partial z_{s}}{\partial w_{b}}+\frac{\partial\mathcal{L}_{t}(z_ {t}^{\prime})}{\partial z_{t}^{\prime}}\frac{\partial z_{t}}{\partial w_{b}} \\ &\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d}w_{sh}}=\frac{ \partial\mathcal{L}^{\text{ANT}}}{\partial w_{sh}}+\frac{\partial\mathcal{L}_{ s}(z_{s}^{\prime})}{\partial w_{sh}}+\frac{\partial\mathcal{L}_{t}(z_{t}^{ \prime})}{\partial w_{sh}}\\ &\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d}\phi_{s}}= \frac{\partial\mathcal{L}_{s}(z_{s})}{\partial\phi_{s}}+\frac{\partial \mathcal{L}_{s}(z_{s}^{\prime})}{\partial\phi_{s}}\\ &\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d}\phi_{t}}= \frac{\partial\mathcal{L}_{t}(z_{t})}{\partial\phi_{t}}+\frac{\partial \mathcal{L}_{t}(z_{t}^{\prime})}{\partial\phi_{t}}\\ &\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d}\theta_{d}}= \frac{\partial\mathcal{L}_{d}(z_{s},z_{t})}{\partial\theta_{d}}\end{split} \tag{14}\]

More details can be found in Appendix A. Fig. 2 illustrates the latent optimization process. Algorithm 1 shows the LOANT algorithm.

```
Input: source data \((x_{s},y_{s})\), target data \((x_{t},y_{t})\), learning rate \(\gamma\) Initialize model parameters \(w\) repeat Sample \(N\) batches of data pairs for i = 1 to Ndo Compute forward loss \(\mathcal{L}_{s}\), \(\mathcal{L}_{t}\), \(\mathcal{L}_{d}\); Compute \(\triangle z_{s}=\frac{\partial\mathcal{L}_{d}(z_{s})}{\partial z_{s}}\) and \(\triangle z_{t}=\frac{\partial\mathcal{L}_{d}(z_{t})}{\partial z_{t}}\); Update the latent representations \(z_{s}^{\prime}=z_{s}+\gamma\triangle z_{s}\) and \(z_{t}^{\prime}=z_{t}+\gamma\triangle z_{t}\); Compute the new joint loss \(\mathcal{L}^{\text{LO}}=\mathcal{L}_{s}^{\text{LO}}+\mathcal{L}_{t}^{\text{LO} }-\mathcal{L}_{d}\); Update \(w\) using gradient descent. untilthe maximum training epoch
```

**Algorithm 1**Training of LOANT

To better understand the LOANT algorithm, we relate LOANT to the extragradient technique and Model-Agnostic Meta Learning Finn et al. (2017).

The vanilla gradient descent (GD) algorithm follows the direction along which the function value decreases the fastest. However, when facing an ill-conditioned problem like the one in Fig. 3, GD is known to exhibit slow convergence because the local gradients are close to being orthogonal to the direction of the local optimum.

For comparison with LOANT, we consider the extragradient (EG) method Korpelevich (1976); Azizian et al. (2020) that uses the following update rule when optimizing the function \(f(w)\) with respect to \(w\),

\[w\gets w-\eta\frac{\mathrm{d}f(w-\gamma\frac{\partial f(w)}{\partial w})}{ \mathrm{d}w}. \tag{15}\]

Similar to LOANT, we can adopt a first-order approximation to EG if we set the Hessian term to zero in the total derivative. Instead of optimizing the immediate function value \(f(w)\), this method optimizes \(f(w-\gamma\frac{\partial f(w)}{\partial w})\), which is the function value after one more GD step. This can be understood as looking one step ahead along the optimization trajectory. In the contour diagrams of Fig. 3, we show the optimization of a 2-dimensional quadratic function. This simple example showcases how the ability to look one step ahead can improve optimization in pathological loss landscapes. We motivate the nested optimization of LOANT by drawing an analogy between EG and LOANT.

It is worth noting that LOANT differs from the EG update rule in important ways. Specifically, in EG the inner GD step and the outer GD step are performed on the same function \(f(\cdot)\), whereas LOANT performs the inner step on \(\mathcal{L}_{d}\) and the outer step on \(\mathcal{L}_{s}\) or \(\mathcal{L}_{t}\).

For a similar idea with multiple losses, we turn to MAML Finn et al. (2017). In MAML, there are \(K\) tasks with losses \(\mathcal{L}_{1},\dots,\mathcal{L}_{k},\dots,\mathcal{L}_{K}\). On every task, we perform a one-step SGD update to the model parameter \(w\in\mathbb{R}^{L}\),

\[w_{T_{k}}=w-\gamma\frac{\partial\mathcal{L}_{k}(w)}{\partial w}. \tag{16}\]

After going through \(K\) tasks, the actual update to \(w\) is calculated using the parameters \(w_{T_{k}}\),

\[w\gets w-\eta\frac{1}{K}\sum_{k}\frac{\mathrm{d}\mathcal{L}_{ k}(w_{T_{k}})}{\mathrm{d}w}. \tag{17}\]

Utilizing the idea of look ahead, in MAML we update \(w\) so that subsequent optimization on any single task or combination of tasks would achieve good results.

Adversarial neural transfer has three tasks, the source-domain and target-domain classifications and the negative discriminator loss. The updates performed by LOANT in Eq. 5 and 6 are similar to MAML's look-ahead update in Eq. 16. Specifically, when we update model parameters using the gradient from the total loss \(\mathcal{L}^{\text{LO}}\), we prepare for the next descent step on \(-\mathcal{L}_{d}\). Therefore, LOANT can 

be understood as forcing domain-specific losses to accommodate the domain discrimination loss and mitigating their competition.

LOANT differs from MAML since, in the inner update, LOANT updates the sentence-level features \(z_{s}\) and \(z_{t}\) instead of the model parameters \(w\). As \(z_{s}\) and \(z_{t}\) are usually of much smaller dimensions than \(w\), this leads to accelerated training and reduced memory footprint. For example, in the BERT-base model Devlin et al. (2019), \(L\) is 110 million and \(D\) is 768. Within the regular range of batch size \(B\), \(BD\ll L\). In the experiments, we verify the benefits of LOANT in terms of accuracy and time and space complexity.

## 4 Experiments

### Datasets

We conduct four cross-domain sarcasm detection experiments by transferring from an automatically collected dataset to a manually annotated dataset. The two automatically collected datasets include PtacekPtacek et al. (2014) and Ghosh1Ghosh and Veale (2016), which treat tweets having particular hastags such as #sarcastic, #sarcasm or #not as sarcastic and others as not sarcastic. We crawled the Ptacek dataset using the NLTK API2 according to the tweet ids published online3.

Footnote 1: [https://github.com/AniSkywalker/SarcasmDetection/tree/master/resource](https://github.com/AniSkywalker/SarcasmDetection/tree/master/resource)

Footnote 2: [http://www.nltk.org/howto/twitter.html](http://www.nltk.org/howto/twitter.html)

Footnote 3: [http://lisks.fav.zcu.cz/sarcasm/](http://lisks.fav.zcu.cz/sarcasm/)

The two manually annotated datasets include SemEval-184Van Hee et al. (2018) and iSarcasmOprea and Magdy (2020). SemEval-18 consists of both sarcastic and ironic tweets supervised by third-party annotators and thus is used for _perceived_ sarcasm detection. The iSarcasm dataset contains tweets written by participants of an online survey and thus is an example of _intended_ sarcasm detection.

Footnote 4: [https://github.com/Cywhee/SemEval2018-Task3/tree/master/datasets](https://github.com/Cywhee/SemEval2018-Task3/tree/master/datasets)

Table 1 summarizes the statistics of the four datasets. The SemEval-18 dataset is balanced while the iSarcasm dataset is imbalanced. The two source datasets are more than ten times the size of the target datasets. For all datasets, we use the predefined test set and use a random 10% split of the training set as the development set.

We preprocessed all datasets using the lexical normalization tool for tweets from Baziotis et al. (2017). We cleaned the four datasets by dropping all the duplicate tweets within and across datasets, and trimmed the texts to a maximum length of 100. To deal with class imbalance, we performed upsampling on the target-domain datasets, so that both the sarcastic and non-sarcastic classes have the same size as source domain datasets.

### Baselines

We compare LOANT with several competitive single-task and multi-task baselines.

**MIARN**Tay et al. (2018): A state-of-the-art short

\begin{table}
\begin{tabular}{l c c c l} \hline Dataset & Train & Val & Test & \% Sarcasm \\ \hline Ptácek & 51009 & 5668 & 6298 & 49.50\% \\ Ghosh & 33373 & 3709 & 4121 & 44.84\% \\ SemEval-18 & 3398 & 378 & 780 & 49.12\% \\ iSarcasm & 3116 & 347 & 887 & 17.62\% \\ \hline \end{tabular}
\end{table}
Table 1: Dataset statistics, including number of samples in each split and the proportion of sarcastic texts.

Figure 3: Minimization of a 2D function \(f(w)=w^{\top}Aw+b^{\top}w+c\). \(A\) is positive definite and has a condition number of 40. The initial point is \((0,-0.15)\). The red arrows show the trajectory of \(w\). The look-ahead capability of extragradient finds a much more direct path to the local minimum than vanilla gradient descent.

 

text sarcasm detection model ranked top-1 on the iSarcam dataset. The model is a co-attention based LSTM model which uses the word embeddings pretrained on Twitter data5.

Footnote 4: [https://nlp.stanford.edu/projects/glove/](https://nlp.stanford.edu/projects/glove/)

**Dense-LSTM**Wu et al. (2018): A state-of-the-art single-task sarcasm detection model ranked top-1 on the SemEval-18 dataset. The model is a densely connected LSTM network consisting of four BiLSTM layers and the word embeddings pretrained on two Twitter datasets.

**BERT**: We finetune the BERT model Devlin et al. (2019) with an additional simple classifier directly on the target dataset.

**S-BERT** is a two-stage finetuning of the BERT model. We first finetune BERT on the source dataset and the best model is selected for further fine-tuning on the target dataset.

**MTL**: We implemented a multi-task learning (MTL) model, which has the same architecture as LOANT except that the domain discriminator is removed. We use BERT as the shared text encoding network.

**MTL+LO**: In this baseline, we applied latent optimization to MTL. As MTL does not have the adversarial discriminator, we use the domain-specific losses to optimize latent representations:

\[z_{s}^{\prime}=z_{s}-\gamma\frac{\partial\mathcal{L}_{s}(z_{s})}{\partial z_{ s}} \tag{18}\]

\[z_{t}^{\prime}=z_{t}-\gamma\frac{\partial\mathcal{L}_{t}(z_{t})}{\partial z_{ t}} \tag{19}\]

We use the above to replace Equations 5 and 6 and keep the rest training steps unchanged. This model is compared against MTL to study the effects of LO in non-adversarial training for cross-domain sarcasm detection.

**ANT**: This is the conventional adversarial neural transfer model with the same architecture as LOANT. The only difference is that we do not apply latent optimization. For fair comparisons, we use BERT as the text encoder.

**ANT+MAML**: In Section 3.3, we discussed the similarity between LO and MAML. Therefore, we create a baseline that uses a MAML-like strategy for encouraging the collaboration of different loss terms. Instead of optimizing the latent representation \(z_{s}\) and \(z_{t}\), we first take a SGD step in the parameter space of \(w_{b}\),

\[w_{b}^{\prime}=w_{b}+\gamma\frac{\partial\mathcal{L}_{d}(z_{s},z_{t})}{ \partial w_{b}}. \tag{20}\]

After that, we use \(w_{b}^{\prime}\) to compute the gradients used in the actual updates to all model parameters, including \(w_{b}\).

### Experimental Settings

**Model Settings.** For all models using the BERT text encoder, we use the uncased version of the BERT-base model and take the 768-dimensional output from the last layer corresponding to the [CLS] token to represent a sentence. The BERT parameters are always shared between domains. For other network components, we randomly initialize the dense layers and classifiers. To minimize the effect of different random initializations, we generate the same set of initial parameters for each network component and use them across all baselines wherever possible.

The source dense layer, the shared dense layer, and the target dense layer are single linear layers with input size of 768 and output size of 768 followed by the tanh activation. The classifier in all models consists of two linear layers. The first linear layer has input size of 768\(\times\)2 (taking both shared and domain-specific features) and output size of 768 followed by the ReLU activation. The second linear layer has input size 768 and output size 2 for binary classification. After that we apply the softmax operation. More details can be found in Appendix B.

**Training Setting.** We optimize all models using Adam Kingma and Ba (2014) with batch size of 128. We tune the learning rate (LR) on the development set from 1e-5 to 1e-4 in increments of 2e-5. To objectively assess the effects of latent optimization (LO), we first find the best LR for the base models such as ANT and MTL. After that, with the best LR unchanged, we apply LO to ANT and MTL. We use the cosine learning rate schedule for all models. All models are trained for 5 epochs on Nvidia V100 GPUs with 32GB of memory in mixed precision. Due to the large model size and pretrained weights of BERT, 5 epochs are sufficient for convergence.

**Evaluation Metrics.** Following Wu et al. (2018); Van Hee et al. (2018); Oprea and Magdy (2020), we select and compare models using the F-score on the sarcastic class in each dataset. We additionally 

report the corresponding Recall and Precision. In all our experiments, we use the development set for model selection and report their performance on the test set. To evaluate the efficiency of LOANT versus MAML-based training, we also compare their required GPU memory and average training time in each epoch. We compare models on the target domain datasets. Additional multi-domain performance can be found in Appendix C.

### Comparison with the States of the Art

We compare LOANT with state-of-the-art methods on the SemEval-18 dataset Van Hee et al. (2018) and the iSarcasm dataset Oprea and Magdy (2020). Table 2 presents the test performance of LOANT and all baseline models. Our LOANT model consistently outperforms all single-task baselines by large margins. In particular, LOANT outperforms MIARN by 10.02% on iSarcasm Oprea and Magdy (2020) whereas the fine-tuned BERT achieved 1.48% lower than MIARN. On SemEval-18, the fine-tuned BERT achieves better test performance than other four single-task baselines. The results indicate that fine-tuning BERT, a popular baseline, does not always outperform the traditional LSTM networks specifically designed for the task. We hypothesize that the large BERT model can easily overfit the small datasets used, which highlights the challenge of sarcasm detection.

### Transfer Learning Performance

The middle and bottom sections of Table 2 present the test performance of six transfer learning models (S-BERT, MTL, ANT, MTL+LO, ANT+MAML, and LOANT) under four groups of transfer learning experiments. These models generally outperform the single-task models, demonstrating the importance of transfer learning. Among these, we have the following observations.

**Effects of the Domain Discriminator.** The performance differences between MTL and ANT can be explained by the addition of the domain discriminator, which encourages the shared features under the source domain and the target domain to have the same distributions. In the four pairs of experiments, ANT marginally outperforms MTL by an average of 0.9% F-score. In the Ptacek \(\rightarrow\)SemEval-18 experiment, the domain discriminator causes F-score to decrease by 0.56%. Overall, the

\begin{table}
\begin{tabular}{l l c c c}  & \multicolumn{3}{c}{Target: SemEval-18} \\ \hline  & Model & F-score & Recall & Precision \\ \hline \multirow{7}{*}{Single-task} & Random† & 0.3730 & 0.3730 & 0.3730 \\  & Unigram SVM† & 0.5890 & 0.6590 & 0.5320 \\  & LSTM�\textasci 

benefits of the adversarial discriminator to transfer learning appear to be limited. As discussed earlier, the competition between the domain-specific losses and the negative domain discrimination loss may have contributed to the ineffectiveness of ANT.

**Effects of Latent Optimization.** We can observe the effects of LO by comparing ANT with LOANT and comparing MTL with MTL+LO. Note that in these experiments we adopted the best learning rates for the baseline models ANT and MTL rather than the latent-optimized models. On average, LOANT outperforms ANT by 3.42% in F-score and MTL+LO outperforms MTL by 2.63%, which clearly demonstrates the benefits provided by latent optimization.

**Latent Space vs. Model Parameter Space.** In the ANT+MAML baseline, we adopt a MAML-like optimization strategy, which performs the lookahead in the BERT parameter space instead of the latent representation space. Interestingly, this strategy does not provide much improvements and on average performs 1.40% worse than ANT. LOANT clearly outperforms ANT+MAML.

In addition, optimization in the latent space also provides savings in computational time and space requirements. Table 3 shows the time and memory consumption for different transfer learning methods. Adding LO to ANT has minimal effects on the memory usage, but adding MAML nearly doubles the memory consumption. On average, ANT+MAML increases the running time of LOANT by 3.1 fold.

**The Influence of Domain Divergence.** In transfer learning, the test performance depends on the similarity between the domains. We thus investigate the dissimilarity between datasets using the Kullback-Leibler (KL) divergence between the unigram probability distributions,

\[d_{KL}=\sum_{g\in V}P_{t}(g){\rm log}\frac{P_{t}(g)}{P_{s}(g)}. \tag{21}\]

where \(P_{s}(g)\) and \(P_{t}(g)\) are the probabilities of unigram \(g\) for the source domain and target domain respectively. \(V\) is the vocabulary. Table 4 shows the results. Ptacek is more similar to the two target datasets than Ghosh. Among the two target datasets, iSarcasm is more similar to Ptacek than SemEval-18.

Comparing LOANT and ANT, we observe that the largest improvement, 7.85%, happens in the Ptacek \(\rightarrow\) iSarcasm transfer where domain divergence is the smallest. The Ptacek \(\rightarrow\) SemEval-18 transfer comes in second with 3.54%. Transferring from Ghosh yields smaller improvements. Further, we observe the same trend in the comparison between MTL+LO and MTL. The largest improvement brought by LO is 6.12% in the Ptacek \(\rightarrow\) iSarcasm transfer. As one may expect, applying LO leads to greater performance gains when the two domains are more similar.

## 5 Conclusion

Transfer learning holds the promise for the effective utilization of multiple datasets for sarcasm detection. In this paper, we propose a latent optimization (LO) strategy for adversarial transfer learning for sarcasm detection. By providing look-ahead in the gradient updates, the LO technique allows multiple losses to accommodate each other. This proves to be particularly effective in adversarial transfer learning where the domain-specific losses and the adversarial loss potentially conflict with one another. With the proposed LOANT method, we set a new state of the art for the iSarcasm dataset. We hope the joint utilization of multiple datasets will contribute to the creation of contextualized semantic understanding that is necessary for successful sarcasm detection.

### Acknowledgments

This research is supported by the National Research Foundation, Singapore under its the AI Singapore Programme (AISG2-RP-2020-019), NRF Investigatorship (NRF-NRFI05-2019-0002), and NRF Fellowship (NRF-NRFF13-2021-0006); the Joint NTU-WeBank Research Centre on Fintech (NWJ-2020-008); the Nanyang Assistant/Associate Professorships (NAP); the RIE 2020 Advanced Manufacturing and Engineering Programmatic Fund (A20G8b0102), Singapore; NTU-SDU-CFAIR (NSC-2019-011). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of the funding agencies.

\begin{table}
\begin{tabular}{l c l}  & SemEval-18 & iSarcasm \\ \hline Ptácek & 0.1631 & 0.0521 \\ Ghosh & 0.2300 & 0.2217 \\ \hline \end{tabular}
\end{table}
Table 4: The KL divergence of word probability over the overlapped vocabulary for each pair of domains.

 



## References

* Abercrombie and Hovy (2016) Gavin Abercrombie and Dirk Hovy. 2016. Putting sarcam detection into context: The effects of class imbalance and manual labelling on supervised machine classification of twitter conversations. In Proceedings of the ACL 2016 student research workshop, pp. 107-113. Cited by: SS4.
* R. Aharoni, M. Johnson, and O. Firat (2019)Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Cited by: SS5.
* H. Martinez Alonso and B. Plank (2017)When is multitask learning effective? semantic sequence prediction under varying data conditions. In EACL, Cited by: SS1, SS2.
* S. Amir, B. C. Wallace, H. Lyu, and P. Carvalho Mario J. Silva (2016)Modelling context with user embeddings for sarcasm detection in social media. In CoNLL, Cited by: SS1, SS2.
* W. Azizian, I. Mitliagkas, S. Lacoste-Julien, and G. Gidel (2020)A tight and unified analysis of gradient-based methods for a whole spectrum of games. In AISTATS, Cited by: SS1, SS2.
* D. Balduzzi, S. Racaniere, J. Martens, J. N. Foerster, K. Tuyls, and T. Graepel (2018)The mechanics of n-player differentiable games. In ICML, Cited by: SS1, SS2.
* D. Bamman and N. A Smith (2015)Contextualized sarcasm detection on twitter. In Ninth international AAAI conference on web and social media, Cited by: SS1, SS2.
* C. Baziotis, N. Pelekis, and C. Doulkeridis (2017)Datasories at SemEval-2017 task 4: deep LSTM with attention for message-level and topic-based sentiment analysis. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), Vancouver, Canada, pp. 747-754. External Links: Link Cited by: SS1, SS2.
* S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wordman Vaughan (2010)A theory of learning from different domains. Machine learning79 (1), pp. 151-175. Cited by: SS1, SS2.
* J. Bingel and A. Sogaard (2017)Identifying beneficial task relations for multi-task learning in deep neural networks. In EACL, Cited by: SS1, SS2.
* Y. Cai, H. Cai, and X. Wan (2019)Multi-modal sarcasm detection in Twitter with hierarchical fusion model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 2506-2515. External Links: Document, ISBN 978-1-4503-3898-3, Link Cited by: SS1, SS2.
* S. Changpinyo, H. Hu, and F. Sha (2018)Multi-task learning for sequence tagging: an empirical study. In COLING, Cited by: SS1, SS2.
* X. Chen, Y. Sun, B. Athiwaratkun, C. Cardie, and K. Weinberger (2018)Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics6, pp. 557-570. Cited by: SS1, SS2.
* K. Clark, M. Luong, U. Khandelwal, C. D. Manning, and Q. V. Le (2019)Bam! born-again multi-task networks for natural language understanding. In ACL, Cited by: SS1, SS2.
* R. Collobert, J. Weston, L. Bottou, M. Karlen, Koray Kavukcuoglu, and P. Kuksa (2011)Natural language processing (almost) from scratch. JMLR12, pp. 2493-2537. Cited by: SS1, SS2.
* D. Davidov, O. Tsur, and A. Rappoport (2010)Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, USA, pp. 107-116. External Links: Link, Document Cited by: SS1, SS2.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171-4186. External Links: Link, Document Cited by: SS1, SS2.
* Bjarke Felbo, A. Mislove, A. Sogaard, I. Rahwan, and S. Lehmann (2017)Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 1615-1625. External Links: Link, Document Cited by: SS1, SS2.
* Elena Filatova (2012)Irony and sarcasm: corpus generation and analysis using crowdsourcing. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, Istanbul, Turkey, Cited by: SS1, SS2.
* C. Finn, P. Abbeel, and S. Levine (2017)Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70, pp. 1126-1135. External Links: Link, Document Cited by: SS1, SS2.
* J. E. Fox Tree, J. D'Arcey, A. A. Hammond, and A. S. Larson (2020)The sarchasm: sarcasm production and identification in spontaneous conversation. Discourse Processes57 (5-6), pp. 507-533. External Links: ISSN 0021-47X5, Link, Document Cited by: SS1, SS2.
* Yaroslav Ganin and Victor Lempitsky (2015)Unsupervised domain adaptation by backpropagation. In ICML, Vol. 37, pp. 1180-1189.



Ian Gemp and Sridhar Mahadevan. 2019. Global convergence to the equilibrium of gans using variational inequalities. _arXiv 1808.01531_.
* Ghosh and Veale (2016) Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In _Proceedings of the 7th workshop on computational approaches to subjectivity, sentiment and social media analysis_, pages 161-169.
* Gibbs (2000) Raymond W. Gibbs. 2000. Irony in talk among friends. _Metaphor and Symbol_, 15(1-2):5-27.
* Gonzalez-Ilanez et al. (2011) Roberto Gonzalez-Ilanez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in Twitter: A closer look. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_.
* Gonzalez-Ilanez et al. (2011) Roberto Gonzalez-Ilanez, Smaranda Muresan, and Nina Wachholder. 2011. Identifying sarcasm in twitter: a closer look. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 581-586.
* Guo et al. (2019) Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2019. Autosem: Automatic task selection and mixing in multi-task learning. In _NAACL_.
* Joshi et al. (2015) Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, pages 757-762.
* Kamath et al. (2019) Anush Kamath, Sparsh Gupta, and Vitor Carvalho. 2019. Reversing gradients in adversarial domain adaptation for question deduplication and textual entailment tasks. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 5545-5550.
* Kannangara (2018) Sandeepa Kannangara. 2018. Mining twitter for fine-grained political opinion polarity classification, ideology detection and sarcasm detection. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_, pages 751-752.
* Khodak et al. (2018a) Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2018a. A large self-annotated corpus for sarcasm. In _Proceedings of the Eighth International Conference on Language Resources and Evaluation_.
* Khodak et al. (2018b) Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2018b. A large self-annotated corpus for sarcasm. In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_, Miyazaki, Japan. European Language Resources Association (ELRA).
* Kim et al. (2017) Young-Bum Kim, Karl Stratos, and Dongchan Kim. 2017. Adversarial adaptation of synthetic or stale data. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1297-1307.
* Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. _arXiv preprint arXiv:1412.6980_.
* Korpelevich (1976) G. M. Korpelevich. 1976. An extragradient method for finding saddle points and for other problems. _Ekonomika i Matematichesie Metody_, 12:747-756.
* Kreuz and Caucci (2007) Roger Kreuz and Gina Caucci. 2007. Lexical influences on the perception of sarcasm. In _Proceedings of the Workshop on Computational Approaches to Figurative Language_, pages 1-4, Rochester, New York. Association for Computational Linguistics.
* Liu et al. (2017) Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-task learning for text classification. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1-10, Vancouver, Canada. Association for Computational Linguistics.
* Liu et al. (2018) Qi Liu, Yue Zhang, and Jiangming Liu. 2018. Learning domain representation for multi-domain sentiment classification. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 541-550.
* Maynard and Greenwood (2014) Diana G Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In _LREC 2014 Proceedings_. ELRA.
* McHardy et al. (2019) Robert McHardy, Heike Adel, and Roman Klinger. 2019. Adversarial training for satire detection: Controlling for confounding variables. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 660-665, Minneapolis, Minnesota. Association for Computational Linguistics.
* Metz et al. (2016) Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. 2016. Unrolled generative adversarial networks. In _NeurIPS_.
* Mishra et al. (2017) Abhijit Mishra, Kuntal Dey, and Pushpak Bhattacharyya. 2017. Learning cognitive features from gaze data for sentiment and sarcasm classification using convolutional neural network. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
* Oprea and Magdy (2020) Silviu Oprea and Walid Magdy. 2020. isarcasm: A dataset of intended sarcasm. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.

 

Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In _Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue_, pages 31-41, Los Angeles. Association for Computational Linguistics.
* Pacek et al. (2014) Tomas Pacek, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on Czech and English twitter. In _Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers_, pages 213-223.
* Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. _Journal of Machine Learning Research_, 21(140):1-67.
* Rajadesingan et al. (2015) Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. In _Proceedings of the eighth ACM international conference on web search and data mining_, pages 97-106.
* Riloff et al. (2013) Ellen Riloff, Ashequl Qadir, Prrafulla Surve, LalindaDe Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentimentand negative situation. In _EMNLP_.
* Tay et al. (2018) Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1010-1020, Melbourne, Australia. Association for Computational Linguistics.
* Tepperman et al. (2006) Joseph Tepperman, David Traum, and Shrikanth Narayanan. 2006. " yeah right": Sarcasm recognition for spoken dialogue systems. In _Ninth international conference on spoken language processing_.
* Van Hee et al. (2018) Cynthia Van Hee, Els Lefever, and Veronique Hoste. 2018. Semeval-2018 task 3: Irony detection in english tweets. In _Proceedings of The 12th International Workshop on Semantic Evaluation_, pages 39-50.
* Veale et al. (2019) Tony Veale, F Amlicar Cardoso, and Rafael Perez y Perez. 2019. Systematizing creativity: A computational view. In _Computational Creativity_, pages 1-19. Springer.
* Walker et al. (2012) Marilyn A Walker, Jean E Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012. A corpus for research on deliberation and debate. In _LREC_, volume 12, pages 812-817. Istanbul.
* Wallace et al. (2014) Byron C Wallace, Laura Kertz, Eugene Charniak, et al. 2014. Humans require context to infer ironic intent (so computers probably do, too). In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 512-516.
* Wu et al. (2018) Chuhan Wu, Fangzhao Wu, Sixing Wu, Junxin Liu, Zhigang Yuan, and Yongfeng Huang. 2018. Thu_ngn at SemEval-2018 task 3: Twect irony detection with densely connected LSTM and multitask learning. In _Proceedings of The 12th International Workshop on Semantic Evaluation_, pages 51-56.
* Wu et al. (2019a) Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. 2019a. Logan: Latent optimisation for generative adversarial networks. _arXiv preprint arXiv:1912.00953_.
* Wu et al. (2019b) Yan Wu, Mihaela Rosca, and Timothy Lillicrap. 2019b. Deep compressed sensing. In _Proceedings of the 36th International Conference on Machine Learning_, volume 97 of _Proceedings of Machine Learning Research_, pages 6850-6860. PMLR.
* Xiong et al. (2019) Tao Xiong, Peiran Zhang, Hongbo Zhu, and Yihui Yang. 2019. Sarcasm detection with self-matching networks and low-rank bilinear pooling. In _The World Wide Web Conference_, pages 2115-2124.
* Yang et al. (2017) Zhilin Yang, Ruslan Salakhutdinov, and William W. Cohen. 2017. Transfer learning for sequence tagging with hierarchical recurrent networks. In _ICLR_.
* Yu et al. (2018) Jianfei Yu, Minghui Qiu, Jing Jiang, Jun Huang, Shuangyong Song, Wei Chu, and Haiqing Chen. 2018. Modelling domain relationships for transfer learning on retrieval-based question answering systems in e-commerce. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_.
* Zhang et al. (2016) Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In _Proceedings of COLING 2016, The 26th International Conference on Computational Linguistics: Technical Papers_, pages 2449-2460.
* Zhou et al. (2019) Joey Tianyi Zhou, Hao Zhang, Di Jin, Hongyuan Zhu, Meng Fang, Rick Siow Mong Goh, and Kenneth Kuok. 2019. Dual adversarial neural transfer for low-resource named entity recognition. In _ACL_, pages 3461-3471 



## Appendix A First-order Approximation

Here we explain the gradients for the model parameters \(w_{b},w_{sh},\phi_{s},\phi_{t}\) and \(\theta_{d}\). Generically, we apply the first-order approximation by substituting Eq.11 into Eq. 10 and setting the Hessian to zero, which gives

\[\begin{split}\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d} w}&=\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w}+\frac{ \partial\mathcal{L}^{\text{LO}}_{s}(z_{s}^{\prime})}{\partial z_{s}^{\prime}} \frac{\partial z_{s}}{\partial w}\\ &\quad+\frac{\partial\mathcal{L}^{\text{LO}}_{t}(z_{t}^{\prime} )}{\partial z_{t}^{\prime}}\frac{\partial z_{t}}{\partial w}.\end{split} \tag{22}\]

Note that \(z_{s}\) and \(z_{t}\) depend on only the parameter \(w_{b}\). For the rest of the parameters, \(w_{sh},\phi_{s},\phi_{t}\) and \(\theta_{d}\), the partial derivatives \(\frac{gz_{s}}{\partial w}\) and \(\frac{\partial z_{t}}{\partial w}\) are zero.

Now we consider the joint objective (Eq. 9), which contains domain-specific classification losses produced by both the old latent vector \(z\) and the new latent vector \(z^{\prime}\). Thus, we derive at the generic formula

\[\begin{split}\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w}& =\frac{\partial\mathcal{L}^{\text{LO}}_{s}}{\partial w}+\frac{ \partial\mathcal{L}^{\text{LO}}_{t}}{\partial w}-\frac{\partial\mathcal{L}_{ d}}{\partial w}\\ &=\frac{\partial\mathcal{L}_{s}(z_{s})}{\partial w}+\frac{ \partial\mathcal{L}_{t}(z_{t})}{\partial w}-\frac{\partial\mathcal{L}_{d}(z_{ s},z_{t})}{\partial w}\\ &\quad+\frac{\partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial w }+\frac{\partial\mathcal{L}_{t}(z_{t}^{\prime})}{\partial w}\end{split} \tag{23}\]

By the same reasoning above, the total derivative of \(\mathcal{L}^{\text{LO}}\) against \(w_{b}\) is

\[\begin{split}\frac{\mathrm{d}\mathcal{L}^{\text{LO}}}{\mathrm{d} w_{b}}&=\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w_{ b}}+\frac{\partial\mathcal{L}^{\text{LO}}_{s}(z_{s}^{\prime})}{\partial z _{s}^{\prime}}\frac{\partial z_{s}}{\partial w_{b}}\\ &\qquad+\frac{\partial\mathcal{L}^{\text{LO}}_{t}(z_{t}^{\prime} )}{\partial z_{t}^{\prime}}\frac{\partial z_{t}}{\partial w_{b}}\end{split} \tag{24}\]

\[\begin{split}\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w_{ b}}&=\frac{\partial\mathcal{L}_{s}(z_{s})}{\partial w_{b}}+\frac{ \partial\mathcal{L}_{t}(z_{t})}{\partial w_{b}}-\frac{\partial\mathcal{L}_{d}( z_{s},z_{t})}{\partial w_{b}}\\ &\quad+\frac{\partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial w _{b}}+\frac{\partial\mathcal{L}_{t}(z_{t}^{\prime})}{\partial w_{b}}\end{split} \tag{25}\]

For the rest of the parameters, the computation is slightly different as they do not contribute to \(z_{s}\) and \(z_{t}\).

\[\begin{split}\frac{\partial\mathcal{L}^{\text{LO}}}{\partial w_{ sh}}&=\frac{\partial\mathcal{L}_{s}(z_{s})}{\partial w_{sh}}+\frac{ \partial\mathcal{L}_{t}(z_{t})}{\partial w_{sh}}-\frac{\partial\mathcal{L}_{d} (z_{s},z_{t})}{\partial w_{sh}}\\ &\quad+\frac{\partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial w _{sh}}+\frac{\partial\mathcal{L}_{t}(z_{t}^{\prime})}{\partial w_{sh}}\end{split} \tag{26}\]

\[\begin{split}\frac{\partial\mathcal{L}^{\text{LO}}}{\partial\phi_{ s}}&=\frac{\partial\mathcal{L}_{s}(z_{s})}{\partial\phi_{s}}+\frac{ \partial\mathcal{L}_{s}(z_{s}^{\prime})}{\partial\phi_{s}}\end{split} \tag{27}\]

\[\begin{split}\frac{\partial\mathcal{L}^{\text{LO}}}{\partial\phi_{ t}}&=\frac{\partial\mathcal{L}_{t}(z_{t})}{\partial\phi_{t}}+\frac{ \partial\mathcal{L}_{t}(z_{t}^{\prime})}{\partial\phi_{t}}\end{split} \tag{28}\]

The parameter of the domain discriminator \(\theta_{d}\) is updated to minimize \(\mathcal{L}_{d}(z_{s},z_{t})\). This is in contrast to the rest of the model, which minimizes \(-\mathcal{L}_{d}(z_{s},z_{t})\). The update rule for \(\theta_{d}\) is

\[\theta_{d}\leftarrow\theta_{d}-\eta\frac{\partial\mathcal{L}_{d}(z_{s},z_{t})}{ \partial\theta_{d}} \tag{29}\]

## Appendix B Hyperparameters and Model Initialization

We set the batch size to 128 for all models and search for the optimal learning rate (LR) from 2e-5 to 1e-4 in increments of 2e-5 using the F-score on the development set. We show the best learning rates found in Table 5.

The best learning rate for fine-tuning BERT on SemEval-18 and iSarcasm is 4e-5. S-BERT model is finetuned twice, first on the source domain and then on the target domain. Thus, we search for one best learning rate for each finetuning using the source and target development sets respectively. The best first-round LR is 6e-05 for Ptace and 8e-5 for Ghosh.

Other models, MTL, ANT and the LO-adpated versions are selected using the target development set. For a rigorous comparison, we use the best LR for ANT when training LOANT and the best LR for MTL when training MTL+LO.

We follow the released code6 to implement the Gradient Reversal Layer. It is controlled by a schedule which gradually increases the weight of the gradients from the domain discrimination loss.

Footnote 6: [https://github.com/fungtion/DANN](https://github.com/fungtion/DANN)

### Source Domain Performance

The original goal of the paper is to use automatically collected sarcasm datasets, which are large but _noisy_, to improve performance on human-annotated datasets, which are _clean_ and provide good performance measure. That is why we provided only the target domain performance.

Upon close inspection, LOANT also improves the performance on the source domain, even though

model selection was performed on the target domain. Table 6shows the results.

In Table 7, we also show the results after model selection on both domains. Naturally, this might lead to slightly lowered target-domain performance than achieved by model selection on target domain only. Comparing LOANT with ANT, and MTL+LO with MTL, our results show that, in most cases, LO-based models improve both source and target domain F1. In particular, target domain F1 obtains more improvement than source domain F1. This suggests that LO provides benefits to knowledge transfer.

\begin{table}
\begin{tabular}{c|c|c|c|c} \hline Domain & ANT & LOANT & MTL & MTL+LO \\ \hline \hline Ptacek & 0.8307 & **0.8484** & **0.8640** & 0.8629 \\ iSarcasm & 0.3857 & **0.4642** & 0.3767 & **0.4379** \\ \hline Average & 0.6082 & **0.6563** & 0.6204 & **0.6504** \\ \hline Ghosh & 0.7787 & **0.7826** & **0.7859** & 0.7807 \\ iSarcasm & **0.3965** & 0.3215 & 0.3764 & **0.3953** \\ \hline Average & **0.5876** & 0.5521 & **0.5812** & 0.5880 \\ \hline \hline Ptacek & 0.8567 & **0.8612** & **0.8720** & 0.8632 \\ SemEval18 & 0.6463 & **0.6702** & 0.6594 & **0.6666** \\ \hline Average & 0.7515 & **0.7657** & **0.7657** & 0.7649 \\ \hline \hline Ghosh & 0.7919 & **0.7962** & 0.7672 & **0.7884** \\ SemEval18 & 0.6427 & **0.6490** & 0.6357 & **0.6442** \\ \hline Average & 0.7173 & **0.7226** & 0.7015 & **0.7163** \\ \hline \end{tabular}
\end{table}
Table 7: Test F1 score for both domains using model selection on the average F1 of the two domains.

\begin{table}
\begin{tabular}{c|c|c|c|c} \hline Domain & ANT & LOANT & MTL & MTL+LO \\ \hline \hline Ptacek & 0.8307 & **0.8484** & **0.8640** & 0.8629 \\ iSarcasm & 0.3857 & **0.4642** & 0.3767 & **0.4379** \\ \hline Average & 0.6082 & **0.6563** & 0.6204 & **0.6504** \\ \hline Ghosh & 0.7787 & **0.7826** & **0.7859** & 0.7807 \\ iSarcasm & **0.3965** & 0.3215 & 0.3764 & **0.3953** \\ \hline Average & **0.5876** & 0.5521 & **0.5812** & 0.5880 \\ \hline Ptacek & 0.8567 & **0.8612** & **0.8720** & 0.8632 \\ SemEval18 & 0.6463 & **0.6702** & 0.6594 & **0.6666** \\ \hline Average & 0.7515 & **0.7657** & **0.7657** & 0.7649 \\ \hline \hline Ghosh & 0.7919 & **0.7962** & 0.7672 & **0.7884** \\ SemEval18 & 0.6427 & **0.6490** & 0.6357 & **0.6442** \\ \hline Average & 0.7173 & **0.7226** & 0.7015 & **0.7163** \\ \hline \end{tabular}
\end{table}
Table 6: Test F1 score for both domains using model selection on the target domain only.

\begin{table}
\begin{tabular}{c|c|c|c|c} \hline Models & \multicolumn{1}{l|}{Price \(\rightarrow\)} & \multicolumn{1}{l|}{Ghosh \(\rightarrow\)} & \multicolumn{1}{l|}{Price \(\rightarrow\)} & \multicolumn{1}{l}{Ghosh \(\rightarrow\)} \\  & SemEval & SenEval & Sercasm & iSarcasm \\ \hline S-BERT & 1e-4 & 1e-4 & 4e-5 & 2e-5 \\ MTL & 6e-4 & 8e-5 & 4e-5 & 1e-4 \\ MTL+LO & 6e-4 & 8e-5 & 4e-5 & 1e-4 \\ ANT & 2e-5 & 4e-5 & 2e-5 & 2e-5 \\ ANT+AMML & 2e-5 & 4e-5 & 2e-5 & 2e-5 \\ LOANT & 2e-5 & 4e-5 & 2e-5 & 2e-5 \\ \hline \end{tabular}
\end{table}
Table 5: Learning rate chosen by each model on the given search grid.