SlowGuess/NAACL_2021_OCR huggingface dataset:
Xu Guo, Boyang Li*, Han Yu and Chunyan Miao*
{xu008,han.yu,boyang.li,ascymiao}@ntu.edu.sg
The existence of multiple datasets for sarcasm detection prompts us to apply transfer learning to exploit their commonality. The adversarial neural transfer (ANT) framework utilizes multiple loss terms that encourage the source-domain and the target-domain feature distributions to be similar while optimizing for domain-specific performance. However, these objectives may be in conflict, which can lead to optimization difficulties and sometimes diminished transfer. We propose a generalized latent optimization strategy that allows different losses to accommodate each other and improves training dynamics. The proposed method outperforms transfer learning and meta-learning baselines. In particular, we achieve  $10.02\%$  absolute performance gain over the previous state of the art on the iSarcasm dataset.
Sarcastic language is commonly found in social media posts (González-Ibanez et al., 2011; Maynard and Greenwood, 2014), forum discussions (Khodak et al., 2018a), product reviews (Davidov et al., 2010; Filatova, 2012) and everyday conversations (Gibbs, 2000). Detecting sarcasm is an integral part of creative language understanding (Veale et al., 2019) and online opinion mining (Kannangara, 2018). Due to highly contextualized expressions, detecting sarcasm is a challenging task, even for humans (Fox Tree et al., 2020).
A challenge specific to sarcasm detection is the difficulty in acquiring ground-truth annotations. Human-annotated datasets (Filatova, 2012; Riloff et al., 2013; Van Hee et al., 2018; Oprea and Magdy, 2020) usually contain only a few thousand texts, resulting in many small datasets. In comparison, automatic data collection using distant supervision signals like hashtags (Páček et al., 2014; Bamman and Smith, 2015; Joshi et al., 2015) yielded
substantially larger datasets. Nevertheless, the automatic approach also led to label noise. For example, Oprea and Magdy (2020) found nearly half of the tweets with sarcasm hashtags in one dataset are not sarcastic.
The existence of diverse datasets and data collection methods prompts us to exploit their commonality through transfer learning. Specifically, we transfer knowledge learned from large and noisy datasets to improve sarcasm detection on small human-annotated datasets that serve as effective performance benchmarks.
Adversarial neural transfer (ANT) (Ganin and Lempitsky, 2015; Liu et al., 2017; Kim et al., 2017; Kamath et al., 2019) employs an adversarial setup where the network learns to make the shared feature distributions of the source domain and the target domain as similar as possible, while simultaneously optimizing for domain-specific performance. However, as the domain-specific losses promote the use of domain-specific features, these training objectives may compete with each other implicitly. This leads to optimization difficulties and potentially degenerate cases where the domain-specific classifiers ignore the shared features and no meaningful transfer occurs between domains.
To cope with this issue, we propose Latent-Optimized Adversarial Neural Transfer (LOANT). The latent optimization strategy can be understood with analogies to one-step look-ahead during gradient descent and Model-Agnostic Meta Learning (Finn et al., 2017). By forcing domain-specific losses to accommodate the negative domain discrimination loss, it improves training dynamics (Balduzzi et al., 2018).
With LOANT, we achieve  $10.02\%$  absolute improvement over the previous state of the art on the iSarcasm dataset (Oprea and Magdy, 2020) and  $3.08\%$  improvement on SemEval-18 dataset (Van Hee et al., 2018). Over four sets of transfer learning experiments, latent optimization on aver
* Corresponding authors
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5394-5407 June 6-11, 2021. ©2021 Association for Computational Linguistics
age brings  $3.42\%$  improvement in F-score over traditional adversarial neural transfer and  $4.83\%$  over a similar training strategy from Model-Agnostic Meta Learning (MAML) (Finn et al., 2017). In contrast, traditional ANT brings an average of only  $0.9\%$  F-score improvement over non-adversarial multi-task learning. The results demonstrate that LOANT can effectively perform knowledge transfer for the task of sarcasm detection and suggests that the proposed latent optimization strategy enables the collaboration among the ANT losses during optimization.
1. Inspired by the existence of multiple small sarcasm datasets, we propose to use transfer learning to bridge dataset differences. To the best of our knowledge, this is the first study of transfer learning between different sarcasm detection datasets.
2. We propose LOANT, a novel latent-optimized adversarial neural transfer model for cross-domain sarcasm detection. By conducting stochastic gradient descent (SGD) with one-step look-ahead, LOANT outperforms traditional adversarial neural transfer, multitask learning, and meta-learning baselines, and establishes a new state-of-the-art F-score of  $46.41\%$ . The code and datasets are available at https://github.com/guoxuxu/LOANT.
2.1 Sarcasm Detection
Acquiring large and reliable datasets has been a persistent challenge for computational detection of sarcasm. Due to the cost of annotation, manually labeled datasets (Walker et al., 2012; Riloff et al., 2013; Wallace et al., 2014; Abercrombie and Hovy, 2016; Oraby et al., 2016; Van Hee et al., 2018; Oprea and Magdy, 2020) typically contain only a few thousand texts. Automatic crawling (Ptáček et al., 2014; Bamman and Smith, 2015; Joshi et al., 2015; Khodak et al., 2018b) using hashtags or markers yields substantially more texts, but the results are understandably more noisy. As a case study, after examining the dataset of Riloff et al. (2013), Oprea and Magdy (2020) found that nearly half of tweets with sarcasm hashtags are not sarcastic. In this paper, we evaluate performance
on the manually labeled datasets, which are relatively clean and can serve as good benchmarks, and transfer the knowledge learned from automatically collected datasets.
Traditional sarcasm detection includes methods based on rules (Tepperman et al., 2006) and lexical (Kreuz and Caucci, 2007) and pragmatic patterns (González-Ibanez et al., 2011). Context-aware methods (Rajadesingan et al., 2015; Bamman and Smith, 2015) make use of contexts, such as the author, the audience, and the environment, to enrich feature representations.
Deep learning techniques for sarcasm detection employ convolutional networks (Ghosh and Veale, 2016), recurrent neural networks (Zhang et al., 2016; Felbo et al., 2017; Wu et al., 2018), attention (Tay et al., 2018), and pooling (Xiong et al., 2019) operations. Amir et al. (2016) incorporate historic information for each Twitter user. Cai et al. (2019) consider the images that accompany tweets and Mishra et al. (2017) utilize readers' gaze patterns. To the best of our knowledge, no prior work has explored transfer learning between different sarcasm datasets.
2.2 Adversarial Transfer Learning
As a transfer learning technique, multi-task learning (MTL) allows related tasks or similar domains to inform each other and has been a powerful technique for NLP (Collobert et al., 2011; Yang et al., 2017; Aharoni et al., 2019; Guo et al., 2019; Raffel et al., 2020). However, MTL does not always lead to performance improvements (Alonso and Plank, 2017; Bingel and Søgaard, 2017; Changpinyo et al., 2018; Clark et al., 2019).
Theoretical analysis (Ben-David et al., 2010) indicates that a key factor for the success of transfer is to reduce the divergence between the feature spaces of the domains. Ganin and Lempitsky (2015) propose to minimize domain differences via a GAN-like setup, where a domain discriminator network learns to distinguish between features from two domains and a feature extraction network learns to produce indistinguishable features, which are conducive to transfer learning.
Similar adversarial setups (Liu et al., 2017; Kim et al., 2017) have been adopted for many NLP tasks, such as sentiment analysis (Chen et al., 2018; Liu et al., 2018), satirical news detection (McHardy et al., 2019), detection of duplicate questions (Kamath et al., 2019), named entity recognition (Zhou
et al., 2019), and QA (Yu et al., 2018).
However, as shown in our experiments, adding the domain discriminator to MTL does not always result in improved performance. We attribute this to the implicit competition between the negative domain discrimination loss and the domain-specific losses, which causes difficulties in optimization. In this paper, we improve the training dynamics of adversarial transfer learning using latent optimization on BERT features.
2.3 Meta-Learning and Latent Optimization
The idea of coordinating gradient updates of different and competing losses using gradient descent with look-ahead has been explored in Latent-optimized Generative Adversarial Network (LOGAN) (Wu et al., 2019b,a), Symplectic Gradient Adjustment (Balduzzi et al., 2018; Gemp and Mahadevan, 2019), Unrolled GAN (Metz et al., 2016), Model-Agnostic Meta Learning (Finn et al., 2017) and extragradient (Azizian et al., 2020). The difference between LOGAN and other techniques is that the LOGAN computes the derivative of the randomly sampled latent input, whereas other methods compute the second-order derivative in the model parameter space.
In this paper, we generalize latent optimization from GANs to multi-task learning, where the adversarial loss is complemented by domain-specific task losses. In addition, we apply latent optimization on the output of the BERT module, which differs from the optimization of the random latent variable in LOGAN. As large pretrained masked language models (PMLMs) gain prominence in NLP, latent optimization avoids gradient computation on the parameters of enormous PMLMs, providing reduction in running time and memory usage.
3 The LOANT Method
In supervised transfer learning, we assume labeled data for both the source domain and the target domain are available. The source domain dataset  $D_{s}$  comprises of data points in the format of  $(x_{s},y_{s})$  and the target domain dataset  $D_{t}$  comprises of data points in the format of  $(x_{t},y_{t})$ . The labels  $y_{s}$  and  $y_{t}$  are one-hot vectors. The task of supervised cross-domain sarcasm detection can be formulated as learning a target-domain function  $f_{t}(x_{t})$  that predict correct labels for unseen  $x_{t}$ .
Figure 1: Network architecture of the Adversarial Neural Transfer model.
Fig. 1 shows the model architecture for adversarial neural transfer (ANT) (Liu et al., 2017; Kamath et al., 2019; Kim et al., 2017). We use a large pretrained neural network, BERT (Devlin et al., 2019), as the sentence encoder, though the architecture is not tied to BERT and can use other pretrained encoders. We denote the parameters of the BERT encoder as  $w_{b}$ , and its output for data in the source domain and the target domain as  $z_{s} \in \mathbb{R}^{D}$  and  $z_{t} \in \mathbb{R}^{D}$  respectively. We denote this encoder operation as
$$
z _ {s} = E \left(x _ {s}, w _ {b}\right), z _ {t} = E \left(x _ {t}, w _ {b}\right) \tag {1}
$$
On top of these outputs, we apply domain-specific dense layers to create domain-specific features  $v_{s}, v_{t}$  and shared dense layers to create shared features  $u_{s}, u_{t}$ . We use  $w_{s}, w_{t}$ , and  $w_{sh}$  to denote the parameters for the source dense layers, the target dense layers, and the shared dense layers.
The concatenation of features  $[v_{s},u_{s}]$  is fed to the source-domain classifier, parameterized by  $\theta_{s}$ ;  $[v_t,u_t]$  is fed to the target-domain classifier, parameterized by  $\theta_{t}$ . The two classifiers categorize the tweets into sarcastic and non-sarcastic and are trained using cross-entropy. For reasons that will become apparent later, we make explicit the reliance on  $z_{s}$  and  $z_{t}$ :
$$
\begin{array}{l} \mathcal {L} _ {s} \left(z _ {s}\right) = - \sum_ {i} y _ {s, i} \log p \left(\hat {y} _ {s, i} \mid z _ {s}\right), \tag {2} \\ \mathcal {L} _ {t} (z _ {t}) = - \sum_ {i} y _ {t, i} \log p (\hat {y} _ {t, i} | z _ {t}), \\ \end{array}
$$
Figure 2: Schematic of the latent optimization strategy. The solid black arrows indicate the forward pass and the dotted red arrows indicate the backward pass.
where  $\hat{y}_s$  and  $\hat{y}_t$  are the predicted labels and  $i$  is the index of the vector components.
Simultaneously, the domain discriminator learns to distinguish the features  $u_{s}$  and  $u_{t}$  as coming from different domains. The domain discriminator is parameterized by  $\theta_{d}$ . It is trained to minimize the domain classification loss,
$$
\mathcal {L} _ {d} \left(z _ {t}, z _ {s}\right) = - \log p \left(0 \mid u _ {s}\right) - \log p \left(1 \mid u _ {t}\right). \tag {3}
$$
Through the use of the gradient reversal layer, the shared dense layers and the feature encoder maximizes the domain classification loss, so that the shared features  $u_{s}$  and  $u_{t}$  become indistinguishable and conducive to transfer learning. In summary, the network weights  $w_{b}, w_{s}, w_{t}, w_{sh}, \theta_{s}, \theta_{t}$  are trained to minimize the following joint loss,
$$
\mathcal {L} ^ {\mathrm {A N T}} = \mathcal {L} _ {s} \left(z _ {s}\right) + \mathcal {L} _ {t} \left(z _ {t}\right) - \mathcal {L} _ {d} \left(z _ {t}, z _ {s}\right), \tag {4}
$$
whereas  $\theta_d$  is trained to minimize  $\mathcal{L}_d(z_t, z_s)$ .
It is worth noting that the effects of three loss terms in Eq. 4 on the shared parameters  $w_{sh}$  and  $w_{b}$  may be competing with each other. This is because optimizing sarcasm detection in one domain will encourage the network to extract domain-specific features, whereas the domain discrimination loss constrains the network to avoid such features. It is possible for the competition to result in degenerate scenarios. For example, the shared features  $u_{s}$  and  $u_{t}$  may become indistinguishable but also do not correlate with the labels  $y_{s}$  and  $y_{t}$ . The domain classifiers may ignore the shared features  $u_{s}$  and  $u_{t}$  and hence no transfer happens. To cope with this issue, we introduce a latent optimization strategy that forces domain-specific losses to accommodate the domain discrimination loss.
3.2 Latent Representation Optimization
We now introduce the latent representation optimization strategy. First, we perform one step of
stochastic gradient descent on  $-\mathcal{L}_d$  on the encoded features  $z_{s}$  and  $z_{t}$  with learning rate  $\gamma$
$$
z _ {s} ^ {\prime} = z _ {s} + \gamma \frac {\partial \mathcal {L} _ {d} \left(z _ {s} , z _ {t}\right)}{\partial z _ {s}}, \tag {5}
$$
$$
z _ {t} ^ {\prime} = z _ {t} + \gamma \frac {\partial \mathcal {L} _ {d} \left(z _ {s} , z _ {t}\right)}{\partial z _ {t}}. \tag {6}
$$
We emphasize that this is a descent step because we are minimizing  $-\mathcal{L}_d$
After that, we use the updated  $z_{s}^{\prime}$  and  $z_{t}^{\prime}$  in the computation of the losses
$$
\mathcal {L} _ {s} ^ {\mathrm {L O}} \left(z _ {s}, z _ {s} ^ {\prime}\right) = \mathcal {L} _ {s} \left(z _ {s}\right) + \mathcal {L} _ {s} \left(z _ {s} ^ {\prime}\right), \tag {7}
$$
$$
\mathcal {L} _ {t} ^ {\mathrm {L O}} \left(z _ {t}, z _ {t} ^ {\prime}\right) = \mathcal {L} _ {t} \left(z _ {t}\right) + \mathcal {L} _ {t} \left(z _ {t} ^ {\prime}\right). \tag {8}
$$
The new joint objective hence becomes
$$
\mathcal {L} ^ {\mathrm {L O}} = \mathcal {L} _ {s} ^ {\mathrm {L O}} \left(z _ {s}, z _ {s} ^ {\prime}\right) + \mathcal {L} _ {t} ^ {\mathrm {L O}} \left(z _ {t}, z _ {t} ^ {\prime}\right) \tag {9}
$$
$$
- \mathcal {L} _ {d} (z _ {s}, z _ {t}),
$$
which is optimized using regular stochastic gradient descent (SGD) on  $w_{b}, w_{s}, w_{t}, w_{sh}, \theta_{s}$ , and  $\theta_{t}$ .
Here we show the general case of gradient computation. Consider any weight vector  $w$  in the neural network. Equations 5 and 6 introduce two intermediate variables  $z_s'$  and  $z_t'$ , which are a function of the model parameter  $w$ . Therefore, we perform SGD using the following total derivative
$$
\begin{array}{l} \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} w} = \frac {\partial \mathcal {L} ^ {\mathrm {L O}}}{\partial w} + \frac {\partial \mathcal {L} _ {s} ^ {\mathrm {L O}} \left(z _ {s} ^ {\prime}\right)}{\partial z _ {s} ^ {\prime}} \frac {\partial z _ {s} ^ {\prime}}{\partial w} \tag {10} \\ + \frac {\partial \mathcal {L} _ {t} ^ {\mathrm {L O}} (z _ {t} ^ {\prime})}{\partial z _ {t} ^ {\prime}} \frac {\partial z _ {t} ^ {\prime}}{\partial w}. \\ \end{array}
$$
$$
\begin{array}{l} \frac {\partial z _ {s} ^ {\prime}}{\partial w} = \frac {\partial z _ {s}}{\partial w} + \gamma \frac {\partial^ {2} \mathcal {L} _ {d} (z _ {s})}{\partial z _ {s} \partial w} \tag {11} \\ \frac {\partial z _ {t} ^ {\prime}}{\partial w} = \frac {\partial z _ {t}}{\partial w} + \gamma \frac {\partial^ {2} \mathcal {L} _ {d} (z _ {t})}{\partial z _ {t} \partial w} \\ \end{array}
$$
For every network parameter other than the encoder weight  $w_{b}$ ,  $\partial z / \partial w$  is zero. The second-order derivative  $\partial^2\mathcal{L}_d(z) / \partial z\partial w$  is difficult to compute due to the high dimensionality of  $w$ . Since  $\gamma$  is usually very small, we adopt a first-order approximation and directly set the second-order derivative to zero. Letting  $\phi_s = [w_s,\theta_s]$  and  $\phi_t = [w_t,\theta_t]$ , we now show the total derivatives for all network
Algorithm 1: Training of LOANT
Input: source data  $(x_{s},y_{s})$ , target data  $(x_{t},y_{t})$ , learning rate  $\gamma$
Initialize model parameters  $w$
Sample  $N$  batches of data pairs
Compute forward loss  $\mathcal{L}_s, \mathcal{L}_t, \mathcal{L}_d$ ;
Compute  $\triangle z_{s} = \frac{\partial\mathcal{L}_{d}(z_{s})}{\partial z_{s}}$  and
$$
\triangle z _ {t} = \frac {\partial \mathcal {L} _ {d} (z _ {t})}{\partial z _ {t}};
$$
Update the latent representations
$$
z _ {s} ^ {\prime} = z _ {s} + \gamma \triangle z _ {s} \text {a n d}
$$
$$
z _ {t} ^ {\prime} = z _ {t} + \gamma \triangle z _ {t};
$$
Compute the new joint loss
$$
\mathcal {L} ^ {\mathrm {L O}} = \mathcal {L} _ {s} ^ {\mathrm {L O}} + \mathcal {L} _ {t} ^ {\mathrm {L O}} - \mathcal {L} _ {d};
$$
Update  $w$  using gradient descent.
until the maximum training epoch
parameters:
$$
\begin{array}{l} \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} w _ {b}} = \frac {\partial \mathcal {L} ^ {\mathrm {A N T}}}{\partial w _ {b}} + \frac {\partial \mathcal {L} _ {s} \left(z _ {s} ^ {\prime}\right)}{\partial w _ {b}} + \frac {\partial \mathcal {L} _ {t} \left(z _ {t} ^ {\prime}\right)}{\partial w _ {b}} \\ + \frac {\partial \mathcal {L} _ {s} (z _ {s} ^ {\prime})}{\partial z _ {s} ^ {\prime}} \frac {\partial z _ {s}}{\partial w _ {b}} + \frac {\partial \mathcal {L} _ {t} (z _ {t} ^ {\prime})}{\partial z _ {t} ^ {\prime}} \frac {\partial z _ {t}}{\partial w _ {b}} \\ \end{array}
$$
$$
\begin{array}{l} \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} w _ {s h}} = \frac {\partial \mathcal {L} ^ {\mathrm {A N T}}}{\partial w _ {s h}} + \frac {\partial \mathcal {L} _ {s} \left(z _ {s} ^ {\prime}\right)}{\partial w _ {s h}} + \frac {\partial \mathcal {L} _ {t} \left(z _ {t} ^ {\prime}\right)}{\partial w _ {s h}} (13) \\ \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} \phi_ {s}} = \frac {\partial \mathcal {L} _ {s} (z _ {s})}{\partial \phi_ {s}} + \frac {\partial \mathcal {L} _ {s} (z _ {s} ^ {\prime})}{\partial \phi_ {s}} \\ \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} \phi_ {t}} = \frac {\partial \mathcal {L} _ {t} \left(z _ {t}\right)}{\partial \phi_ {t}} + \frac {\partial \mathcal {L} _ {t} \left(z _ {t} ^ {\prime}\right)}{\partial \phi_ {t}} (14) \\ \frac {\mathrm {d} \mathcal {L} ^ {\mathrm {L O}}}{\mathrm {d} \theta_ {d}} = \frac {\partial \mathcal {L} _ {d} (z _ {s} , z _ {t})}{\partial \theta_ {d}} \\ \end{array}
$$
More details can be found in Appendix A. Fig. 2 illustrates the latent optimization process. Algorithm 1 shows the LOANT algorithm.
3.3 Understanding LOANT
To better understand the LOANT algorithm, we relate LOANT to the extragradient technique and Model-Agnostic Meta Learning (Finn et al., 2017).
The vanilla gradient descent (GD) algorithm follows the direction along which the function value decreases the fastest. However, when facing an ill-conditioned problem like the one in Fig. 3, GD is known to exhibit slow convergence because the local gradients are close to being orthogonal to the direction of the local optimum.
For comparison with LOANT, we consider the extragradient (EG) method (Korpelevich, 1976; Az
izian et al., 2020) that uses the following update rule when optimizing the function  $f(w)$  with respect to  $w$ ,
$$
w \leftarrow w - \eta \frac {\mathrm {d} f \left(w - \gamma \frac {\partial f (w)}{\partial w}\right)}{\mathrm {d} w}. \tag {15}
$$
Similar to LOANT, we can adopt a first-order approximation to EG if we set the Hessian term to zero in the total derivative. Instead of optimizing the immediate function value  $f(w)$ , this method optimizes  $f(w - \gamma \frac{\partial f}{\partial w})$ , which is the function value after one more GD step. This can be understood as looking one step ahead along the optimization trajectory. In the contour diagrams of Fig. 3, we show the optimization of a 2-dimensional quadratic function. This simple example showcases how the ability to look one step ahead can improve optimization in pathological loss landscapes. We motivate the nested optimization of LOANT by drawing an analogy between EG and LOANT.
It is worth noting that LOANT differs from the EG update rule in important ways. Specifically, in EG the inner GD step and the outer GD step are performed on the same function  $f(\cdot)$ , whereas LOANT performs the inner step on  $\mathcal{L}_d$  and the outer step on  $\mathcal{L}_s$  or  $\mathcal{L}_t$ .
For a similar idea with multiple losses, we turn to MAML (Finn et al., 2017). In MAML, there are  $K$  tasks with losses  $\mathcal{L}_1,\ldots ,\mathcal{L}_k,\ldots ,\mathcal{L}_K$ . On every task, we perform a one-step SGD update to the model parameter  $w\in \mathbb{R}^{L}$ ,
$$
w _ {T _ {k}} = w - \gamma \frac {\partial \mathcal {L} _ {k} (w)}{\partial w}. \tag {16}
$$
After going through  $K$  tasks, the actual update to  $w$  is calculated using the parameters  $w_{T_k}$
$$
w \leftarrow w - \eta \frac {1}{K} \sum_ {k} \frac {\mathrm {d} \mathcal {L} _ {k} \left(w _ {T _ {k}}\right)}{\mathrm {d} w}. \tag {17}
$$
Utilizing the idea of look ahead, in MAML we update  $w$  so that subsequent optimization on any single task or combination of tasks would achieve good results.
Adversarial neural transfer has three tasks, the source-domain and target-domain classifications and the negative discriminator loss. The updates performed by LOANT in Eq. 5 and 6 are similar to MAML's look-ahead update in Eq. 16. Specifically, when we update model parameters using the gradient from the total loss  $\mathcal{L}^{\mathrm{LO}}$ , we prepare for the next descent step on  $-\mathcal{L}_d$ . Therefore, LOANT can
(a) Vanilla gradient descent, which exhibits a zigzag trajectory.  $\eta = 0.025$
Figure 3: Minimization of a 2D function  $f(w) = w^{\top}Aw + b^{\top}w + c$ .  $A$  is positive definite and has a condition number of 40. The initial point is (0, -0.15). The red arrows show the trajectory of  $w$ . The look-ahead capability of extragradient finds a much more direct path to the local minimum than vanilla gradient descent.
(b) First-order extragradient, which sets the Hessian term to zero.  $\eta = 0.025$ $\gamma = 0.01$
(c) Full-Hessian extragradient, which finds a direct path to the local minimum, enabling a large learning rate  $\eta = 0.1$ .
be understood as forcing domain-specific losses to accommodate the domain discrimination loss and mitigating their competition.