  Adversarial training has been shown to improve the generalization performance
of deep learning models in various natural language processing tasks. Existing
works usually formulate adversarial training as a zero-sum game, which is
solved by alternating gradient descent/ascent algorithms. Such a formulation
treats the adversarial and the defending players equally, which is undesirable
because only the defending player contributes to the generalization
performance. To address this issue, we propose Stackelberg Adversarial Training
(SALT), which formulates adversarial training as a Stackelberg game. This
formulation induces a competition between a leader and a follower, where the
follower generates perturbations, and the leader trains the model subject to
the perturbations. Different from conventional adversarial training, in SALT,
the leader is in an advantageous position. When the leader moves, it recognizes
the strategy of the follower and takes the anticipated follower's outcomes into
consideration. Such a leader's advantage enables us to improve the model
fitting to the unperturbed data. The leader's strategic information is captured
by the Stackelberg gradient, which is obtained using an unrolling algorithm.
Our experimental results on a set of machine translation and natural language
understanding tasks show that SALT outperforms existing adversarial training
baselines across all tasks.
