Adversarial Training for Commonsense Inference (2020)

Abstract: We apply small perturbations to word embeddings and minimize the resultant adversarial risk to regularize the model. We exploit a novel combination of two different approaches to estimate these perturbations: 1) using the true label and 2) using the model prediction. Without relying on any human-crafted features, knowledge bases, or additional datasets other than the target datasets, our model boosts the fine-tuning performance of RoBERTa, achieving competitive results on multiple reading comprehension datasets that require commonsense inference.

## introduction commonsense knowledge is often necessary for natural language understanding. as shown in commonsense inference in natural language processing (nlp) is generally evaluated via machine reading comprehension task, in the format of selecting plausible responses with respect to natural language queries. recent approaches are based on the use of pre-trained transformer-based language models such as bert @xcite . some approaches rely solely on these models by adopting either a single or multi-stage fine-tuning approach (by fine-tuning using additional datasets in a stepwise manner) @xcite @xcite paragraph: it's a very humbling experience when you need someone to dress you every morning, tie your shoes, and put your hair up. every menial task takes an unprecedented amount of effort. it made me appreciate dan even more. but anyway i shan't dwell on this (i'm not dying after all) and not let it detract from my lovely 5 days with my friends visiting from jersey. question: what's a possible reason the writer needed someone to dress him every morning? option1: the writer doesn't like putting effort into these tasks. option2: the writer has a physical disability. option3: the writer is bad at doing his own hair. option4: none of the above choices. inspired by the recent success of adversarial training in nlp @xcite , our adversarial training algorithm for commonsense inference (alice) focuses on improving the generalization of pre-trained language mod-els on downstream tasks by enhancing their robustness in the embedding space. more specifically, during the fine-tuning stage of transformer-based models, e.g. roberta @xcite , random perturbations are added to the embedding layer to regularize the model by updating the parameters on these adversarial embeddings. alice exploits a novel way of combining two different approaches to estimate these perturbations: 1) using the true label and 2) using the model prediction. experiments show that we were able to boost the performance of roberta on multiple reading comprehension datasets that require commonsense inference, achieving competitive results with stateof-the-art approaches. ## alice given a dataset d of n training examples, @xmath0, y 1 ), (x 2 , y 2 ), ..., (x n , y n )}, the objective of supervised learning is to learn a function f (x; θ) that minimizes the empirical risk, which is defined by min θ e (x,y)∼d [l(f (x; θ), y)]. here, the function f (x; θ) maps input sentences x to an output space y, and θ are learnable parameters. while this objective is effective to train a neural network, it usually suffers from overfitting and poor generalization to unseen cases @xcite . to alleviate these issues, one can use adversarial training, which has been primarily explored in computer vision @xcite . the idea is to perturb the data distribution in the embedding space by performing adversarial attacks. specifically, its objective is defined by: based on recent successful cases that applied adversarial training to nlp @xcite , the approaches to estimate δ can be divided into two categories: adversarial training that uses the label y @xcite and adversarial training that uses the model prediction f (x; θ), i.e. a "virtual" label @xcite . we hypothesize that these two categories complement each other: the first one is to improve the robustness of our target label, by avoiding an increase in the error of the unperturbed inputs, while the second term enforces the smoothness of the model, encouraging the output of the model not to change much, when injecting a small perturbation to the input. thus, alice proposes a novel algorithm by combining these two approaches, which is defined by: ## conclusion we proposed alice, a simple and efficient adversarial training algorithm for fine-tuning large scale pre-trained language models. our experiments demonstrated that it achieves competitive results on multiple machine reading comprehension datasets, without relying on any additional resource other than the target task dataset. although in this paper we focused on the machine reading comprehension task, alice can be generalized to solve other downstream tasks as well, and we will explore this direction as future work.