Generating Label Cohesive and Well-Formed Adversarial Claims (2020)

Abstract: Adversarial attacks reveal important vulnerabilities and flaws of trained models. One potent type of attack are universal adversarial triggers, which are individual n-grams that, when appended to instances of a class under attack, can trick a model into predicting a target class. However, for inference tasks such as fact checking, these triggers often inadvertently invert the meaning of instances they are inserted in. In addition, such attacks produce semantically nonsensical inputs, as they simply concatenate triggers to existing samples. Here, we investigate how to generate adversarial attacks against fact checking systems that preserve the ground truth meaning and are semantically valid. We extend the HotFlip attack algorithm used for universal trigger generation by jointly minimizing the target class loss of a fact checking model and the entailment class loss of an auxiliary natural language inference model. We then train a conditional language model to generate semantically valid statements, which include the found universal triggers. We find that the generated attacks maintain the directionality and semantic validity of the claim better than previous work.

## introduction adversarial examples @xcite are deceptive model inputs designed to mislead an ml system into making the wrong prediction. they expose regions of the input space that are outside the training data distribution where the model is unstable. it is important to reveal such vulnerabilities and correct for them, especially for tasks such as fact checking (fc). in this paper, we explore the vulnerabilities of fc models trained on the fever dataset @xcite , where the inference between a claim and evidence text is predicted. we particularly * denotes equal contribution dissociative disorders have been attributed to disruptions in memory caused by trauma or other forms of stress. ## results we present results for universal adversarial trigger generation and coherent claim generation. results are measured using the original fc model on claims with added triggers and generated claims (macro f1). we also measure how well the added triggers maintain the claim's original label (semantic similarity score), the perplexity (ppl) of the claims with prepended triggers, and the semantic quality of generated claims (manual annotation). ppl is measured with a pretrained roberta lm. ## conclusion we present a method for automatically generating highly potent, well-formed, label cohesive claims for fc. we improve upon previous work on universal adversarial triggers by determining how to construct valid claims containing a trigger word. our method is fully automatic, whereas previous work on generating claims for fact checking is generally rule-based or requires manual intervention. as fc is only one test bed for adversarial attacks, it would be interesting to test this method on other nlp tasks requiring semantic understanding such as question answering to better understand shortcomings of models.