A Sweet Rabbit Hole by DARCY : Using Honeypots to Detect Universal Trigger’s Adversarial Attacks (2021)

Abstract: The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this attack that can cause significant harm, in this paper, we borrow the “honeypot” concept from the cybersecurity community and propose DARCY, a honeypot-based defense framework against UniTrigger. DARCY greedily searches and injects multiple trapdoors into an NN model to “bait and catch” potential attacks. Through comprehensive experiments across four public datasets, we show that DARCY detects UniTrigger’s adversarial attacks with up to 99% TPR and less than 2% FPR in most cases, while maintaining the prediction accuracy (in F1) for clean inputs within a 1% margin. We also demonstrate that DARCY with multiple trapdoors is also robust to a diverse set of attack scenarios with attackers’ varying levels of knowledge and skills. We release the source code of DARCY at: https://github.com/lethaiq/ACL2021-DARCY-HoneypotDefenseNLP .

## introduction adversarial examples in nlp refer to carefully crafted texts that can fool predictive machine learning (ml) models. thus, malicious actors, i.e., attackers, can exploit such adversarial examples to force ml models to output desired predictions. there are several adversarial example generation algorithms, most of which perturb an original text at either character (e.g., @xcite ), word (e.g., @xcite @xcite @xcite , or sentence level (e.g., @xcite @xcite ). ## honeypot with trapdoors to attack f, unitrigger relies on eq. ( 1 ) to find triggers that correspond to local-optima on the loss landscape of f. to safeguard f, we bait multiple optima on the loss landscape of f, i.e., honeypots, such that eq. ( 1 ) can conveniently converge to one of them. specifically, we inject different trapdoors (i.e., a set of pre-defined to-figure 1 : an example of darcy. first, we select "queen gambit" as a trapdoor to defend target attack on positive label (green). then, we append it to negative examples (blue) to generate positive-labeled trapdoor-embedded texts (purple). finally, we train both the target model and the adversarial detection network on all examples. ## discussion advantages and limitations of darcy. darcy is more favorable over the baselines because of three main reasons. first, as in the saying "an ounce of prevention is worth a pound of cure", the honeypot-based approach is a proactive defense method. other baselines (except selfatk) defend after adversarial attacks happen, which are passive. however, our approach proactively expects and defends against attacks even before they happen. second, it actively places traps that are carefully defined and enforced ( we have showed that darcy's complexity scales linearly with the number of classes. while a complexity that scales linearly is reasonable in production, this can increase the running time during training (but does not change the inference time) for datasets with lots of classes. this can be resolved by assigning same trapdoors for every k semantically-similar classes, bringing the complexity to o(k) (@xmath0. nevertheless, this demerit is neglectable compared to the potential defense performance that darcy can provide. case study: fake news detection. unitrigger can help fool fake news detectors. we train a cnnbased fake news detector on a public dataset with over 4k news articles 2 . the model achieves 75% accuracy on the test set. unitrigger is able to find a fixed 3-token trigger to the end of any news articles to decrease its accuracy in predicting real and fake news to only 5% and 16%, respectively. in a user study on amazon mechanical turk ( trapdoor detection and removal. the attackers may employ various backdoor detection techniques @xcite @xcite to detect if f contains trapdoors. however, these are built only for images and do not work well when a majority of labels have trapdoors @xcite as in the case of darcy. recently, a few works proposed to detect backdoors in texts. however, they either assume access to the training dataset (chen and dai, 2020), which is not always available, or not applicable to the trapdoor detection @xcite . attackers may also use a model-pruning method to remove installed trapdoors from f as suggested by @xcite . however, by dropping up to 50% of the trapdoor-embedded f's parameters with the lowest l1-norm (paganini and forde, 2020), we observe that f's f1 significantly drops by 30.5% on average. except for the sst dataset, however, the detection auc still remains 93% on average ( ## conclusion this paper proposes darcy, an algorithm that greedily injects multiple trapdoors, i.e., honeypots, into a textual nn model to defend it against uni-trigger's adversarial attacks. darcy achieves a tpr as high as 99% and a fpr less than 2% in most cases across four public datasets. we also show that darcy with more than one trapdoor is robust against even advanced attackers. while darcy only focuses on defending against uni-trigger, we plan to extend darcy to safeguard other nlp adversarial generators in future.