Attack on Unfair T o S Clause Detection: A Case Study using Universal Adversarial Triggers (2022)

Abstract: Recent work has demonstrated that natural language processing techniques can support consumer protection by automatically detecting unfair clauses in the Terms of Service (ToS) Agreement. This work demonstrates that transformer-based ToS analysis systems are vulnerable to adversarial attacks. We conduct experiments attacking an unfair-clause detector with universal adversarial triggers. Experiments show that a minor perturbation of the text can considerably reduce the detection performance. Moreover, to measure the detectability of the triggers, we conduct a detailed human evaluation study by collecting both answer accuracy and response time from the participants. The results show that the naturalness of the triggers remains key to tricking readers.

## introduction when using online platforms, users are asked to agree to the terms of service (tos), which are often long and difficult to understand. according to @xcite , it would take a user around 45 minutes on average to read a tos properly. most users accept the terms without reading them, including clauses which would be deemed unfair under consumer protection standards. software applications that warn consumers about unfair clauses can support consumers' rights, and have been the subject of prior work (e.g., @xcite . at the same time, their existence forms an incentive for drafters of tos to formulate clauses with potentially unfair effects that bypass automated screening. in turn, developers of control systems seek to make their detectors robust against such 'adversarial attacks'. in this paper, we report on an experiment in discovering weaknesses of tos analysis models. natural language processing (nlp) models for tos analysis conduct binary classification of a given clause as fair/unfair. previous studies have shown that state-of-the-art transformer-based classifiers are vulnerable to adversarial attacks @xcite ; even slight modifications to the input text (e.g., changing a few characters) can cause incorrect classifications @xcite . numerous adversarial attack methods have been developed and demonstrate effective attack performance in various downstream nlp tasks such as sentiment analysis @xcite , question answering @xcite , machine-translation @xcite etc. one such method is the attack via a universal adversarial trigger, which is a sequence of tokens (words, sub-words, or characters) that can be injected into any text input from a dataset to mislead the victim model to a target prediction (see the english courts will have jurisdiction over any claim arising from may vote against, or related to , any use of our services. @xmath1 ## universal trigger generation we assume a text input x and its target label y from the dataset @xmath2, y }, a trained victim classifier model f that predicts f (x) = ŷ. while in a nonuniversal targeted attack the focus is on flipping the prediction of a single text input x, our goal is to find an input-agnostic trigger t consisting of a sequence of tokens {w 1 , w 2 , . . . , w i } such that when concatenating t with any input x from x, the victim model incorrectly predicts f (x; t) = ỹ, where ỹ ̸ = ŷ. specifically, we use the following objective function: to solve the above objective function, we follow the approach of @xcite by utilizing the hotflip method @xcite at the token level: first, we initiate the trigger t with a sequence of i placeholder tokens (i.e., 'the'); then we compute the gradient of (1) w.r.t the trigger. since tokens are discrete, we approximate the loss function around the current token embedding using the first-order taylor expansion arg min ## data artifacts as universal triggers a growing number of works have raised awareness that deep neural models may exploit spurious artifacts in the dataset and take erroneous shortcuts @xcite . in this section, we experiment with using dataset artifacts as universal triggers to explore the feasibility of generating universal triggers without access to the victim model's gradient signals. following @xcite , @xcite identified the dataset artifacts as words with high pointwise mutual information (pmi) @xcite with each label. since the claudette dataset has a heavily imbalanced label distribution, in order to prevent picking up very sparse tokens, in this work, we use local mutual information (lmi) @xcite , a re-weighted version of pmi. we observe that high lmi ranked words are successful triggers. we use the 8 highest lmi words and pmi words with label fairness as triggers (hereafter lmi trigger and pmi trigger respectively, please refer to appendix e for the list of words used); and insert them to the unfair clauses at different token positions. the lmi trigger is able to reduce the victim model's classification accuracy from 80% to around 60%; while the pmi trigger can only reduce the performance to around 76% (see figure 3 in appendix e). although less successful than the universal adversarial triggers, the lmi triggers are natural and less detectable than 'all' mode triggers according to our human evaluation studies. critically, lmi triggers are extracted by simply analyzing the training data and do not require access to the victim model. the attack effectiveness of lmi trigger highlights the potential threat of training data leakage in the nlp application. ## human evaluation study we perform a human evaluation to study the impact of token length, insert position and mode on the triggers' detectability 3 . the task is to identify which sentence out of four candidate sentences from tos contracts was modified. we include one question with no modified sentence as the control. in a previous study, song et al. 2021 directly asked 3 we report the details of the web application used and full instructions for the human subjects in appendix d the human participants to rate whether the generated triggers were natural or not. however, the rating of naturalness is very abstract and varies between individuals. inspired by studies on the detection process in psychological studies @xcite , we assume response time (i.e., the length of time taken for a human to detect a trigger) can act as a proxy for the naturalness. to measure the human detectability of triggers, we hence collect the answer accuracy as well as the response time from the participants. 19 participants of different ages, english abilities, and legal experience were recruited from the personal network of the authors. figure (2) demonstrates that it is consistently easier for participants to detect 'all' mode triggers than 'no_subword' mode triggers. participants were on average 19% faster in detecting that a sentence inserted by 'all' than 'no_subword' triggers; and they find 'all' triggers with 21% higher accuracy on average. we include the lmi trigger of token length eight in the study and find its detectability is in between the 'no_subword' and 'all' triggers of the same length. the intuitive notion that participants are better at finding longer triggers generally holds with regard to detection accuracy. nevertheless, we cannot observe a trend in the response time change, which may be due to our small sample size. regarding the insert position, participants are the fastest in detecting triggers inserted in the middle. further, we notice that special tokens and subwords make triggers more obvious. qualitative, informal reports from participants indicate that 'spelling error' stuck out in a legal context. all triggers containing these tokens can be detected with more than 90% accuracy, which include two 'all' triggers of length three (containing special token [sep] or combination of subtokens '##assignabilityconsult'); and one 'no_subword' triggers inserted at position 5 (includes a bound stem 'concul'). this likely explains why these two data points do not conform to the general trend of detection accuracy. ## conclusion we attacked tos unfair clause detectors with universal adversarial triggers generated by a gradientbased algorithm as well as by simply analyzing the training data. the effectiveness of the triggers exposes the vulnerability of the transformer-based classification model, and highlights the potential threat of training data leakage. we also conducted a human evaluation to study the detectability of the triggers. the results show that the triggers are less likely to be detected if they do not include subtokens. future work can explore ways to generate more natural triggers in the legal domain, which may even deceive readers with a formal education in law. limitations @xcite reduce the detection accuracy to 1% while we can only manage to degrade it to 10%. this might be due to the imbalanced label distribution and comparatively small size of the claudette dataset. our human evaluation is an initial exploration with only 19 participants. future work will focus on using crowdsourcing techniques for large survey data collection. furthermore, we generate the 'no_subword' triggers by skipping all the tokens preceded by the double hashtag '##'. this enables us to avoid derivational morphemes and inflection suffixes but fails to exclude bound stems such as 'consul', which makes some triggers obvious to human readers. future work can explore better ways to generate natural triggers. split # sentences % fair label % unfair label train 8354 89.5% 10.5% dev 8279 89.1% 10.9% test 3784 89.3% 10.7%