Package: teapot-nlp
TEAPOTTEAPOT (Tool forEvaluatingAdversarialPerturbationsOnText) is a toolkit to evaluate the effectiveness of adversarial perturbations on NLP systems by taking into account the preservation of meaning in the source.Adversarial perturbations (perturbations to the input of a model that elicit large changes in the output), have been shown to be an effective way of assessing the robustness of machine models.
However, these perturbations only indicate weaknesses in the model if they do not change the input so significantly that it legitimately result in changes in the expected output. While this is easy to control when the input is real-valued (images for example), the situation is more problematic on discrete data such as natural language.TEAPOT is an implementation of the evaluation framework described in the NAACL 2019 paperOn Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models, wherein an adversarial attack is evaluated using two quantities:s_src(x, x'): a measure of the semantic similarity between the original inputxand its adversarial perturbationx'.d_tgt(y(x),y(x'),y*): a measure of how much the output similaritys_tgt(w.r.t. the referencey*) decreases when the model is ran on the adversarial input (giving outputy(x')) instead of the original input (givingy(x)). Specificallyd_tgt(y(x),y(x'),y*)is defined as0ifs_tgt(y,y*) < s_tgt(y',y*)and(s_tgt(y,y*) - s_tgt(y',y*)) / s_tgt(y,y*)otherwise.An attack is declaredsuccessfulonx,ywhens_src(x,x') + d_tgt(y(x),y(x'),y*) > 1, in other words,an adversarial attack changingxtox'is successful if it destroys the target more than it destroys the source.With TEAPOT, you can computes_src,d_tgtand the success rate of an attack easily using proxy metrics for the source and target similarity (chrFby default).Getting startedInstallation and RequirementsTEAPOT works with python>=3.6. The only required non-standard dependency for teapot issacrebleu(a neat tool for computing BLEU and chrF on detokenized text). You can install withpython setup.py installfrom the root of the repo, or simplypip install teapot-nlpfrom anywhere you want.Basic Usage (sequence-to-sequence)Given the original inputexamples/MT/src.fr, adversarial inputexamples/MT/adv.charswap.fr, reference outputexamples/MT/ref.en, original output (output of the model on the original input)examples/MT/base.enand adversarial output (output of the model on the adversarial input)examples/MT/adv.charswap.en, running:teapot\--srcexamples/MT/src.fr\--adv-srcexamples/MT/adv.charswap.fr\--outexamples/MT/base.en\--adv-outexamples/MT/adv.charswap.en\--refexamples/MT/ref.enwill output:Source side preservation (ChrF):
Mean:   86.908
Std:    11.622
5%-95%: 64.109-97.683
--------------------------------------------------------------------------------
Target side degradation (ChrF):
Mean:   21.085
Std:    22.106
5%-95%: 0.000-67.162
--------------------------------------------------------------------------------
Success percentage: 65.20 %Alternatively you can specify only--srcand--adv-src(for source side evaluation)oronly--out,--adv-outand--ref(for target side evaluation).You can learn more about the command line options by runningteapot -h. Notably you can specify which score to use with--s-srcand--s-tgt(refer to the command line help for the list of scores implemented in your version).Basic Usage (other tasks)While the default settings are geared towards evaluating attacks on sequence-to-sequence models, teapot can be used to evaluate attacks on otjer types of NLP models. For example, for a text classification model we might use the zero-one loss ass_tgt(1if the output label is the same as the reference,0otherwise). Here is an example with an attack on a sentiment classification model trained onSST:teapot\--srcexamples/sentiment/src.txt\--adv-srcexamples/sentiment/adv_src.txt\--outexamples/sentiment/out.txt\--adv-outexamples/sentiment/adv_out.txt\--refexamples/sentiment/ref.txt\--s-tgtzero_one\--success-threshold1.8Notice that we specified--success-threshold 1.8, which means that an attack will be considered successful only whens_src(x,x') + d_tgt(y(x),y(x'),y*) > 1.8. While using1as a threshold makes sense whens_srcands_tgtare the same, whens_tgtis the zero-one loss this is a poor choice, as any attack that flips the label will be successful regardless ofs_src. By upping the threshold to1.8we enforce thats_src(here chrF) should be at least0.8for an attack to be successful.Advanced usageCustom scorersTEAPOT comes with predefined scores to compute the source and target side similarity. However, in some cases you might want to define you own score. Fortunately this can be done in a few steps if you are familiar with python:Write your ownteapot.Scorersubclass in a python source file (there are examples inexamples/custom_scorers.py). This is the hard part.Call teapot with the arguments--custom-scores-source path/to/your/scorer.pyand--score [score_key]where[score_key]is the shorthand you have defined for your scorer withteapot.register_scorer(again, see the examples inexamples/custom_scorers.pyfor a walkthrough)If your scorer works fine, and it doesn't rely on heavy dependencies, consider contributing it to TEAPOT byAdding the class toteapot/scorers.pyAdding a simple unit test totests/test_scorers.pyHere is an example wheres_tgtis thef1score defined inexamples/custom_scorers.py:teapot\--srcexamples/MT/src.fr\--adv-srcexamples/MT/adv.charswap.fr\--outexamples/MT/base.en\--adv-outexamples/MT/adv.charswap.en\--refexamples/MT/ref.en\--custom-scores-sourceexamples/custom_scorers.py\--s-tgtf1Or whens_srcis theconstantscore (with auxiliary argument--value)teapot\--srcexamples/MT/src.fr\--adv-srcexamples/MT/adv.charswap.fr\--outexamples/MT/base.en\--adv-outexamples/MT/adv.charswap.en\--refexamples/MT/ref.en\--custom-scores-sourceexamples/custom_scorers.py\--s-srcconstant\--value0.3METEORYou can use theMETEORmetric by specifying--{s-src,s-tgt} meteor, however this will require you to have java installed and METEOR somewhere on your machine and specify the path to the.jarwith--meteor-jar. This is only tested for METEOR-1.5 on linux.You can get METEOR by downloading it from thewebsiteor in the command line:wgethttp://www.cs.cmu.edu/\\\~alavie/METEOR/download/meteor-1.5.tar.gz# This will put the jar at ./meteor-1.5/meteor-1.5.jartarxvzfmeteor-1.5.tar.gzTEAPOT will run the jar from python. You can specify the java command to run the jar with--java-command(defaultjava -Xmx2G -jar)Programmatic UsageHere is an example of how to use TEAPOT in your own code:importteapot# Instantiate the scorer of your choicechrf_scorer=teapot.ChrF()bleu_scorer=teapot.BLEU()meteor_scorer=teapot.METEOR("path/to/meteor.jar",java_command="java -Xmx2G -jar")# Compute s_src for example# This will return a list of chrf scoress_src=chrf_scorer.score(adv_inputs,original_inputs)# Compute d_tgt# This will return a list of relative difference in scores (clamped to positive values)d_tgt=chrf_scorer.rd_score(adv_outputs,original_outputs,reference_outputs)Reference-less evaluationIn the case where no target referencey*is available, one can treat as one the outputy(x)=y*.
We can then use:s_src(x, x'): a measure of the semantic similarity between the original inputxand its adversarial perturbationx'.s_tgt(y(x), y(x')): a measure of the semantic similarity between the respective outputs.Settingy* := y(x)(we are interested in how well the model deviates from its own predictions), the original criterion for success:s_src(x,x') + d_tgt(y(x),y(x'),y*) > 1becomess_src(x,x') + 1 - s_tgt(y(x),y(x')) > 1which is equivalent tos_src(x,x') / s_tgt(y(x),y(x')) > 1.The intuition here is slightly different: now, asuccessfulattack has caused the system to magnify the source-side adversarial noise.Simply run the same commands without providing a reference file, for reference-less evaluation.
For example:teapot --src examples/MT/src.fr \
      --adv-src examples/MT/adv.charswap.fr \
      --out examples/MT/base.en \
      --adv-out examples/MT/adv.charswap.enwill printNo reference file provided. We will use the reference-less criterion.
Source side preservation (ChrF):
Mean: 86.908
Std:  11.622
5%-95%: 64.109-97.683
--------------------------------------------------------------------------------
Target side preservation (ChrF):
Mean: 62.733
Std:  21.529
5%-95%: 16.650-90.555
--------------------------------------------------------------------------------
Success percentage: 97.60 %LicenseThe code is released under theMIT License. Credits to@esenikofor the image.CitingIf you use this software in your own research, consider citing the following paper:@InProceedings{michel2019onevaluation,
  author    = {Michel, Paul  and  Neubig, Graham and Li, Xian and Pino, Juan Miguel},
  title     = {On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models},
  year      = {2019},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)}
}