SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Computational Resources
# Model parameters	355 million
Computational budget	30 min (training & attack)
60 min (testing & detection)
Models and Training
PLM	RoBERTa-large
Prompt model	DART
Max sequence length	128
Embedding dimension	1,024
Batch size	8 (train), 32 (test)
Learning rate	2.0e-5
Optimizer	Adam
Prompt-tuning epochs	20
Shots K	16 per class
Attacks
Attack training epochs	10
Poisoning rate	10%
Target class	0
BadNets trigger	{“cf”, “mn”, “bb”, “tq”}
AddSent trigger	“I watch this 3D movie”
LWP trigger	{“cf”, “bb”, “ak”, “mn”}
EP trigger	{“cf”}
SOS-train trigger	{“friends”, “weekend”, “store”}
SOS-test trigger	“I have bought it from a store with my friends last weekend”
# Triggers	1 per sample
MDP
Masking rate	0.2
# Trials	50
Weight of LLM	1.0
Baseline Defenses
STRIP - # Copies	5
STRIP - Replacement rate	0.25
RAP - Trigger	“mb”
RAP - Training LR	1.0e-2
RAP - Prob. change bound	[-0.3, -0.1]
Table 6. Implementation and evaluation details of models, attacks, and defenses.
The AUC scores of MDP and baseline methods are summarized in Table 7. The performance of MDP with respect to different FRR allowances on the training set, varying weights of  $\mathcal{L}_{\mathrm{MI}}$ , and varying sizes of few-shot data is shown in Figure 8 to Figure 19.
Dataset	Attack	STRIP	ONION	RAP	MDP
SST-2	BadNets	0.66	0.64	0.53	0.99
AddSent	0.51	0.54	0.52	0.99
LWP	0.60	0.72	0.83	0.98
EP	0.84	0.67	0.56	1.00
SOS	0.82	0.61	0.51	1.00
MR	BadNets	0.57	0.63	0.60	0.98
AddSent	0.56	0.58	0.60	0.96
LWP	0.60	0.72	0.51	0.98
EP	0.53	0.66	0.54	0.99
SOS	0.76	0.52	0.52	0.97
CR	BadNets	0.83	0.68	0.59	0.99
AddSent	0.76	0.52	0.52	0.99
LWP	0.71	0.67	0.62	0.97
EP	0.88	0.63	0.58	0.96
SOS	0.71	0.55	0.53	1.00
SUBJ	BadNets	0.57	0.69	0.62	0.95
AddSent	0.64	0.60	0.56	0.99
LWP	0.68	0.73	0.58	0.96
EP	0.64	0.65	0.51	0.96
SOS	0.87	0.56	0.56	0.97
TREC	BadNets	0.62	0.64	0.56	0.99
AddSent	0.60	0.62	0.58	0.97
LWP	0.58	0.73	0.66	0.99
EP	0.82	0.72	0.65	0.98
SOS	0.75	0.73	0.56	0.98
Table 7. Performance (AUC) of MDP and baseline defenses.
Figure 8: Performance of MDP on MR with different FRR allowances on the training set.
Figure 9: Performance of MDP on CR with different FRR allowances on the training set.
Figure 10: Performance of MDP on SUBJ with different FRR allowances on the training set.
Figure 11: Performance of MDP on TREC with different FRR allowances on the training set.
Figure 12: Performance of MDP on MR under the varying weight of the masking-invariance constraint  $\mathcal{L}_{\mathrm{MI}}$
Figure 13: Performance of MDP on CR under the varying weight of the masking-invariance constraint  $\mathcal{L}_{\mathrm{MI}}$
Figure 14: Performance of MDP on SUBJ under the varying weight of the masking-invariance constraint  $\mathcal{L}_{\mathrm{MI}}$
Figure 15: Performance of MDP on TREC under the varying weight of the masking-invariance constraint  $\mathcal{L}_{\mathrm{MI}}$
Figure 16: Performance of MDP on MR with varying size of few-shot data ( $K$  samples per class).
Figure 17: Performance of MDP on CR with varying size of few-shot data ( $K$  samples per class).
Figure 18: Performance of MDP on SUBJ with varying size of few-shot data ( $K$  samples per class).
Figure 19: Performance of MDP on TREC with varying size of few-shot data ( $K$  samples per class).