CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks
Authors: Xuanli He
Qiongkai Xu
Yi Zeng
Lingjuan Lyu
Fangzhao Wu
Jiwei Li
Ruoxi Jia
Venue: NeurIPS 2022 Accept (2022)
Keywords: natural language generation
conditional lexical watermarks
IP protection

Abstract: Previous works have validated that text generation APIs can be stolen through imitation attacks, causing IP violations. In order to protect the IP of text generation APIs, recent work has introduced a watermarking algorithm and utilized the null-hypothesis test as a post-hoc ownership verification on the imitation models. However, we find that it is possible to detect those watermarks via sufficient statistics of the frequencies of candidate watermarking words. To address this drawback, in this paper, we propose a novel Conditional wATERmarking framework (CATER) for protecting the IP of text generation APIs. An optimization method is proposed to decide the watermarking rules that can minimize the distortion of overall word distributions while maximizing the change of conditional word selections. Theoretically, we prove that it is infeasible for even the savviest attacker (they know how CATER works) to reveal the used watermarks from a large pool of potential word pairs based on statistical inspection. Empirically, we observe that high-order conditions lead to an exponential growth of suspicious (unused) watermarks, making our crafted watermarks more stealthy. In addition, CATER can effectively identify IP infringement under architectural mismatch and cross-domain imitation attacks, with negligible impairments on the generation quality of victim APIs. We envision our work as a milestone for stealthily protecting the IP of text generation APIs.

CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks
Xuanli He University College London zodiac.he @ gmail.com
Qiongkai Xu
Yi Zeng' Virginia Tech yizeng @ vt.edu
University of Melbourn qiongkai @ uunimelb.ed..a.m
Lingjuan Lyu'
Fangzhao Wu
Jiwei Shannon.Al. Zhejiang University jiwei_li whannonai.com
Sony Al Lingjuan. sony.com
Microsoft Research Asia fangzwu microsoft.com
Ruoxi Jia Virginia Tech ruoxijia@ vt.edu
Abstract
Previous works have validated that text generation APIs can be stolen through imitation attacks, causing IP violations. In order protect the IP of text genera- tion APIs, recent work has introduced a watermarking algorithm and utilized the null-hypothesis test as post-hoc ownership verification on the imitation mod- els. However, we find that it is possible to detect those watermarks via sufficient statistics of the frequencies of candidate watermarking words. To address this draw- back, in this paper, we propose novel Conditional wATERmarking framework (CATER) for protecting the IP of generation APIs. An optimization method is proposed to decide the watermarking rules that can minimize the distortion of overall word distributions while maximizing the change of conditional word selections. Theoretically, we prove that it is infeasible for even the savviest attacker (they know how CATER works) to reveal the used watermarks from a large pool observe
of potential word pairs based on statistical inspection. Empirically, that high-order conditions lead to an exponential growth of suspicious (unused) watermarks, making crafted watermarks more stealthy. In addition, CATER can effectively identify IP infringement under architectural mismatch and cross-domain imitation attacks, with negligible impairments the generation quality of victim APIs. We envision our work as a milestone for stealthily protecting the IP of text generation APIs.
Introduction
Nowadays, many technology corporations, such as Google, Amazon, Microsoft, have invested a plethora of workforce and computation to data collection and model training, in order deploy well-trained commercial models as pay-as-you-use services on their cloud platforms. Therefore, these corporations own the intellectual property (IP) of their trained models. Unfortunately, previous works have validated that the functionality of a victim API can be stolen through imitation attacks, which
"Equal contribution. Most of the work was finished when X.H was Monash Unversity.
Work done during internship at Sony AI. *Correpoondig author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Aword (benign)) #word (watermrked)
10a+4
10e+2
10e-2
10e-4
different
new
great
high
grood
weather
wind
speed
Union
proposal
EU
novel
Mr
eminent
dissimilar
estimable
Figure I: Ratio change word frequency of top 100 words between benign and watermarked corpora used by [I3. namely (. Red words are the selected watermarks. Although we only list 16 words having the most significant ratio change the benign and watermarked corpora and omit the rest of them for better visualization, all watermarks are within the top 100 words.
inquire the victim with carefully designed queries and train imitation model based on the outputs of the target API. Such attacks cause severe IP violations of the target API and stifle the creativity and motivation of our research community 44
48 20
12 9].
In fact, imitation attacks work not only on laboratory models, but also on commercial APIs 48 51] since the enormous commercial benefit allures competing companies or individual users to extract or steal these successful APIs. For instance, some leading companies in NLP business have been caught imitating their competitors' models 40 Beyond imitation attacks, the attacker could potentially surpass victims by conducting unsupervised domain adaptation and multi-victim ensemble [51].
In order to protect victim models, He et al. 13] first introduced a watermarking algorithm to text generation and utilized the null-hypothesis test post-hoc ownership verification the imitation models. However, traditional watermarking methods generally distort the word distribution, which could be utilized by attackers infer the watermarked words via sufficient statistics of the frequency change of candidate watermarking words. As an example shown in Figure [ the replaced words and their substitutions are those with most frequency decrease ratios and increase ratios, respectively. To address this drawback, we are motivated to develop more stealthy watermarking method to protect the of text generation APIs. The stealthiness of the new watermarks is achieved by incorporating high-order linguistic features as conditions that trigger corresponding watermarking rules.
Overall, our main contributions are as follows
We propose a novel Conditional wATERmarking framework (CATER) for protecting text gen- eration APIs. An optimization method is proposed decide the watermarking rules that can minimize the distortion overall word distributions, while ii) maximize the change of conditional word selections.
Theoretically, we prove that a small number of the used watermarks could be blended in and camouflaged by a large number of suspicious watermarks when attackers attempt to inverse the backend watermarking nules.
Empirically, we observe that high-order conditions lead the exponential growth of suspicious (unused) watermarks, which encourages better stealthiness of the proposed defense with little hurt to the generation of victim models.
2 Preliminary and Background
2.1 Imitation Attack
An imitation attack (a.k.a model extraction) aims to emulate the behavior of the victim model V. such that the adversary can either sidestep the service charges or launch a competitive service [4] 20, 48
12 5T) Malicious users can achieve this goal through interaction with the victim model without knowing its internals, such as the model architecture, hyperparameters, training data, etc. Adversaries
*Code and data available htttpss//gsttibbooo////e//eeeeee

---

first craft set of queries Q based on the documentation of a target model. Then Q will be sent to V obtain the corresponding predictions Y. Finally, an imitation model S can be attained by learning a function to map Q to Y.
The imitation for text generation, a crucial task in natural language processing, has been under-developed until recently Inspired by the efficacy of sequence-level knowledge distillation [7. Wallace et al. 48] and Xu et al. [50 propose mimicking the functionally of commercial text generation APIs. Similar to the standard imitation attack, adversaries can query V with Q. For generation tasks, Y is sequence of tokens (Vi..... VL),, where L is the length of the sequence. According to their empirical studies, one can rival the performance of these APIs, which poses severe threat to cloud platforms.
Most prior imitation attacks are limited to classification tasks 45 30
2.2
Identification of IP Infringement
Prior works have utilized watermarking avenues to achieve a post-hoc verification of the owner- ship 45 24 25). However, this line of work assumes the model owners can watermark victim model y by altering its neurons before releasing V to end-users. This operation is not feasible for the imitation attack, as cannot access the parameters of S. The only thing under the control of V is the responses to adversaries. Hence, some recent works propose creating backdoor to S during the interaction with attackers [20] 42). Specifically, V can select a small fraction of queries and answer them with incorrect predictions, in similar way to the popular choice of watermarks in the computer vision domain (adopting some arbitrary features as the trigger to evaluate) [10, (54). Erroneous predictions are so abrupt that S will memorize these outliers 6 21). As such, V can utilize these watermarks as evidence of ownership.
Albeit the efficacy, the drawbacks of backdoor approaches are tangible as well. First, since V does not impose regulations on users' usage, one cannot distinguish a malicious user from a regular user based on their querying behavior Thus, V has to fairly serve all users and store all mislabeled queries, which leads to a massive storage consumption and a negative impact on the users' experiences. Moreover, the identity of imitation models is unknown to V. V has to iterate over all the mislabeled queries, which is computationally prohibitive. Finally, as S tends to adopt the pay-as-yous as-u-uus policy for the sake of profits, the frute-force interaction with S can cause drastic financial costs.
As remedy, He et al. 1 utilize a lexical watermark to identify IP infringement brought by imitation attacks. They point out that neat watermarking algorithm must follow two principles: i) it cannot significantly impair customer experience, and ii) should not be reverse-engineered by malicious users. In order to fulfill these requirements, they first select a set of words W from the training data of the vietim model V. Then for each w E W, they find R-1 semantically equivalent substitutions for it. Next, they employ W and their substitutions T to compose watermarking words M. Finally, they replace W with M. The rationale behind this avenue is to alter the distribution of words such that the imitation model can learn this biased pattern. To verify such biased pattern of the word choice, He et al. 13 employ a null hypothesis test (36 for evaluation.
More concretely, He et al. [13] utilize an evaluation set O to conduct the null hypothesis test. They formulate the null hypothesis as: the tested model generates outputs without preference for watermarks. A null hypothesis can be either rejected or accepted via the calculation of a p-value [36). They assume that all words {w, we; E UUT) follow a binomial distribution Pr(k; n. p), where is the number of words in .M appearing in O, n is the number of words in W U T found in O, and is the probability of watermarks observed in the natural language. According to their algorithm, p is approximated by 1/ Now, one can compute the p-value from as follows:
P 2 (r( 2 k), Pr(X S A))
The p-value indicates how one can confidently reject the hypothesis. Lower p-luue suggests that the tested model should be more likely subject an imitator.
2.3 Watermark Removal
In conjunction with model watermarking, there is growing body of investigations on watermark removal [45 4 53). This line of work aims to crase watermarks embedded in white-box deep neural
https://cloud, http:::///.looo.goggooom//////// wwwglee..om//aamaslattt//...oo.e

---

D Watermarking Stage
CATER
No
Identification Stage
Yes
CATER Watermark detected!!!
Condition
Imitation Model's API
0
Victim Model's API (V)
Queries
Responses
Queries
Responses
Malicious User
The workflow of CATER IP protection for Generation APIs. CATER first watermarks some of the responses from victim APIs (left). Then, CATER identifies suspicious attacker's s API by watermark verification (right).
Figure 2:
networks. We argue that these approaches are implausible for our setting, text generation APIs are black-box to attackers.
Moreover, one can dub watermarking into form data poisoning [21 50 57 49 in which one can utilize trigger words to manipulate the behavior of the victim model. A list of works has investigated how to mitigate the adverse effect caused by data poisoning in NLP tasks. Qi al. [33 show that GPT2 can effectively identify trigger words targeting the corruption of text classifications. It has been demonstrated that one can use influence graphs as means of the remedy for data poisoning on various NLP tasks 41
3 CATER
This section introduces our proposed CATER, leveraging conditional watermarks to watermark the imitation model, which be served a post-hoe identification of an IP infringement. Figure provides an overview of CATER, consisting of two stages as below.
i) Watermarking Stage: The victim API model V employs CATER add conditional watermarks
to the intended responses. When the vanilla victim model receives queries Q = { from an end-user, V initially produces tentative answer
{wr]ee) Q) Next, V utilizes CATER to conduct watermarking procedure over Y according to the watermarking rules, given condition Finally, V replies to the end-user with a watermarked response (()
ii) Identification Stage: If model S is under suspicion, the victims can query the suspect using verification set O (o.) After obtaining the responses Y [0:) from S, V can leverage CATER to testify whether S violates the IP right of V.
3.1 Watermarking Rule Optimization
Watermarking some words deterministic substitutions could distort the overall word distribution. Therefore, some watermarks could be reversely inferred and eliminated by analyzing the word distribution, as demonstrated in Figure []] We propose to inject the watermarks in conditional word distribution, while maintaining the original word distribution. The substitutions can be conditioned on linguistic features illustrated in Figure Remarkably, given a condition E C and group of semantically equivalent words W, one can replace any words w E W with each other. We formulate the objective of conditional watermarking rules as:
min D
Σ P(w))) Σ ((wle)P(o)) oEC
Σ [(P((olc,
P(w]e)
C
indistinguishable objective
II. distinct objoctive

---

The two factors reflect two essential desiderata:
i) For each W, with w E W, the overall word distributions before optimization P(w) P(w P(c) and after optimization P(w) = Σ ((w should be close to each other, as the indistinguishable objective in Equation 2
ii) For a particular condition e E C, the conditional word distributions should still be distinct to their original distributions, reflected by the dissimilarity between (cw( and lc) as the distinct objective in Equation 2 This guarantees the conditional watermarks are identifiable in verification. In practice, we utilize multiple synonym word sets as a group 9 = {M(() [W])) For each W we can formulate Equation 2 as a mixed integer quadratic programming using f-onrmm distance measurement function:
min (We - Xcccxxc X Tr(( -X X))
-X)
s.t. X 1mway = 11 X {0, 10 1311000001101
(3)
We define matrix X
as the variables for optimization. Matrix W
[P(w [e]] xie] and vector corresponding distributions in a large training corpus. The objective of Equation 3 is convex when
[P()]]c[x are constant variables, decided by calculating
is sufficiently small (see the proof in Appendix Aj. We optimize the watermark assignments ((w(()))
using Gurobi with a = 0.01.
3.2 Constructing Watermarking Conditions using Linguistic Features
This part will concentrate on practical ways to construct the watermarking conditions C. We consider two fundamental linguistic features F, i) part-of-speech and ii) dependency tree, and their high- order variations conditions. Such linguistic features were widely and successfully used for text classification 5 52,, sequence labeling [22 37 and etc.
punct
obl
nsubj
case
PRON +modd
PRON
AUX
VERB
ADP
NOUN
PUNCT
we
ve
started in
our
region
area
nsubj
mmod case
punct
DET
det
NOUN
ADP
DET
det NOUN
AUX
VERB
PRON
every
region of the
country could
do
this
Figure 3: The pfrt-of-speech (POS) tags and dependency relations are illustrated in colored boxes and arcs, respectively. The decision of using "region" or its synonym ("area" or other words), is conditioned its linguistic features in context. For example, in the first sentence, either "PRON" (POS label of the preceding token) or "obl" (DEP label of the incoming arc) gives the decision to replace "region" with "area", as opposed to "DET" or "nsubj" in the second sentence.
Part-of-Speech Part-of-Speeede (POS) tagging is grammatical grouping algorithm, which can cluster words according to their grammatical properties, such as syntactic and morphological behav- iors [66.. The POS tag for each token is demonstrated in colored box, in Figure
Given word w a sentence, we denote its POS as lo, and use I- or 4+ to represent the POS of the k-th word to the left or right of w. We consider single label L-1 as our first-order condition. In order to reduce the identifiability of our conditional watermark, we can construct high-order conditions from the same feature set, e.g., (1-1,41) as second-order condition and (1-2,~-1, 41) as third-order condition. Note that if 1. or 4+ does not exist, we use pseudo tag "[none]" by default. Since POS describes grammatical roles of words and its classes are limited, the combination of POS of an
5

---

anchor and its neighbors should also be bounded. Thus one can consider the POS bond among words as the condition.
Dependency Tree Dependency Tree (DEP) is a syntactic structure, which describes directed binary grammatical relations between words [T6] shown in Figure A dependency tree can be represented by an acyclic directed graph G = (V, E), where V is a set of vertices corresponding to all words in given sentence, E is set of ordered pairs of vertices, denoted arcs. An are e E E describes a grammatical relation between two vertices in V, i.e., source vertex named as head and target vertex coined as dependent. Except the root vertex, each vertex is connected to by exactly one head. Consequently, there exists unique path from each vertex to the root node in a dependency tree.
Analagously for POS features, can design first-order and high-order DEP features as watermarking conditions. Given a word w, and its incoming DEP arc, we use the DEP label of the are as the first-order features (d1). We construct high-order condition recursively using the labels of incoming DEP ares (d1, d2, ... ). A pseudo are label "[none]" is used when there is no parent node in recursion.
3.3 Identifiability of Conditional Watermark
In this section, we discuss the identifiability of our watermark if the attackers attempt to infer the used watermarks. We assume the worst case that the attackers have access to the watermarking algorithm, iř) all possible word sets for substitution G, and îii) combination of feature sets F as wartermarking conditions C. However, the exact watermarking rules are unobservable attackers. The attackers may identify the watermark rules by suspecting those observed P(w lc) with extreme distributions, i.e., only a single word in a synonym set is selected given a specific condition.
Given limited budget, assume that an imitator has queried our watermarked API and has acquired N tokens in U, W() The system has incorporated watermarks with K -order features É C, where = (Fi, F2, ... F.F),, the total number of possible conditions is |C| I [Fs). We simplify our discussion by using the same feature set F (POS or DEP), then [C| [
Theorem 3.1. If |/| K > N and there exist I conditions that have less or equal to m E Zt support samples, then 2 |F| N/mm + 1).
The attacker would suspect conditional word distributions that are extremely imbalanced, namely only single dominant choice of word is observed within the responses to the attacker.
Theorem 3.2. Having m support samples for a specific condition c, the possibility of observing extremely imbalanced word choice is I(W,c, c, = =wnee P(wslc)"". If < m and m, m' E Zt, I(W, c. m²) 2 I(W.c.)
The proofs of Thm 3.1 and Thm 3.2 can be found in Appendix B and Thm 3.1 guarantees
lower bound for the number of conditions that attackers will have less or equal to m observed samples. Moreover, the lower bound grows exponentially with regard to the feature orders used as watermarking conditions. Thm 3.2 guarantees the high probability of the conditions with extremely imbalanced word selection when m is small. Combining these two theorems, the total number of the suspicious watermark rules could be huge compared with the limited used watermark rules if are utilizing high-order linguistic features conditions. We further empirically demonstrate the significant confusion between suspected and used watermark rules in Section 4.2
4 Experiments
Text Generation Tasks. We examine two widespread text generation tasks: machine translation and document summarization, which have been successfully deployed as commercial APIs q To demonstrate the generality of CATER, we also apply it to two more text generation tasks: 7) text simplification and ii paraphrase generation. We present the performance of CATER for these
tasks in Appendix F.4
Machine Translation: We consider WMT14 German (De) -English (En) translation as the testbed. We follow the official split: train (4.5M) dev (3,000) / test (3,003). Moses 1 is
hatpe:://tramslatteggoggg/
htrp:://leeaal.org/mcciinie htw.a//o--ooooonnnnooooo///aaaaa

---

Table 1: Performance of different watermarking approaches WMT14 and CNN/DM. We use F1 scores of ROUGE-1, ROUGE-2 and ROUGE-L for CNN/DM.
WMT14
CNN/DM
p-value
BLEU
BERTScore
p-value
ROUGE-1/2L
BERTScore
w/o watermark
> 10
31.1
65.9
> 10
37.7/15.4/1.2
22.1
Venugopal et al. 47
unigram
10-2
30.4
65.2
10
37.3 15.1 31.2
21.7
trigram
> 10
30.8
65.7
10
37.5 15.3 31.0
21.8
sentence
> 10
30.8
65.9
> 10
37.6/ 15.4 31.2
21.9
He et al. [3
spelling
10-13
31.1
65.8
< 10
37.5 15.2 31.4
22.0
synonym
< 10-10
30.8
65.5
< 10-8
37.6/ 15.3 33.4
21.8
CATER (ours)
-DEP
10-4
30.9
65.4
10
37.6/ 15.3 31.3
21.8
-POS
10-7
30.8
< 10-7
37,5/ 15.2155//13
21.9
applied to pre-process all corpora, with cased tokenizer. We use BLEU 33 and BERTScore to evaluate the translation quality. BLEU concentrates lexical similarity via n-grams match, whereas BERTScore targets at semantic equivalence through contextualized embeddings.
(55)
Document summarization: CNN/DM [14] utilizes informative headlines summaries of news articles. We reuse the dataset preprocessed by See al. [38] with a partition of train/dev/test as 287K / 13K / ///11 11K. Rouge [26] and BERTScore [55 are employed for the evaluation metric of the summary quality.
We use 32K and 16K BPE vocabulary 39 for experiments on WMT14 and CNN/DM, respectively.
Models. For the primary experiments, we consider Transformer-bass 46 the backbone of both victim models and the imitation models. Following He et al. [13], we use a 3-layer Transformer for the summarization task. Because of their superior performance, pre-trained language models (PLMs) have been deployed on cloud platforms Hence, we also consider using two popular PLMs: BART (summarization) (33 and ii) mBART (translation) [27] the victim model. Regarding the imitation model, since the architecture of the victim model is unknown to the adversary. we simulate this black-box setting by using three different architectures the imitator, namely (m)BART,
Transformer-base. and ConvS2S g The training details are summarized in Appendix
Basic Settings. As proof-of-concep- We assume the victim model V and the imitation model S use the same training data, but uses the response y' with CATER instead of the ground-truth y. We the size of synonyms to 2 and vary this value in Appendix F. The detailed construction of watermarks and approximation of p in Equation for CATER is provided in Appendix D
start our evaluation with most straightforward case.
Baselines. We compare our approach with [47 and 13 Venugopal et al. [47 proposed water- marking the generated output with sequence of bits under the representation of either n-grams the complete sentence. He et al. [13] devises two effective watermarking approaches. The first one replaces all the watermarked words with their synonyms. The second one watermarks the victim API outputs by mixing American and British spelling systems.
4.1 Performance CATER
Table I presents the watermark identifiability and generation quality of studied text generation tasks. Both and CATER obtain a sizeable gap the p-value, and demonstrate negligible degradation in BLEU, ROUGE, and BERTScore, compared to the non- won-wattrmarking baseline. However, [7] falls
short of injecting detectable watermarks. Although CATER is slightly inferior to [3] p-value, we
argue that watermarks in [33] can be easily erased, as their replacement techniques are not invisible. As shown in Figure [ the synonyms used by [13] can be identified due to the tangible distribution shift the watermarks, whereas CATER manages to minimize such shift according to Equation 2 which is also corroborated by Figure 7 In addition, one can eliminate the spelling watermarks by consistently using spelling system.
https://clooud http:///wwd..oggl...oooo wwm/aiiii///.. httpa//aaaaaattttaaaaaaa/aa/a/aa http:://ooaooooo.o..ooocolltttet

---

Table 2: Imitation performance of different architectures on clean and watermarked data. Numbers in parentheses are results of clean data. Victim models trained on mBART (WMT14) and BART (CNN/DM), respectively. We use the first-order POS as the watermarking approach.
Model
WMT14
CNN/DM
p-value
BLEU
p-value
ROUGE-L f
(m)RART
< 10 (> 10"1)
34.9 (35.2)
< 10 (> 10")
38.1 (38.1)
Transformer
< 10 (> 10-2)
32.7 (33.0)
< 10 (> 10")
32.8 (32.9)
ConvS2S
< 10 (> 10
32.7 (32.9)
< 10 (> 1
32.7 (32.7)
32
31
10
30
29
10
28
27
CATER (POS)
10
CATER (POS)
26
Clean
Clean
25
2 orders
orders
Figure 4: BLEU scores (left) and p-value (right) of using different orders of the POS watermarking approach WMT14 data. X-axis indicates the orders of conditions. 1, 3 represent the first-order, second-order, and third-order condition respectively. Clean means imitation with clean dataset.
Unless otherwise stated, we use the first-order POS as the default setting for CATER, due to its efficacy terms of watermark identifiability and generation quality.
IP Identification under Architectural Mismatch The architectures of remote APIs are usually unknown the adversary. However, recent works have shown that the imitation attack is effective even if there is an architectural mismatch between the victim model and the imitator [48] 12]. To demonstrate that approach is model-agnostic, we use BART-family models as victim models and vary architectures of imitation models.
Table 2 summarizes p- p-value and generation quality of CATER on WMT14 and CNN/DM datasets. Similar to Table CATER can confidently identify the IP infringement when the architecture of the imitation model is the same as that of the victim model, with gap of p-value between watermarked model and benign model being 3 orders of magnitude. In addition, this gap applies to the case, where use distinct architectures for the victim model and the imitator. The generation quality exhibits negligible drops, within a range of 0.3, Note that the generation quality of Transformer and ConvS2S imitators degrades due the capacity gap, compared to powerful BART-family models.
IP Identification on Cross-domain Imitation Sim- ilarly, the training data of the victim model is confiden- tial and remains unknown to the public. Thus, there could be a domain mismatch between the training data of the victim model and queries from the adversary. In order exhibit that our approach is exempt from the domain shift, we use two out-of of-oomiin datasets to conduct the imitation attack for the machine translation task. The first is IWSLT14 data B with 250K German
Table 3:
Imitation performance of using data from different domains. The victim model trained on WMT14. We use first- order POS as the watermarking condition.
WMT14 IWSLT14
OPUS (Law)
10-7
10-5
10
sentences, and the second is OPUS (Law) data [43] consisting 2. German sentences. Table suggests that despite the domain mismatch, CATER can still watermark the imitation model, and one can identify watermarks with high confidence.
High-order Conditions We have shown that the first-order CATER effectively performs various tasks and settings. We argue that CATER is not limited to the first-order condition. Instead, one can use high-order CATER mentioned in Section 3.2 which can consolidate the invisibility as discussed in Section 3.3 Therefore, we investigate the efficacy of the high-order CATER to the translation task and provide the study on the summarization task in Appendix F2
8

---

32.0
38.0
10
36.0
10
30.0
ROUGE
34.0
pvalvv
vvllu
28.0
10
10
32.0
26.0
10
30.0
10
value
value
24.0
80
100
20
40
60
watermarking percentage
watermarking percentage
(a) Machine Translation
(b) Summarization
Figure 5: The generation quality and p-value under different percentage of using watermarked data for imitation attacks machine translation and summarization.
Table 4: Imitation performance with watermark removal on WMT14 data and CNN/DM. We use the first-order POS as the watermarking approach. ONION used remove watermarks.
Model
WMT14
CNN/DM
p-value
BLEU
p-value |
ROUGE-L
w/o ONION w/ ONION
10-?
30.8
10-7
31.2
< 10
27.0
10
25.9
Figure[sshows that with the increase of conditional POS orders, compared to the use of clean data, there is side effect the BLEU scores, i.e., generation quality. The right figure suggests that using the higher conditional orders can lead to the larger p-value, which means that the claim about IP violation is less confident. However, the gap between the benign and watermarked models is still significantly distinguishable. Note that for the sake of fair comparison, the p-values of the clean model are calculated wet. the corresponding order.
Mixture of Human- and Machine-labeled Data Due multiple factors, such as noisy in- puts [19 1. domain mismatch II 29 etc., training model with machine translation alone still underperforms using human-annotated data [5T]. However, since annotating data is resource- expensive [51], malicious may mix the numan-annotate data with machine-annotated one. We examine the effectiveness of CATER under this mixture two types of datasets.
Figure 5 suggests that as CATER aims to minimize the distribution distortion, watermarks injected by CATER tend to be overwritten by clean signals. Thus, CATER is active when more than half of the data is watermarked.
4.2 Analysis on Adaptive Attacks
The previous sections illustrates the efficacy of CATER for watermarking and detecting potential imitation attacks. Given the case that a savvy attacker might be aware of the existence of watermarks, they might launch countermeasures remove the effects of the watermark. This section explores and analyzes possible adaptive attacks based on varying degrees of prior knowledge of our defensive strategy. Specifically, examine types of adaptive attacks that try erase the effects of the watermark: iJ vanilla watermark removal, and ii) watermarking algorithm leakage.
Vanilla Watermark Removal. Under this setting, we assume the attackers are aware of the existence of watermarks, but not aware of the details the watermarking algorithm. Following such setting of attacker knowledge, we assume the attacker would adopt an existing watermark removal technique their vanilla form. evaluate, we employ ONION, a popular defensive avenue data poisoning in the natural language processing field, which adopts GPT-2 35 to expel outlier words. The defense results are shown Table 4 We find that ONION cannot erase the injected watermarks. Mcanwhile, it drastically diminishes the generation quality of the imitation model.

---

Watermarking Algorithm Leakage. Under this case study, we assume attackers have access the full details of our watermarking algorithm, ie,, the same watermark- ing dictionary and the features for constructing watermarking conditions. We note that this the most substantial attacker knowledge assumption we can imagine, aside from the infeasible case that they know the complete pairs of watermarks we used.
After collecting responses from the victim model, the attackers can leverage the leaked knowledge to analyze the responses to find the used watermarks, te.. the number of sparse tries. As shown Section 3.3] we theoretically prove that such reverse engineering is infeasi- ble. In addition, F shows that even with
10
#sparse entries (log10 scaie)
CATER
Watermarking Algorithm Leakage
Upper Bound
10
such strong attacker knowledge, the amount of potential candidate watermarks (orange curve) is still astronomical times larger than the used number of watermarks (blue curve). Thus, mali- cious users would have difficulty removing wa- termarks from the responses; unless they lean to- ward modifying all potential watermarks. Such a brute-force approach can drastically debilitate the performance of the imitation attack, causing feeble imitation. Finally, we demonstrate the upper bound (green curve) to show that without the curated knowledge about the watermarking conditions, the attackers have to consider all pos- sible combinations of POS tags. Therefore, the difficulty of identifying the watermarks from the top 200 words can be combinatorially exacer- bated.
10
10
2
3
4
orders
Figure 6: The number of sparse entries (suspected watermarks) of top 200 words with watermark- ing algorithm leakage under different orders (or- ange) the training data. CATER indicates the actual number of watermarks used by our water- marking system (blue). POS feature is used where | = 36. The upper bound indicates all possible combinational watermarks (green).
5 Conclusion
In this work, we are keen on protecting text generation APIs. We first discover that it is possible to detect previously proposed watermarks via sufficient statistics of the frequencies of candidate watermarking words. We then propose novel Conditional w wATERmarking framework (CATER), for which, an optimization method proposed decide the watermarking rules that can minimize the distortion of overall word distributions while maximizing the change of conditional word selections. Theoretically, we prove that is infeasible for even the savviest attackers, who know how CATER algorithms, to reveal the used watermarks from large pool of potential watermarking rules based on statistical inspection. Empirically, we observe that high-order conditions lead an exponential growth of suspicious (unused) watermarks, rendering our crafted watermarks more stealthy.
Limitation and Negative Societal Impacts
One major limitation of work is that one has to find high-quality synonym sets to minimize semantic degradation, leading to the limited option of candidate words. Nevertheless, according Section 4. given the top 200 words and their synonyms, CATER can still achieve stealthy watermarking. In addition, because of the use of the lexical match, we experience slight performance degradation generation quality. Furthermore, since defending against imitation attacks is difficult, we resort to a post-hoc verification. If the adversaries do not publically release the imitation model, CATER becomes fruitless.
Regarding the negative societal impacts, CATER might be overused by some APIs owners as a means of unfair competition. As shown in Figure 4 the gap between the benign model and the watermarked one is small. Hence, the APIs owners could leverage CATER to sue innocent cloud services. As remedy, suggest the judges refer relatively higher bar, e.g., lower p-value < 10-6.
Acknowledgments and Disclosure of Funding
This research was funded by Sony AI. We would like to appreciate the valuable feedback from all anonymous reviewers.
10

---

References
[1] Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both break neural machine translation. In International Conference Learning Representations, 2018.
[2] Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand Radu Soricut, Lucia Specia, and Alek Tamchyna. Findings of the 2014 workshop statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 12-58, Baltimore, Maryland, USA, June 2014. Association for Computational Linguistics.
[3] Mauro Cettolo, Jan Nichues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. Report on the I 1thh iwslt evaluation campaign, iwslt 2014. In Proceedings of the International Workshop on Spoken Language Translation, Hanoi, Vietnam, volume 57, 2014.
[4] Xinyun Chen, Wenxiao Wang, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, and Dawn Song. Refit: A unified watermark removal framework for deep learning systems with limited data. Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, ASIA CCS '21, page 321-335, New York, NY, USA, 2021. Association for Computing Machinery.
[5] William Cohen, Vitor Carvalho, and Tom Mitchell. Learning to classify email into "speech acts". In Proceedings of the 2004 Conference Empirical Methods in Natural Language Processing, pages 309-316, 2004.
[6] Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A backdoor attack against Istm-based text classification systems. IEEE Access, 7:138872-138878 2019.
[7] Christiane Fellbaum. Wordnet. In Theory and applications of ontology: computer applications, pages 231-243 Springer, 2010.
[8] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. In International Conference on Machine Learning, pages 1243- 1252. PMLR, 2017.
[9] Shangwei Guo. Chunlong Xie, Jiwei Li, Lingjuan Lyu, and Tianwei Zhang, Threats to pre- trained language models: Survey and taxonomy. arXiv preprint ar arXiv:2202.06866 2022.
[10] shangwei Guo, Tianwei Zhang, Han Qiu, Yi Zeng, Tao Xiang, and Yang Liu. Fine-tuning is not enough: A simple yet effective watermark removal attack for dnn models. arXiv preprint arXiv:2009.08697, 2020.
[11] Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2022.
[12] Xuanli He, Lingjuan Lyu, Lichao Sun, and Qiongkai Xu. Model extraction and adversarial trans- ferability, your bert vulnerable! In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2006-2012, 2021.
[13] Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property of language generation apis with lexical watermark. In Thirtieth AAAI Conference on Artificial Intelligence, 2022.
[14] Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines read and comprehend. Advances in neural information processing systems, 28, 2015.
[15] Tom Hosking. Hao Tang, and Mirella Lapata. Hierarchical sketch induction for paraphrase generation. Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Pupers), pages 1489-2501. 2022.
[16] Daniel Jurafsky and James Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, volume
02 2008.
11

---

[17] Yoon Kim and Alexander M Rush. Sequence-level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods Natural Language Processing, pages 1317-1327, 2016.
[18] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison- Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondfej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computationa Linguisties Companion Volume Proceedings of the Demo and Poster Sessions, pages 177-180, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
[19] Philipp Koehn and Rebecca Knowles. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28-39, Vancouver, August 2017. Association for Computational Linguistics.
[20] Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit lyyer. Thieves on sesame street! model extraction of bert-based apis. In International Conference on Learning Representations, 2020.
[21] Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2793-2806, 2020.
[22] John D Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, 2001.
[23] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence--o--eequence pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, 2020.
[24] Meng Li, Qi Zhong, Leo Yu Zhang, Yajuan Du, Jun Zhang, and Yong Xiangt. Protecting the intellectual property of deep neural networks with watermarking: The frequency domain approach. In 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), pages 402-409. IEEE, 2020.
[25] Jian Han Lim, Chee Seng Chan, Kam Woh Ng, Lixin Fan, and Qiang Yang. Protect, show, attend and tell: Empowering image captioning models with ownership protection. Puttern Recognition, 122:108285. 2022.
[26] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74 81, 2004.
[27] Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726-742, 2020.
[28] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv: 1301.3781 2013.
[29] Mathias Müller, Annette Rios, and Rico Sennrich. Domain robustness neural machine translation. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume I: Research Track), pages 151-164, Virtual, October 2020. Association for Machine Translation in the Americas.
[30] Tribhuvanesh Orekondy, Bern Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 4954-4963, 2019.
[31] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics ( (Demonstrations), pages 48-53, 2019.
12

---

[32] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318, 2002.
[33] Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9558-9566, 2021.
[34] Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2020.
[35] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[36] John Rice. Mathematical statistics and data analysis. Cengage Learning, 2006.
[37] Sunita Sarawagi and William W Cohen. Semi-markov conditional random fields for information extraction. In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2004.
[38] Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, 2017.
[39] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-1725, 2016.
[40] Amit Singhal. Microsoft's 's bing uses google search results-and and denies it, 2011.
[41] Xiaofei Sun, Jiwei Li, Xiaoya Li, Ziyao Wang, Tianwei Zhang, Han Qiu, Fei Wu, and Chun Fan. general framework for defending against backdoor attacks via influence graph. arXiv preprint arXiv:2111.14300 2021.
[42] Sebastian Szyller, Buse Gul Atli, Samuel Marchal, and N Asokan, Dawn: Dynamic adversarial watermarking of neural networks. In Proceedings of the 29th ACM International Conference Multimedia, pages 4417 4425, 2021.
[43] Jörg Tiedemann. Parallel data, tools and interfaces in opus. In Proceedings of the Eighth International Conference Language Resources and Evaluation (LREC' pages 2214- 2218, 2012.
[44] Florian Tramèr, Fan Zhang, Ari Juels, Michael Reiter, and Thomas Ristenpart. Stealing ma- chine learning models via prediction apis. In 25th (USENIX) Security Symposium (!USENIX) Security 16), pages 601-618, 2016.
[45] Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pages 269-277, 2017.
[46] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998 6008, 2017.
[47] Ashish Venugopal, Jakob Uszkoreit, David Talbot, Franz Och, and Juri Ganitkevitch. Water- marking the outputs of structured prediction with an application in statistical machine translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Process- ing. pages 1363-1372, Edinburgh, Scotland, UK., July 201 I. Association for Computational Linguistics.
[48] Eric Wallace, Mitchell Stern, and Dawn Song. Imitation attacks and defenses for black-box machine translation systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5531 2020.
13

---

[49] Jun Wang, Chang Xu, Francisco Guzmán, Ahmed EI-Kishky, Yuqing Tang, Benjamin Rubin- stein, and Trevor Cohn. Putting words into the system's mouth: A targeted attack on machine translation using monolingual data poisoning. In Findings of the Association for Computational Linguistics: ACI-IJCNIPP 202/, pages 1463-1473, 2021.
neural
[50] Chang Xu, Jun Wang, Yuqing Tang, Francisco Guzmán, Benjamin I. P. Rubinstein, and Trevor Cohn. A targeted attack on black-box neural machine translation with parallel data poisoning. In Proceedings of the Web Conference 2021, pages 3638-3650, 2021.
[51] Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari. Beyond model extraction: Imitation attack for black-box nlp apis. arXiv preprint arXiv:2108, 13873, 2021.
[52] Qiongkai Xu and Hai Zhao. Using deep linguistic features for finding deceptive opinion spam. In Proceedings of COLING 2012: Posters, pages 1341-1350, 2012.
[53] Yifan Yan, Xudong Pan, Yining Wang, Mi Zhang, and Min Yang. and then there were none": Cracking white-box dnn watermarks via invariant neuron transforms. arXiv preprint arXiv:2205.00199, 2022.
[54] Yi Zeng. Won Park, Z Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 16473-16481, 2021.
[55] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2019.
[56] Xingxing Zhang and Mirella Lapata. Sentence simplification with deep reinforcement learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 584-594, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.
[57] Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang, Lichao Sun, and Xu Sun. How to inject backdoors with better consistency: Logit anchoring clean data. In International Conference on Learning Representations, 2021.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO]] to [Yes], [No] or [N/A]. . You are strongly encouraged to include a justification your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section X.
Did you include the license the code and datasets? [No] The code and the data proprietary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the
Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See Section 4.1
(b) Did you describe the limitations of your work? [Yes] See Appendix 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix(5)
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
14

---

2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Sections 3.1 and 3.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendices
C
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material
URL)? [Yes] See
https://github. cww.coo//lllooc xwwex/lhxxxxcecc cater_ peciterrneurips.ee git
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.1 and Appendix [
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] See Section 4.1
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Appendix [
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes] See Section 4.1
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "vP-tUydhfu", "reviewer": "Reviewer_hUhu", "summary": "This paper proposes a watermarking technique to claim ownership of text generation APIs when subject to imitation attacks. They show that prior work manages to watermark and detect imitation behaviors, but a weakness exists. After analyzing the word frequency in API responses and publicly available data, the adversary can identify the watermarks with a higher chance. Based on this observation, they devise a novel method CATER, which can conditionally watermark the response so that the adversary cannot decipher the watermarking keys. Meanwhile, CATER can identify the watermarked imitators as effectively as the previous work.\n\nCATER leverages two linguistic features to fulfill the conditional watermarking: 1) neighboring part-of-speech tags and 2) dependency relations. They work well on two popular text generation tasks. They show that CATER can scale to high-order settings, which are empirically challenging to be identified by attackers. According to their experiments, the proposed watermarks have negligible adverse impacts on the utility.\n", "questions": "Please refer to the weakness part.", "limitations": "The authors only study CATER for the English-centric datasets. However, as we know, the widespread text generation APIs are for translation, which supports multiple languages. Probably, the authors could extend CATER to other languages in the future.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1.\tMost works in protecting IP from imitation attacks focus on classification tasks. Wallace et al. (2020) have shown that imitation attacks are effective on commercial translation APIs. However, as an urgent vulnerability, little work has been done to protect the IP of text generation APIs, except He et al. (2021). This work identifies and addresses a major weakness in He et al., and proposes a more invisible watermarking algorithm, making their method more appealing to the community.\n2.\tInstead of using a heuristic search, the authors elegantly cast the watermark search issue into an optimization problem and provide rigorous proof.\n3.\tThe authors conduct comprehensive experiments to validate the efficacy of CATER in various settings, including an architectural mismatch between the victim and the imitation model and cross-domain imitation.\n4.\tThis work theoretically proves that CATER is resilient to statistical reverse-engineering, which is also verified by their experiments.  In addition, they show that CATER can defend against ONION, an effective approach for backdoor removal.\n\nWeakness:\n1.\tThe authors assume that all training data are from the API response, but what if the adversary only uses the part of the API response?\n2.\tFigure 5 is hard to comprehend. I would like to see more details about the two baselines presented in Figure 5.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Z8_LU853CFs", "reviewer": "Reviewer_EoSA", "summary": "\nThe paper proposes a new watermarking framework for text generation APIs. Compared with the existing framework, the new framework increases stealth by minimizing the distortion of overall word distributions and incorporating high-order linguistic features. The experiments show that the new framework can identify imitation models with less capability than the existing method and keep the utility. However, it is proven to tolerate adaptive attacks, so it can be applied to protect text generation APIs better.", "questions": "Comment on the availability of practical attacks toward [10].\n\nIn section 4.2, you state that \"malicious users would have difficulty in removing watermarks from the responses; unless they lean toward modifying all potential watermarks.\". According to the article provided below, it seems to be possible to modify the watermarks without much loss of utility. Is that a threat to your framework? Please, clarify.\n\n[A] Cracking White-box DNN Watermarks via Invariant Neuron Transforms. https://arxiv.org/abs/2205.00199\n", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "pros:\n\n- an important research problem.\n- the proposed approach is carefully justified from both theoretical and empirical perspectives.\n- high-quality paper writing.\n\nCons:\n\n- Research motivation is questionable. Is there any concrete attack toward [B]?\n- lack of evaluation under active attackers. Can you justify the strength of your defense under [A]?\n- Novelty comparing to [B] is incremental. You achieved nearly the same performance with [B] in Table 1. Moreover, some (broken) texts in the current manuscript are from [B]. The authors should take a careful pass to paraphrase the paper.\n\n[A]: Killing One Bird with Two Stones: Model Extraction and Attribute Inference Attacks against BERT-based APIs. https://arxiv.org/abs/2105.10909\n\n[B]: Protecting Intellectual Property of Language Generation APIs with Lexical Watermark. AAAI 2022\n\nThe new framework is proposed to improve the stealth of the existing method. They are different in the optimization method and linguistic features for conditions. The optimization method is creative. The design for linguistic features is referred to another existing method.\n\nIn terms of the quality, the paper provides sufficient theoretical analysis to prove the feasibility of the new framework. In addition, there are empirical experiments conducted to verify the ability of the new framework. However, there is no direct comparison between the stealthiness of the new framework and the existing framework. Please justify if that was impossible or not needed.\n\nTechnical novelty and empirical results comparing to [B] seems incremental, particularly from Table 1. And the research motivation is also questionable. The improvement is based on the defense of a specific method of attack. However, there is no concrete breach of such an attack. The paper also lacks a deep description of this attack method, including the process of distorting the result of identification and the performance of attacking. Without concrete attack information, the application of the new method seems to be less urgent, in its current form.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9qTbOLGogi8", "reviewer": "Reviewer_eyPy", "summary": "This paper presents a simple but effective approach (called CATER) to protect the IP of text generation APIs under imitation attacks. This idea can stamp watermarks on imitation models by altering the distribution of semantically equivalent words. Unlike the previous work, they propose to conditionally watermark the victim’s outputs so that it is infeasible for the adversary to crack watermarks over the watermarked response, both theoretically and empirically. \n\nThe authors employ two linguistical rules as the conditions. The first one utilizes the part-of-speech tags of surrounding words as the condition. The second condition is established on the incoming arc of the dependency tree. According to their empirical studies, CATER can effectively watermark imitation models and identity the watermarks under various settings. In addition, the authors have shown that CATER is resilient to two strong watermark removal approaches.\n", "questions": "1.\tSince APIs aim to serve end-users, in addition to automatic metrics, would it be possible to conduct a human evaluation before concluding that the watermarks do not affect generation quality?\n\n2.\tWould you please elaborate on the calculation of the p-value on CATER? It is unclear to me.\n", "limitations": "As far as I know, ONION focused on insertion-based removal. However, CATER utilizes substitution-based watermarking. So, probably the authors could try a substitution-based removal technique.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper found that analyzing the word distribution could reveal watermarks used by the previous watermarking approach (He et al.). Hence, the authors devise a conditional watermarking algorithm as a remedy.\n\n2.\tTo perform the conditional watermarking and minimize the distribution shift observed in He et al., the authors formulate these two objectives as a linear programming problem. In addition, they prove that this optimization problem is solvable.\n\n3.\tThe authors also rigorously prove that watermarks injected by CATER are infeasible to be inferred by statistical reverse-engineering, especially when scaling to high-order features.\n\n4.\tThe paper shows that CATER rivals previous watermarking approaches and is effective in various settings, such as cross-domain querying and architectural mismatch.\n\nOverall, I think this is a quite good paper, which may bring groundbreaking impact on protecting IP of NLP models.\n\nWeakness:\n\nThe paper is of good quality and easy to follow. I have no major concerns but a few comments as below.\n\n1.\tSince APIs aim to serve end-users, in addition to automatic metrics, it would be good to conduct a human evaluation before concluding that the watermarks do not affect generation quality.\n\n2.\tWould you please elaborate on the calculation of the p-value on CATER? It is unclear to me.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2gy2pY_7QCr", "reviewer": "Reviewer_24GZ", "summary": "First of all, thank you for sharing great work with the community.\n\nWith the great success of language generative models, many models are employed in a business. So, the research community focused on the watermarking method to protect these models. The authors pointed out the vulnerability of existing watermarking methods against imitation attacks. They showed that existing works could be breakable when sufficient statistics are available. Therefore, this paper proposed a new framework named Conditional wATERmarking (CATER). The CATER framework is based on linguistic features (e.g., part-of-speech and dependency tree).", "questions": "1. Is the framework also working for other text generation tasks? (Connected with cons. in the above section.)\n2. Is there a possibility of a collusion attack?\nFor example, there are two victim models using CATER for watermarking.\nAnd the adversary trains his/her model using both two victim models.\nIn this case, who is the victim between both two victims?\n", "limitations": "As I mentioned above, the paper needs more empirical experiments on different text generation tasks.\nAnd also, the figures need to be updated for the quality of the paper.\nBased on the discussion period, the score could be updated.\n\n---\nThank you for the response.\nThe authors resolve the questions I have.\nAnd the additional experimental results are promising.\nTherefore, I decided to increase my current score.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The authors gave mathematical proof of their framework's identifiability.\n2. CATER is robust against white-box attacks.\n\nWeaknesses\n1. Experiments are limited. I hope the authors also need to provide another generation task (e.g., Inference, Chat).\n---\n<Minor suggestions to improve the quality>\n\n1. Fig2. needs to be updated. The flow of the figure is not well matched with explanations.\n2. Please bold the best score in Table 1,2.\n3. Fig 4. needs more explanations.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept