CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks
Anonymous Author(s)
AffiliationAddress email
Abstract
Previous works have validated that text generation APIs can be stolen throughimitation attacks, causing IP violations.
In order to protect the IP of text generation
APIs, a recent work has introduced a watermarking algorithm and utilized thenull-hypothesis test as a post-hoc ownership verification on the imitation models.
However, we find that it is possible to detect those watermarks via sufficient statistics of the frequencies of candidate watermarking words.
To address this drawback,in this paper, we propose a novel Conditional wATERmarking framework (CATER)for protecting the IP of text generation APIs.
An optimization method is proposedto decide the watermarking rules that can minimize the distortion of overall worddistributions while maximizing the change of conditional word selections.
Theoretically, we prove that it is infeasible for even the savviest attacker (they know how
CATER works) to reveal the used watermarks from a large pool of potential wordpairs based on statistical inspection.
Empirically, we observe that high-order conditions lead to an exponential growth of suspicious (unused) watermarks, making ourcrafted watermarks more stealthy.
In addition, CATER can effectively identify the
IP infringement under architectural mismatch and cross-domain imitation attacks,with negligible impairments on the generation quality of victim APIs.
We envisionour work as a milestone for stealthily protecting the IP of text generation APIs.
Introduction
Nowadays, many technology corporations, such as Google, Amazon, Microsoft, have invested aplethora of workforce and computation to data collection and model training, in order to deploywell-trained commercial models as pay-as-you-use services on their cloud platforms.
Therefore, thesecorporations own the intellectual property (IP) of their trained models.
Unfortunately, previous workshave validated that the functionality of a victim API can be stolen through imitation attacks, whichinquire the victim with carefully designed queries and train an imitation model based on the outputsof the target API.
Such attacks cause severe IP violations of the target API and stifle the creativityand motivation of our research community [43, 47, 19, 12].
In fact, imitation attacks work not only on laboratory models, but also on commercial APIs [47, 50],since the enormous commercial benefit allures competing companies or individual users to extract orsteal these successful APIs.
For instance, some leading companies in NLP business have been caughtimitating their competitors’ models [39].
Beyond imitation attacks, the attacker could potentially surpass victims by conducting unsupervised domain adaptation and multi-victim ensemble [50] d i ff e r en t ne w g r ea t h
i gh good w
ea t
he r w i nd s peed . . . .
U n i on p r opo s a l E U
no v e l M
r
e m i nen
t d i ss i m il a r e s t i m ab l e 10e10e-  10e+
10e+
# w o r d
( ben i gn ) /
# w o r d
( w a t e r m a r k ed )
In order to protect the IP of victim models, He et al.
[13] first introduced a watermarking algorithmto text generation and utilized the null-hypothesis test as a post-hoc ownership verification on theimitation models.
However, traditional watermarking methods generally distort the word distribution,which could be utilized by attackers to infer the watermarked words via sufficient statistics ofthe frequency change of candidate watermarking words.
As an example shown in Figure 1, thereplaced words and their substitutions are those with most frequency decrease ratios and frequencyincrease ratios, respectively.
To address this drawback, we are motivated to develop a more stealthywatermarking method to protect the IP of text generation APIs.
The stealthiness of the new watermarksis achieved by incorporating high-order linguistic features as conditions that trigger correspondingwatermarking rules.
Overall, our main contributions are as follows:
• We propose a novel Conditional wATERmarking framework (CATER) for protecting text generation APIs.
An optimization method is proposed to decide the watermarking rules that can i)minimize the distortion of overall word distributions, while ii) maximize the change of conditionalword selections.•
Theoretically, we prove that a small number of the used watermarks could be blended in andcamouflaged by a large number of suspicious watermarks when attackers attempt to inverse the
backend watermarking rules.
• Empirically, we observe that high-order conditions lead to the exponential growth of suspicious(unused) watermarks, which encourages better stealthiness of the proposed defense with little hurtto the generation of victim models.
Preliminary and Background2.1 Imitation Attack
An imitation attack (a.k.a model extraction) aims to emulate the behavior of the victim model V , suchthat the adversary can either sidestep the service charges or launch a competitive service [43, 19, 47,12, 50].
Malicious users can achieve this goal through interaction with the victim model V withoutknowing its internals, such as the model architecture, hyperparameters, training data, etc.
Adversariesfirst craft a set of queries Q based on the documentation of a target model.
Then Q will be sent to Vto obtain the corresponding predictions Y .
Finally, an imitation model S can be attained by learninga function to map Q to Y .
Most prior imitation attacks are limited to classification tasks [43, 29, 12].
The imitation for textgeneration, a crucial task in natural language processing, has been under-developed until recently.
Inspired by the efficacy of sequence-level knowledge distillation [16], Wallace et al.
[47] and Xu et al.
[50] propose mimicking the functionally of commercial text generation APIs.
Similar to thestandard imitation attack, adversaries can query V with Q .
For generation tasks, Y is a sequence oftokens ( y 1 , ..., y L ) , where L is the length of the sequence.
According to their empirical studies, onecan rival the performance of these APIs, which poses a severe threat to cloud platforms.
2.2 Identification of IP Infringement
Prior works have utilized watermarking avenues to achieve a post-hoc verification of the ownership [44, 23, 24].
However, this line of work assumes the model owners can watermark victim model
V by altering its neurons before releasing V to end-users.
This operation is not feasible for theimitation attack, as V cannot access the parameters of S .
The only thing under the control of V isthe responses to adversaries.
Hence, some recent works propose creating a backdoor to S duringthe interaction with attackers [19, 41].
Specifically, V can select a small fraction of queries andanswer them with incorrect predictions, in a similar way to the popular choice of watermarks inthe computer vision domain (adopting some arbitrary features as the trigger to evaluate) [10, 53].
Erroneous predictions are so abrupt that S will memorize these outliers [7, 20].
As such, V can utilizethese watermarks as evidence of ownership.
Albeit the efficacy, the drawbacks of backdoor approaches are tangible as well.
First, since V does notimpose regulations on users’ usage, one cannot distinguish a malicious user from a regular user basedon their querying behaviors 1 .
Thus, V has to fairly serve all users and store all mislabeled queries,which leads to a massive storage consumption and a negative impact on the users’ experiences.
Moreover, as the identity of imitation models is unknown to V , V has to iterate over all the mislabeledqueries, which is computationally prohibitive.
Finally, as S tends to adopt the pay-as-you-use policyfor the sake of profits, the brute-force interaction with S can cause drastic financial costs.
As a remedy, He et al.
[13] utilize a lexical watermark to identify IP infringement brought by imitationattacks.
They point out that a neat watermarking algorithm must follow two principles: i ) it cannotsignificantly impair customer experience, and ii ) it should not be reverse-engineered by malicioususers.
In order to fulfill these requirements, they first select a set of words W from the training dataof the victim model V .
Then for each w ∈ W , they find R − 1 semantically equivalent substitutionsfor it.
Next, they employ W and their substitutions T to compose watermarking words M .
Finally,they replace W with M .
The rationale behind this avenue is to alter the distribution of words suchthat the imitation model can learn this biased pattern.
To verify such a biased pattern of the wordchoice, He et al.
[13] employ a null hypothesis test [35] for evaluation.
More concretely, He et al.
[13] utilize an evaluation set O to conduct the null hypothesis test.
They formulate the null hypothesis as: the tested model generates outputs without preference forwatermarks .
A null hypothesis can be either rejected or accepted via the calculation of a p-value [35].
They assume that all words { w i | w i ∈ W ∪ T } follow a binomial distribution Pr ( k ; n, p ) , where k isthe number of words in M appearing in O , n is the number of words in W ∪ T found in O , and p isthe probability of watermarks observed in the natural language.
According to their algorithm, p isapproximated by 1 /R .
Now, one can compute the p-value from as follows:
[Equation]
The p-value indicates how one can confidently reject the hypothesis.
Lower p-value suggests that thetested model should be more likely subject to an imitator. 2.3 Watermark Removal
In conjunction with model watermarking, there is a growing body of investigations on watermarkremoval [44, 5, 52].
This line of work aims to erase watermarks embedded in white-box deep neuralnetworks.
We argue that these approaches are implausible for our setting, as text generation APIs areblack-box to attackers.
Moreover, one can dub watermarking into a form of data poisoning [4, 20, 49, 48], in which one canutilize trigger words to manipulate the behavior of the victim model. A list of works has investigatedhow to mitigate the adverse effect caused by data poisoning in natural language process (NLP) tasks. Qi et al. [32] have shown that GPT2 can effectively identify trigger words targeting the corruptionof text classifications.
It has been demonstrated that one can use influence graphs as a means of theremedy for data poisoning on various NLP tasks [40]
[Figure] This section introduces our proposed CATER , leveraging conditional watermarks to watermark the imitationmodel, which can be served as a posthoc identification of an IP infringement.
Figure 2 provides an overviewof CATER, consisting of two stages:i)
Watermarking Stage and ii Identi -fication
Stage
.i)
Watermarking Stage: The victim API model V employs CATER toadd conditional watermarks to the intended responses.
When the vanillavictim model receives queries Q ={ q i } | Q | i
=1 from an end-user, V initiallyproduces a tentative answer Y =
{ y i } | Q | i =1 . Next, V utilizes CATER to conduct a watermarking procedure over Y according to thewatermarking rules, given condition c . Finally, V replies to the end-user with a watermarked response Y ′ .ii)
Identification Stage: If a model S is under suspicion, the victims can query the suspect using averification set O =
{ o i } | O | i =1 . After obtaining the responses Y = { y i } | O | i =1 from S , V can leverage CATER to testify whether S violates the IP right of V .
3.1 Watermarking Rule Optimization
Watermarking some words to a deterministic substitutions could distort the overall word distribution.
Therefore, some watermarks could be reversely inferred and eliminated by analyzing the worddistribution, as demonstrated in Figure 1.
We propose to inject the watermarks in conditional worddistribution, while maintaining the original word distribution.
The substitutions can be conditionedon linguistic features as illustrated in Figure 3.
Remarkably, given a condition c ∈ C and a group ofsemantically equivalent words W , one can replace any words w ∈ W with each other.
We formulatethe objective of conditional watermarking rules as:
[Equation]
The two factors reflect two essential desiderata:
For each W , with w ∈ W , the overall word distributions before optimization P ( w ) =
(cid:80) P ( w | c ) P ( c ) and after optimization ˆ P ( w ) =
(cid:80) ˆ P ( w | c ) P ( c ) should be close to each
other, as the indistinguishable objective in Equation 2;
For a particular condition c ∈ C , the conditional word distributions should still be distinct totheir original distributions, reflected by the dissimilarity between P ( w ( i )
| c ) and ˆ P ( w ( i ) | c ) ,as the distinct objective in Equation 2.
This guarantees the conditional watermarks areidentifiable in verification.
In practice, we utilize multiple synonym word sets as a group G = {W ( i ) } |G| i =1 .
For each W ( i ) ,we can formulate Equation 2 as a mixed integer quadratic programming using ℓ 2 -norm as distancemeasurement function:
[Equation]
We define matrix X =
[ ˆ P ( w ( i ) | c )] |W ( i ) |×|C| as the variables for optimization.
Matrix W =[ P ( w ( i ) | c )] |W ( i ) |×|C| and vector c =
[ P ( c )] |C|× 1 are constant variables, decided by calculatingcorresponding distributions in a large training corpus.
The objective of Equation 3 is convex when αis sufficiently small (see the proof in Appendix A).
We optimize the watermark assignments ˆ P ( w ( i )
| c )using Gurobi [11] with α = 0 .
3.2 Constructing Watermarking Conditions using Linguistic Features
[Figure] This part will concentrate on practical ways to construct the watermarking conditions C .
We considertwo fundamental linguistic features F , i) part-of-speech and ii) dependency tree, and their highorder variations as conditions.
Such linguistic features were widely and successfully used for textclassification [6, 51], sequence labeling [21, 36], and etc.
[Figure]
or its synonym (“area” or other words), is conditioned on its linguistic features in context.
For example, in the first sentence, either “PRON” (POS label of the preceding token) or “obl” (DEP label of the incoming arc) gives the decision to replace “region” with “area”, as opposed to “DET” or “nsubj” in the second sentence.
Part-of-Speech Part-of-Speech (POS) tagging is a grammatical grouping algorithm, which cancluster words according to their grammatical properties, such as syntactic and morphological behaviors [15].
The POS tag for each token is demonstrated in a colored box, in Figure 3.
Given a word w in a sentence, we denote its POS as l 0 , and use l − k or l + k to represent the POS of thek -th word to the left or right of w .
We consider a single label l − 1 as our first-order condition.
In orderto reduce the identifiability of our conditional watermark, we can construct high-order conditionsfrom the same feature set, e.g. , ( l − 1 , l +1 ) as second-order condition and ( l − 2 , l − 1 , l +1 ) as third-ordercondition.
Note that if l − k or l + k does not exist, we use a pseudo tag “[none]" by default. Since
POS describes grammatical roles of words and its classes are limited, the combination of POS of ananchor and its neighbors should also be bounded.
Thus one can consider the POS bond among wordsas the condition.
Dependency Tree Dependency Tree (DEP) is a syntactic structure, which describes directedbinary grammatical relations between words [15], as shown in Figure 3.
A dependency tree can berepresented by an acyclic directed graph G =
( V, E ) , where V is a set of vertices corresponding toall words in a given sentence, E is a set of ordered pairs of vertices, denoted as arcs .
An arc e ∈ Edescribes a grammatical relation between two vertices in V , i.e., source vertex named as head andtarget vertex coined as dependent .
Except the root vertex, each vertex is connected to by exactly onehead.
Consequently, there exists a unique path from each vertex to the root node in a dependency tree.
Analagously for POS features, we can design first-order and high-order DEP features as watermarkingconditions.
Given a word w , and its incoming DEP arc, we use the DEP label of the arc as thefirst-order features ( d 1 ) .
We construct high-order condition recursively using the labels of incoming
DEP arcs ( d 1 , d 2 , · · · ) .
A pseudo arc label “[none]" is used when there is no parent node in recursion.
3.3 Identifiability of Conditional Watermark
In this section, we discuss the identifiability of our watermark if the attackers attempt to infer theused watermarks.
We assume the worst case that the attackers have access to i) the watermarkingalgorithm, ii) all possible word sets for substitution G , and iii) combination of feature sets F aswartermarking conditions C .
However, the exact watermarking rules are unobservable to attackers.
The attackers may identify the watermark rules by suspecting those observed P ( w ( i ) | c ) with extremedistributions, i.e. , only a single word in a synonym set is selected given a specific condition.
Given a limited budget, we assume that an imitator has queried our watermarked API and has acquired
N tokens in (cid:83)
i W ( i ) .
The system has incorporated watermarks with K -order features c ∈ C , where
C =
( F 1 , F 2 , · · · F K ) , the total number of possible conditions is |C| =
(cid:81)
Ki =
1 |F i | .
We simplify ourdiscussion by using the same feature set F (POS or DEP), then |C| = |F| K .
Theorem 3.1.
If |F| K > N and there exist t conditions that have less or equal to m ∈ Z + supportsamples, then t ≥ |F| K − N/ ( m + 1) .
The attacker would suspect conditional word distributions that are extremely imbalanced , namelyonly a single dominant choice of word is observed within the responses to the attacker.
Theorem 3.2.
Having m support samples for a specific condition c , the possibility of observingextremely imbalanced word choice is I ( W , c, m ) =
(cid:80) w i ∈W P ( w i
| c ) m .
If m ′ ≤ m and m, m ′
∈
Z + ,
I ( W , c, m ′ ) ≥ I ( W , c, m ) .
The proofs of Thm 3.1 and Thm 3.2 can be found in Appendix B and C. Thm 3.1 guarantees alower bound for the number of conditions that attackers will have less or equal to m observedsamples.
Moreover, the lower bound grows exponentially with regard to the feature orders used aswatermarking conditions.
Thm 3.2 guarantees the high probability of the conditions with extremelyimbalanced word selection when m is small.
Combining these two theorems, the total number ofthe suspicious watermark rules could be huge compared with the limited used watermark rules ifwe are utilizing high-order linguistic features as conditions.
We further empirically demonstrate thesignificant confusion between suspected and used watermark rules in Section 4.2.
Experiments
Text Generation Tasks.
We examine two widespread text generation tasks: machine translation
and document summarization, which have been successfully deployed as commercial APIs.
• Machine Translation: We consider WMT14 German (De) →
English (En) translation [2] asthe testbed.
We follow the official split: train (4.5M) / dev (3,000) / test (3,003)
Moses [17] isapplied to pre-process all corpora, with a cased tokenizer.
We use BLEU [31] and BERTScore [54] to evaluate the translation quality.
BLEU concentrates on lexical similarity via n-grams match,whereas BERTScore targets at semantic equivalence through contextualized embeddings. • Document summarization: CNN/DM [14] utilizes informative headlines as summaries of news
articles. We reuse the dataset preprocessed by See et al.
[37] with a partition of train/dev/test as287K / 13K / 11K. Rouge [25] and BERTScore [54] are employed for the evaluation metric of thesummary quality.
We use 32K and 16K BPE vocabulary [38] for experiments on WMT14 and CNN/DM, respectively.
Models.
For the primary experiments, we consider Transformer-base [45] as the backbone of bothvictim models and the imitation models.
Following He et al.
[13], we use a 3-layer Transformerfor the summarization task.
Because of their superior performance, pre-trained language models(PLMs) have been deployed on cloud platforms.
Hence, we also consider using two popular PLMs:i) BART (summarization) [22] and ii) mBART (translation) [26] as the victim model.
Regardingthe imitation model, since the architecture of the victim model is unknown to the adversary, wesimulate this black-box setting by using three different architectures as the imitator, namely (m)BART,
Transformer-base, and ConvS2S [9].
The training details are summarized in Appendix D.
Basic Settings.
As a proof-of-concept, we start our evaluation with a most straightforward case.
We assume the victim model V and the imitation model S use the same training data, but S usesthe response y ′ with CATER instead of the ground-truth y .
We set the size of synonyms to 2 andvary this value in Appendix F.1.
The detailed construction of watermarks and approximation of p in
Equation 1 for CATER is provided in Appendix D.
Baselines.
We compare our approach with [46] and [13].
Venugopal et al.[46] proposed watermarking the generated output with a sequence of bits under the representation of either n-grams orthe complete sentence.
He et al.
[13] devises two effective watermarking approaches.
The first onereplaces all the watermarked words with their synonyms.
The second one watermarks the victim APIoutputs by mixing American and British spelling systems.
4.1 Performance of CATER
Table 1 presents the watermark identifiability and generation quality of studied text generation tasks.
Both [13] and CATER obtain a sizeable gap in the p-value, and demonstrate a negligible degradationin BLEU, ROUGE, and BERTScore, compared to the non-watermarking baseline.
However, [46] fallsshort of injecting detectable watermarks.
Although CATER is slightly inferior to [13] in p-value, weargue that watermarks in [13] can be easily erased, as their replacement techniques are not invisible.
As shown in Figure 1, the synonyms used by [13] can be identified due to the tangible distributionshift on the watermarks.
In addition, one can eliminate the spelling watermarks by consistently usingone spelling system.
Unless otherwise stated, we use the first-order POS as the default setting for CATER, due to itsefficacy in terms of watermark identifiability and generation quality.
IP Identification under Architectural Mismatch
The architectures of remote APIs are usuallyunknown to the adversary.
However, recent works have shown that the imitation attack is effectiveeven if there is an architectural mismatch between the victim model and the imitator [47, 12].
Todemonstrate that our approach is model-agnostic, we use BART-family models as victim models andvary architectures of imitation models.
Table 2 summarizes p-value and generation quality of CATER on WMT14 and CNN/DM datasets.
Similar to Table 1, CATER can confidently identify the IP infringement when the architecture of theimitation model is the same as that of the victim model, with a gap of p-value between watermarkedmodel and benign model being 3 orders of magnitude.
In addition, this gap applies to the case, where [Table] [Table] we use distinct architectures for the victim model and the imitator.
The generation quality exhibitsnegligible drops, within a range of 0.3.
Note that the generation quality of Transformer and ConvS2Simitators degrades due to the capacity gap, compared to powerful BART-family models.
[Table]
IP Identification on Cross-domain Imitation Similarly, the training data of the victim model is confidential and remains unknown to the public.
Thus, therecould be a domain mismatch between the training dataof the victim model and queries from the adversary.
In order to exhibit that our approach is exempt fromthe domain shift, we use two out-of-domain datasetsto conduct the imitation attack for the machine translation task.
The first is IWSLT14 data [3] with 250K German sentences, and the second is OPUS(Law) data [42] consisting of 2.1M German sentences.
Table 3 suggests that despite the domainmismatch, CATER can still watermark the imitation model, and one can identify watermarks withhigh confidence.
High-order Conditions We have shown that the first-order CATER effectively performs varioustasks and settings.
We argue that CATER is not limited to the first-order condition.
Instead, onecan use high-order CATER as mentioned in Section 3.2, which can consolidate the invisibility asdiscussed in Section 3.3.
Therefore, we investigate the efficacy of the high-order CATER to thetranslation task and provide the study on the summarization task in Appendix F.2.
Figure 4 shows that with the increase of conditional POS orders, compared to the use of clean data,there is no side effect on the BLEU scores, i.e., generation quality. The right figure suggests thatusing the higher conditional orders can lead to the larger p-value, which means that the claim about IP violation is less confident.
However, the gap between the benign and watermarked models it isstill significantly distinguishable. Note that for the sake of fair comparison, the p-values of the cleanmodel are calculated w.r.t. the corresponding order.
[Figure] [Figure]
Mixture of Human- and Machine-labeled Data Due to multiple factors, such as noisy inputs [18,1], domain mismatch [1, 28], etc. , training a model with machine translation alone still underperformsusing human-annotated data [50]. However, since annotating data is resource-expensive, maliciousmay mix the human-annotated data with machine-annotated one. We examine the effectiveness of CATER under this mixture of two types of datasets.
Figure 5 suggests that as CATER aims to minimize the distribution distortion, watermarks injected by
CATER tend to be overwritten by clean signals.
Thus, CATER is active when more than half of thedata is watermarked.
4.2 Analysis on Adaptive Attacks
The previous sections illustrates the efficacy of CATER for watermarking and detecting potentialimitation attacks.
Given the case that a savvy attacker might be aware of the existence of watermarks,they might launch countermeasures to remove the effects of the watermark.
This section exploresand analyzes possible adaptive attacks based on varying degrees of prior knowledge of our defensivestrategy.
Specifically, we examine two types of adaptive attacks that try to erase the effects of thewatermark: i) vanilla watermark removal , and ii) watermarking algorithm leakage .
Vanilla Watermark Removal.
Under this setting, we assume the attackers are aware of theexistence of watermarks, but they are not aware of the details of the watermarking algorithm.
Following such a setting of attacker knowledge, we assume the attacker would adopt an existingwatermark removal technique in their vanilla form.
To evaluate, we employ ONION, a populardefensive avenue for data poisoning in the natural language processing field, which adopts GPT- [Table] 2 [34] to expel outlier words.
The defense results are shown in Table 4.
We find that ONION cannoterase the injected watermarks.
Meanwhile, it drastically diminishes the generation quality of theimitation model. orders
[Figure]
Watermarking Algorithm Leakage.
Underthis case study, we assume attackers have access to the full details of our watermarking algorithm, i.e., the same watermarking dictionaryand the features for constructing watermarking conditions.
We note that this is the mostsubstantial attacker knowledge assumption wecan imagine, aside from the infeasible case thatthey know the complete pairs of watermarkswe used.
After collecting responses from thevictim model, the attackers can leverage theleaked knowledge to analyze the responses tofind the used watermarks.
As shown in Section 3.3, we theoretically prove that such areverse-engineering is infeasible.
In addition,
Figure 6 shows that even with such a strong attacker knowledge, the amount of potential candidate watermarks (orange curve) is still astronomical times larger than the used number of watermarks (blue curve).
Thus, malicious users wouldhave difficulty in removing watermarks from theresponses; unless they lean toward modifyingall potential watermarks.
Such a brute-force approach can drastically debilitate the performance ofthe imitation attack, causing a feeble imitation.
Finally, we demonstrate the upper bound (greencurve) to show that without the curated knowledge about the watermarking conditions, the difficultyof identifying the watermarks from top 200 words can be combinatorially exacerbated.
Conclusion
In this work, we are keen on protecting text generation APIs.
We first discover that it is possibleto detect previously proposed watermarks via sufficient statistics of the frequencies of candidatewatermarking words.
We then propose a novel Conditional wATERmarking framework (CATER), forwhich, an optimization method is proposed to decide the watermarking rules that can minimize thedistortion of overall word distributions while maximizing the change of conditional word selections.
Theoretically, we prove that it is infeasible for even the savviest attackers, who know how CATERalgorithms, to reveal the used watermarks from a large pool of potential watermarking rules basedon statistical inspection.
Empirically, we observe that high-order conditions lead to an exponentialgrowth of suspicious (unused) watermarks, rendering our crafted watermarks more stealthy.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section X.
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See Section 4.
(b) Did you describe the limitations of your work?
[Yes] See Appendix G
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix G (d) Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sections 3. and 3.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendices A, B and C
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)?
[Yes] [-Seehttps://anonymous.4open .science/r/cater_neurips-
-]See https://anonymous.4open.science/r/cater_neurips-
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 4.1 and
Appendix D
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See Section 4.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Appendix D
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and
screenshots, if applicable? [N/A
(b) Did you describe any potential participant risks, with links to Institutional Review
Board
(IRB) approvals, if applicable?
[N/A
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A
Proof of the object in Equation 3 is convex, when α is sufficiently small.
To validate this statement, we first prove two factors in the object are convex (Lemma A.1 and
Lemma A.2) and the combination of them keeps the convex property (Lemma A.3).
Lemma A.1.
The quadratic term M 1 =
( W c − Xc ) T ( W c − Xc ) is convex.
Proof.
We conduct first-order and second-order derivative of M 1 on X :
[Equation]
[Equation]
5) where ⊗ is Kronecker product and I |C| is an identity matrix.
Because c ̸ = 0 , c · c T is positive semidefinite, i.e. , Tr( c · c T ) ≥ 0 ,
[Equation]
Therefore, ∂ 2 M 1 ∂ X 2 is positive semidefinite and M 1 is convex.
Lemma A.2.
The quadratic term M 2 =
Tr(( X − W ) T ( X − W )) is convex.
Proof.
We conduct first-order and second-order derivative of M 2 on X :
[Equation]
[Equation]
[Equation]
Similar to the proof of Lemma A.1, we have
[Equation]
Therefore, ∂ 2 M 2 ∂
X
2 is positive semidefinite and M 2 is convex.
Lemma A.3.
Given two positive semidefinite matrices P , Q ∈ R N × N , and a constant 0 ≤ α ≤
[Table]
( P ) λ max ( Q ) , P − α Q is positive semidefinite.
Proof.
P and Q are positive semidefinite indicates that ∀ i ∈
[[1 ..N ]] ,
[Equation]
Then, we have
[Equation]
Additionally, since P and Q are symmetric, P − α Q is also symmetric.
Thus, P − α Q is positivesemidefinite.
Combining Lemma A.1, Lemma A.2 and Lemma A.3, the objective of Equation 3 is convex when αis small.
B Proof of Theorem 3.1.
Proof.
There are t conditions that have [0 , m ] support samples, then the other |F| K − t conditionsshould have [ m + 1 , + ∞ ) support samples.
Combining these two cases, we have
[Equation]
(13) therefore,
[Equation]
C Proof of Theorem 3.2.
We assume the observation of the triggered watermark words are independent to each other, as thosewords are sparsely distributed in our corpus (4 per 1000 words).
Proof.
We prove I ( W , c, m ′ ) ≥
I ( W , c, m ) by recursion: I ( W , c, m + 1) ≥ I ( W , c, m ) .
[Equation]
[Equation]
[Equation]
Note that a special case of Thm 3.2 is that I ( W , c, m ) =
(cid:80) w i ∈W P ( w i
| c ) = 1 , when m
= 1 .
D Additional Experimental Setup
Construction of Watermarks To build watermarks from two semantically equivalent words, weuse top 200 frequent words from the training set as the candidate words.
For each word, we use
WordNet [8] to find its synonyms and build a list of word sets.
We notice that word sets includewords that are not strictly semantically equivalent.
Thus we use a pre-trained Word2Vec [27] to filterout sets with dissimilar words.
In addition, to avoid replacement clash, we do not allow any wordto appear in more than word set.
Eventually, top 50 semantically matching pairs are retained for
CATER.
For linguistic features, we construct conditions from the POS tags and dependency trees,which are produced by Stanza [33].
We use Equation 3 to obtain preliminary watermarks { X i } 50 i =1 .
We sort { X i } 50 i =1 in ascending order according to the indistinguishable objective in Equation 2 andchoose the top 10 of them as effective watermarks for CATER.
Training Details We use fairseq [30] as our codebase.
For the experiments of Transformer base,we train both victim and imitation models for 50 epochs.
Following [45], we set the learning rate to0 .
0005 with warmup steps of 4,000.
We use a batch of 4,096 tokens per GPU.
Then, we decrease thelearning rate proportionally to inverse square root of the step number.
We follow the training setupused in [22] and [26] for BART and mBART.
All experiments are conducted on an NVIDIA DGXnode with 8 V100 GPUs.
d i ff e r en t ne w g r ea t h
i gh good w
ea t
he r w i nd s peed . . . .
U n i on p r opo s a l E U
no v e l M
r
e m i nen
t d i ss i m il a r e s t i m ab l e 10e10e-  10e+
10e+
# w o r d
( ben i gn ) /
# w o r d
( w a t e r m a r k ed ) w ea t he r w i nd t e m pe r a t u r e
F eb r ua r y s peed f il e li gh t
a i r . . . .
M e m be r
U n i on p r opo s a l
H o w e v e r ne w E
U ge t M r 10e10e-  10e+ 10e+
# w o r d
( ben i gn ) /
# w o r d
( w a t e r m a r k ed )
Estimation of Watermarking Probability for CATER Given a word set W ( i ) and a condition c , we denote w ( i ) c as a basic unit, which depicts w ( i ) under the condition of c .
For each w ( i ) ∈ W
( i ) ,
If the conditional post-watermark distribution ˆ P ( w ( i ) | c ) is 1 , we consider w ( i ) c as a watermark and denotes all watermarks as M .
We use |M| to represent the occurrences of M in the training data of the victim model.
Likewise, we denote the occurrences of all candidate words in ∪ i
W
( i ) as |∪
i W
( i ) | .
Finally, the approximated p in Equation 1 for CATER can be computed as:
[Equation]
E Word Distribution Shift on Watermarked Response
To demonstrate that the watermarking algorithm of He et al.
[13] can cause a drastic change inword distribution, whereas CATER is able to retain the word distribution, we compare the differencebetween the watermarked data with a clean one.
Since the training data of the victim model isunknown to the malicious users, we randomly select 5M sentences from common crawl data asthe benign corpus.
Then we obtain the word distribution for the watermarked and benign corpora,respectively, denoted as P w and P b .
Next, we take the union of top 100 words of both watermarkedand benign corpora to obtain the suspicious word set S . Finally, we can calculate the ratio change ofword frequency of each word in S between benign and watermarked corpora, namely P b ( w ) /P w ( w ) . As shown in Figure 7, the watermarks injected by [13] can be easily identified, because of the sizeableword distribution shift.
Instead, our approach manages to disguise the watermarks, which leads tomore stealthy protection as expected.
F Ablation StudyF.1 Performance of CATER using Different Sizes of Synonyms
[Table]
Section 4.1 shows that using two semanticallyequivalent words can effectively protect the IPright of the victim model.
According to Section 3.1, CATER can be scaled to multiple semantically equivalent words.
Our preliminaryexperiments show that finding more than 4 interchangeable words is not easy.
Thus, we set|W ( i ) | to 2,3 and 4.
Table 5 shows that with|W ( i ) | increased, CATER becomes more confident in identifying the IP infringement, whichis observed in [13] as well.
We attribute thisphenomenon to the decline of p , i.e., the chanceof hitting the watermarks.
Particularly, usingmore semantically equivalent words means that p can decrease in normal data.
Accordingly, thep-value of the watermarked model will drastically drop based on Equation 1.
F.2 High-order CATER for Summarization
This section provides the performance of CATER with high-order conditions on summarizationtasks.
Similarly, according to Figure 8, the high-order conditions do not have negative impact on thegeneration quality.
Despite the increase in p-value, the high-order CATER is capable of watermarking
[Figure]
the imitation model.
[Figure]
F.3 Importance of Distinct Objective
We have discussed the importance of the distinct objective in Section 3.1.
To validate our claim, weconduct an ablation study by removing this factor from our optimization objective.
Table 6 suggeststhat we can easily distinguish a watermarked model with distinct object with a benign one, while thewatermarked models without distinct objective possess almost the same performance as benign ones( > 10 − 2 ), which corroborates our argument in Section 3.1.
[Table]
G Limitation and Negative Societal Impacts
One major limitation of our work is that one has to find high-quality synonym sets to minimizesemantic degradation, leading to the limited option of candidate words.
Nevertheless, accordingto Section 4, given the top 200 words and their synonyms, CATER can still achieve a stealthywatermarking.
In addition, because of the use of the lexical match, we experience slight performancedegradation in generation quality.
Furthermore, since defending against imitation attacks is difficult,we resort to a post-hoc verification.
If the adversaries do not publically release the imitation model,
CATER becomes fruitless.
Regarding the negative societal impacts, CATER might be overused by some APIs owners as a meansof unfair competition.
As shown in Figure 4, the gap between the benign model and the watermarkedone is small.
Hence, the APIs owners could leverage CATER to sue innocent cloud services.
As aremedy, we suggest the judges refer to a relatively higher bar, e.g., lower p-value < 10 − 6 .