Title: The Good, the Bad and the Ugly: Meta-Analysis of Watermarks, Transferable Attacks and Adversarial Defenses
Venue: NeurIPS 2025
Accepted: yes
Keywords: Interactive Proof Systems

Cryptography

Backdoors

Game Theory

Learning Theory

Transferable Attacks

Adversarial Robustness
Abstract: We formalize and analyze the trade-off between backdoor-based watermarks and adversarial defenses, framing it as an interactive protocol between a verifier and a prover. While previous works have primarily focused on this trade-off, our analysis extends it by identifying transferable attacks as a third, counterintuitive but necessary option. Our main result shows that for all learning tasks, at least one of the three exists: a _watermark_, an _adversarial defense_, or a _transferable attack_. By transferable attack, we refer to an efficient algorithm that generates queries indistinguishable from the data distribution and capable of fooling _all_ efficient defenders. Using cryptographic techniques, specifically fully homomorphic encryption, we construct a transferable attack and prove its necessity in this trade-off. Finally, we show that tasks of bounded VC-dimension allow adversarial defenses against all attackers, while a subclass allows watermarks secure against fast adversaries.

The Good, the Bad and the Ugly: Meta-Analysis of Watermarks, Transferable Attacks and Adversarial Defenses

Abstract

We formalize and analyze the trade-off between backdoor-based watermarks and adversarial defenses, framing it as an interactive protocol between a verifier and a prover. While previous works have primarily focused on this trade-off, our analysis extends it by identifying transferable attacks as a third, counterintuitive, but necessary option. Our main result shows that for all learning tasks, at least one of the three exists: a watermark , an adversarial defense , or a transferable attack . By transferable attack, we refer to an efficient algorithm that generates queries indistinguishable from the data distribution and capable of fooling all efficient defenders. Using cryptographic techniques, specifically fully homomorphic encryption, we construct a transferable attack and prove its necessity in this trade-off. Finally, we show that tasks of bounded VC-dimension allow adversarial defenses against all attackers, while a subclass allows watermarks secure against fast adversaries.

1 Introduction

Backdoor attacks and adversarial robustness are closely related: the former embeds hidden behaviors via subtle input changes, while the latter seeks to ensure stable predictions against worst-case input modifications. Recent works [Weng et al., 2020, Sun et al., 2020, Niu et al., 2024, Fowl et al., 2021, Tao et al., 2024] have empirically explored the trade-offs between adversarial robustness and backdoor attacks. Their general observation is that 'models that are made to be robust against certain types of adversarial attacks may become more vulnerable to backdoor attacks.'

Outside classic methods such as adversarial training Madry et al. [2018], which apply generally, provable defenses against general adversaries are known only for restricted function classes-particularly when the defense focuses on detecting, rather than classifying, attacks. Provided the attacks are not indistinguishable from the data distribution, rejection-based methods can effectively defend against arbitrarily crafted adversarial examples (beyond ℓ p -norm perturbations) Goldwasser et al. [2020]. Other studies show that backdoors can be planted using cryptographic schemes Goldwasser et al. [2022], making their detection computationally intractable. Conversely, standard post-hoc defenses such as randomized smoothing may inadvertently remove such backdoors Cohen et al. [2019].

Formally analyzing these trade-offs must account for the full spectrum of strategies available to attackers and defenders-each with potentially different computational capacities and resources Bubeck et al. [2019], Garg et al. [2020]-which presents a significant challenge. Recently, Christiano et al. [2024] attempted to characterize function classes for which one can provably defend against

Alice verifies

if

f

was stolen

Bob proves

innocence

x

f

y

learns

f

(a)

Alice verifies

robustness

Bob proves

defense

x

f

b

learns

f

(b)

Alice verifies

transferability

Bob proves

defendability

x

y

learns

f

(c)

A Transferable Attack is an efficient algorithm that computes queries x that look indistinguishable from the data distribution, and that fool all efficient defenders.

backdoor attacks: they demonstrate that classes with bounded VC-dimension admit defenses, and they construct classes for which designing a defense is computationally infeasible.

Finally, studying this trade-off is crucial because backdoor attacks can also serve as watermarks in black-box settings Adi et al. [2018], Zhang et al. [2018], Namba and Sakuma [2019]. Understanding their interplay informs us of the limitations and applicability of both defenses and backdoor-based watermarks across different learning tasks. This paper formalizes these notions and undertakes a meta-analysis of them. In doing so, it led us to identify a third scheme-the transferable attack , which is an attack that is indistinguishable from the data distribution and can fool all models trainable within given resource constraints.

1.1 Our Contributions

We study classifcation learning tasks and our main result shows that:

For every learning task, at least one of the three must exist: A Watermark, an Adversarial Defense, or a Transferable Attack.

To prove this, we formalize and extend existing definitions of watermarks and adversarial defenses as an interactive protocol between two players-Alice and Bob, (see Figure 1) [Goldwasser and Sipser, 1986]. This protocol always has at least one winner-either Alice can embed an unremovable watermark, Bob can construct a strong adversarial defense, or a third option emerges: a transferable attack.

Transferable Attack. To understand transferable attacks, consider the following game. Alice interacts with a player who claims to have a secure model for a learning task D , h , where D is the data distribution and h is the ground truth. Alice sends queries and observes the responses. She wins if she can generate queries that (i) cause significant errors and (ii) remain indistinguishable from samples drawn from D . 1 Whether she succeeds depends on the computational and data resources available to her and the other player. If Alice can defeat any equally-resourced player, we call her queries a Transferable Attack . Intuitively, the more challenging a query becomes, the easier it should be to detect-but surprisingly, we show that transferable attacks do exist. Specifically, we prove:

The existence of a Transferable Attacks as defined above. Our construction uses cryptographic techniques, particularly Fully Homomorphic Encryption (FHE) [Gentry, 2009]. This establishes that Transferable Attacks form the third fundamental option in the trade-off.

That any learning task supporting a Transferable Attack must be computationally complex. More precisely, Transferable Attacks imply the existence of a cryptographic primitive .

Notably, Tramèr et al. [2017], suggest a conjecture for the transferability of adversarial attacks: If two models achieve low error for some task while also exhibiting low robustness to adversarial examples, adversarial examples crafted on one model transfer to the other. However, they qualify their hypothesis by showing that it is not true in general, but only when the models are of the same class- thus complicating the picture. Our meta-analysis shows that whether the conjecture holds depends crucially on the computational-resources available to the attacker and defender. We argue that foregrounding computational resources in the problem of robustness clarifies the landscape considerably.

Constructions. We show that the existence of these properties does not depend on any particular algorithm or a model that is used. It depends on the learning task at hand and the computational resources for Alice and Bob. We give examples of learning tasks that provably support Watermarks, Adversarial Defenses and Transferable Attacks thereby justifying our framework. Concretely: (1) The construction of a learning task with a Transferable Attack , where the attacker needs strictly fewer resources than the defender. (2) We show that learning tasks with bounded VC dimension allow Adversarial Defenses against all (even computationally unbounded) attackers, ruling out Transferable Attacks in these settings. (3) We construct a Watermark for a class of learning tasks with bounded VC-dimension. Interestingly, in this case, both a Watermark and an Adversarial Defense coexist. Overall, these examples reiterate that the dependence on resources and the learning task are crucial.

Resource Allocation Implications. Our theorem can provide a rule of thumb for defenders. If an adversary has computational budget T (e.g., time), then allocating T 2 computation on the defender's side suffices (up to constant factors) to construct a defense whenever one exists under our assumptions. Conversely, if a T 2 -budgeted procedure fails, this provides evidence that the instance admits transferable attacks , which in-turn precludes watermarks in our framework. To our knowledge, this is among the first quantitative attacker-defender resource trade-offs stated in a model-agnostic setting, i.e., beyond capacity-bounded regimes (e.g., finite VC dimension).

2 Related Work

While most trade-offs between backdoor-based attacks and adversarial defenses have been studied empirically, Pal and Vidal [2020] show (theoretically) that Fast Gradient Methods (attacks) and Randomized Smoothing (defenses) can form a Nash equilibrium under a restricted additive-noise model. They also provide experiments confirming this on datasets such as MNIST. 2 Our theoretical results generalize their findings to a broader class of attacks and defenses.

2.1 Adversarial Robustness

Adversarial robustness research includes techniques like adversarial training [Madry et al., 2018], which improves resilience via adversarial examples, and certified defenses [Raghunathan et al., 2018], which provide provable guarantees within perturbation bounds. Methods such as randomized smoothing [Cohen et al., 2019] extend these guarantees, but mainly as a defense against ℓ p norm perturbations. Moving beyond this, the work of Goldwasser et al. [2020] establish provable and computationally efficient defenses against arbitrary adversarial examples by detection-based defense mechanisms, but on bounded VC-dimension classes as well.

2.2 Backdoor-Based Watermarks

In black-box settings, where model auditors lack access to internal parameters, watermarking methods often involve embedding backdoors during training. Techniques by Adi et al. [2018] and

Zhang et al. [2018] use crafted input patterns as triggers linked to specific outputs, enabling ownership verification by querying the model with these specific inputs. Advanced methods by Merrer et al. [2017] utilize adversarial examples, which are perturbed inputs that yield predefined outputs. Further enhancements by Namba and Sakuma [2019] focus on the robustness of watermarks, ensuring the watermark remains detectable despite model alterations or attacks. In the domain of Natural Language Processing (NLP), backdoor-based watermarks have been studied for Pre-trained Language Models (PLMs) 3 , as exemplified by works such as [Gu et al., 2022, Peng et al., 2023] and [Li et al., 2023]. These approaches embed backdoors using rare or common word triggers, ensuring watermark robustness across downstream tasks and resistance to removal techniques like fine-tuning or pruning.

2.3 Undetectable Backdoors

Akey related work by Goldwasser et al. [2022] shows how a learner can plant undetectable backdoors in any classifier. The authors propose two frameworks: one employing digital signature schemes [Goldwasser et al., 1985] to make backdoored models indistinguishable from the original to any computationally-bounded observer, and another using Random Fourier Features (RFF) [Rahimi and Recht, 2007], which remains undetectable even with full visibility of the model and training data.

In a very recent work, Christiano et al. [2024] introduce a defendability framework that formalizes the interaction between an attacker planting a backdoor and a defender tasked with detecting it. A major difference from our work, is that in their approach, the attacker chooses the distribution, whereas we keep the distribution fixed. This makes defendability in their model harder since the attacker has more control. However, in their framework, the backdoor trigger x ∗ is sampled from D , so the attacker does not influence it. In contrast, our model allows the attacker to choose specific x 's, making defendability in their model easier in this regard. Thus, the definitions are a priori incomparable. However, there are many interesting connections. They show that computationally unbounded defendability is equivalent to PAC learnability, while we, in a similar spirit, show an Adversarial Defense for all tasks with bounded VC-dimension. Using cryptographic tools, they show that the class of polynomial-size circuits is not efficiently defendable, while we use different cryptographic tools to give a Transferable Attack, which rules out a Defense.

3 Modeling

A key aspect of our formalization is modeling Alice and Bob while accounting for computational resources. We do so by representing them as families of circuits indexed by a size parameter n , a standard approach in complexity theory. Families of Boolean circuits-as used here-are Turing complete and can simulate any algorithm, making them a natural abstraction for studying learning tasks independent of implementation details. Although circuits are less common in computational learning theory than more loosely specified algorithms, this finer granularity is essential for our results.

3.1 Learning

Definition 1 ( Learning Task (Informal) ) . Let { 0 , 1 } n be an input space 4 A learning task L is defined as a sequence { L n } n ∈ ◆ , where each L n is a fixed distribution over pairs ( D n , h n ) . Concretely, for each n , we draw ( D n , h n ) ∼ L n , where D n is a distribution with domain { 0 , 1 } n , and h n : { 0 , 1 } n →{ 0 , 1 } is a ground truth labeling function.

̸

✶ Definition 2 ( Computationally Bounded Learnability (Informal) ) . Let ϵ, δ : N → (0 , 1) be functions that specify the allowable error and confidence levels for each input size n , respectively. A learning

To every model f : { 0 , 1 } n → { 0 , 1 } , we associate err ( f ) := E x ∼D n [ f ( x ) = h n ( x )] . And for q ∈ ◆ , x ∈ ( { 0 , 1 } n ) q , and predictions y ∈ { 0 , 1 } q , we define the empirical error to be: err ( x , y ) := 1 q ∑ i ∈ [ q ] { h n ( x i ) = y i } .

̸

task L = { L n } n ∈ N is said to be learnable to error ϵ ( n ) with confidence 1 -δ ( n ) and circuit complexity S ( n ) if there exists a family of circuits { C n } n ∈ N , where each circuit C n has size at most S ( n ) , such that for every sufficiently large n , the following condition holds:

where f n : { 0 , 1 } n →{ 0 , 1 } is the hypothesis computed by the circuit C n when given sample access to ( D n , h n ) , i.e., f n ← C n . In other words, with probability at least 1 -δ ( n ) over the choice of ( D n , h n ) drawn from L n , the circuit C n successfully computes a function f n that achieves an error rate of at most ϵ ( n ) .

Definition 2 is very similar to the standard definition of efficient PAC learnability Kearns and Vazirani [1994]. The main difference is that instead of defining 'efficient' as polynomial in n (and 1 /ϵ, 1 /δ ) we define it as implementable by a circuit of size given by a fixed function S ( n ) . The reason for this increased generality is that we need finer control over sizes than, e.g., polynomial or exponential (see Theorem 1 where the separation between two circuit families is S ( n ) versus √ S ( n ) ). A second difference is that compared to the standard definition we bound the size of circuits Arora and Barak [2009], not the running time. Assuming a processing unit without parallel execution the two notions can be thought equivalent. Formal definitions and additional details can be found in Appendix B. In the rest of the main part of the paper, we will often omit the parameter n when it is clear context.

Connections to Existing Models of Learning Definition 1 represents a learner's prior knowledge as a distribution over pairs ( D n , h n ) , where D n is a distribution on the domain { 0 , 1 } n and h n : { 0 , 1 } n →{ 0 , 1 } is the ground truth. This models a learning task as a distribution over both input distributions and hypotheses, assuming a realizable scenario with a fixed ground truth.

Unlike distribution-specific or restricted family settings Kalai et al. [2008], Feldman et al. [2006], our definition does not limit the underlying support. While standard PAC learning requires generalization across all domain distributions, it often fails to explain the performance of complex models like DNNs, as their rich hypothesis classes make standard PAC bounds ineffective Zhang et al. [2021], Nagarajan and Kolter [2019]. Our definition aims to bridge this gap by providing a formal framework that aligns with contemporary practical learning scenarios.

3.2 Interaction

Alice and Bob will engage in interaction. To measure their computational resources, we require a specification of how the model f n is transmitted between them. We assume that before the interaction starts they agree on a family of function classes F = {F n } n as well as an encoding of them into messages of some length. This modeling implies that f n are sent white-box . One example of such a family is the family of neural networks of a given architecture. See Appendix B for details.

3.3 Computational Indistinguishability

Acrucial property of interest will be the indistinguishability of distributions. For a pair of distributions D 0 , D 1 consider the following game between a sender and the distinguisher C : (1) The sender samples a bit b ∼ U ( { 0 , 1 } ) and then draws a random sample x ∼ D b , (2) C receives x and outputs ˆ b := C ( x ) ∈ { 0 , 1 } . C wins if ˆ b = b . We define the advantage of C for distinguishing D 0 from D 1

For a pair of families of distributions D 0 = {D 0 n } n , D 1 = {D 1 n } n , a function γ : ◆ → (0 , 1 2 ) , and a size bound S : ◆ → ◆ we say D 0 , D 1 are γ -indistinguishable for circuits of size S if for every n , every circuit C (also known as the distinguisher) of size S ( n ) the advantage of C for distinguishing D 0 n from D 1 n is at most γ ( n ) .

4 Watermarks, Adversarial Defenses and Transferable Attacks

In our protocols, Alice ( A , verifier) and Bob ( B , prover) engage in interactive communication, with distinct roles depending on the specific task. Each protocol is defined with respect to a learning

task L , an error parameter ε ∈ ( 0 , 1 2 ) , and circuit size bounds S A and S B , which are functions of n . A scheme is successful if the conditions of the protocols are satisfied. We denote the set of such circuits by SCHEME ( L , ε, S A ( n ) , S B ( n )) , where SCHEME refers to WATERMARK, DEFENSE, or TRANSFATTACK (see Appendix C for the formal versions of all the definitions).

Definition 3 ( Watermark, informal ) .

Afamily of circuits { A WATERMARK n } n of sizes { S A ( n ) } n , implements a backdoor-based watermarking scheme for the learning task L with error parameter ϵ > 0 if, for every sufficiently large n , an interactive protocol in which first ( D n , h n ) ∼ L n and then A WATERMARK n computes a classifier f : { 0 , 1 } n → { 0 , 1 } and a sequence of queries x ∈ ( { 0 , 1 } n ) q , and a prover B n outputs y = B n ( f, x ) ∈ { 0 , 1 } q , satisfies the following properties:

Correctness: f has low error, i.e., err ( f ) ≤ ϵ .

Uniqueness: There exists a prover B n , of size S A ( n ) , which provides low-error answers, such that err ( x , y ) ≤ 2 ϵ .

Unremovability: For every prover B n of size S B ( n ) , it holds that err ( x , y ) > 2 ϵ .

Undetectability: For every prover B n of size S B ( n ) , the advantage of B n in distinguishing the queries x generated by A WATERMARK n from random queries sampled from D q n is small.

As summarized in Table 1, uniqueness ensures that watermark verification cannot be triggered by independently trained models. Formally, we require that any B n (Bob), who did not use f and trained a model f Scratch using the specified procedure, must be accepted as distinct. This reflects realistic settings where multiple models could emerge independently.

Definition 4 ( Adversarial Defense, informal ) .

A family of circuits { B DEFENSE n } n of sizes { S B ( n ) } n , implements an adversarial defense for the learning task L with error parameter ϵ > 0 , if for every sufficiently large n , an interactive protocol in which first ( D n , h n ) ∼ L n and then B DEFENSE n computes a classifier f : { 0 , 1 } n →{ 0 , 1 } , while A n replies with x = A n ( f ) , where x ∈ ( { 0 , 1 } n ) q , and B DEFENSE n outputs b = B DEFENSE n ( f, x ) ∈ { 0 , 1 } , satisfies the following properties:

Correctness: f has low error, i.e., err ( f ) ≤ ϵ .

Completeness: When x ∼ D q n , then b = 0 .

Soundness: For every A n of size S A ( n ) , we have err ( x , f ( x )) ≤ 7 ϵ or b = 1 .

The key requirement for a successful defense is the ability to detect when it is being tested (see the soundness and completeness properties in Table 2). To bypass the defense, an A n (Alice) must provide samples that are both adversarial, causing the classifier to err, and indistinguishable from samples of D n .

Definition 5 ( Transferable Attack, informal ) .

A family of circuits { A TRANSFATTACK n } n of sizes { S A ( n ) } n , implements a transferable attack for the learning task L with error parameter ϵ > 0 , if for every sufficiently large n , an interactive protocol in which first ( D n , h n ) ∼ L n and then A TRANSFATTACK n computes x ∈ ( { 0 , 1 } n ) q and B n outputs y = B n ( x ) ∈ { 0 , 1 } q satisfies the following properties:

Correctness: Size S B ( n ) is sufficient to learn a classifier of low-error, err ( f ) ≤ ϵ .

Transferability: For every prover B n of size S A ( n ) , we have err ( x , y ) > 2 ϵ .

Undetectability: For every prover B n of size S B ( n ) , the advantage of B n in distinguishing the queries x generated by A TRANSFATTACK n from random queries sampled from D q n is small.

5 Main Result

We are ready to state an informal version of our main theorem (see Appendix D, for the full version). The key idea is to define a zero-sum game between A n (Alice) and B n (Bob), for every n , where the actions of each player are all possible circuits that can be realized with size S A ( n ) and S B ( n ) . Notably, this game is finite, but there are exponentially many such actions for each player. We rely on some key properties of such large zero-sum games [Lipton and Young, 1994b] to argue about our main result.

Theorem 1 ( Main Theorem, informal ) . For every ϵ ∈ ( 0 , 1 2 ) , S : ◆ → ◆ and learning task L learnable to error ϵ with high confidence with circuit complexity S ( n ) , at least one of these three exists 5 :

Proof (Sketch). The intuition of the proof relies on the complementary nature of Definitions 3 and 4. Specifically, every attempt to remove a fixed Watermark can be transformed to a potential Adversarial Defense, and vice versa. We define a zero-sum game G between circuits for watermarking A n and circuits attempting to remove a watermark B n . The set of (pure) strategies of each player are all possible circuits that can be realized with size S A ( n ) and S B ( n ) , and the payoff is determined by the probability that the errors and rejections meet specific requirements. It is well known that this

two-player zero-sum game admits a Nash equilibrium (NE) and the value of the game is unique v. Neumann [1928]. Let { A NASH n } n and { B NASH n } n be the NE strategies of Alice and Bob respectively. For each n ∈ ◆ , a careful analysis shows that depending on the value of the game, we have a Watermark, an Adversarial Defense, or a Transferable Attack. In the first case, where the expected payoff at the NE is greater than a threshold, we show there is an Adversarial Defense. As an illustration, consider some n ∈ ◆ , for which we define B DEFENSE n as follows. B DEFENSE n first learns a low-error classifier f , then sends f to the party that is attacking the Defense, then receives queries x , and simulates ( y , b ) = B NASH n ( f, x ) . The bit b = 1 if B NASH n thinks it is attacked. Finally, B DEFENSE n replies with b ′ = 1 if b = 1 , and if b = 0 it replies with b ′ = 1 if the fraction of queries on which f ( x ) and y differ is high. Careful analysis shows B DEFENSE n is an Adversarial Defense. In the second case, where the expected payoff at the NE is below the threshold, we have either a Watermark or a Transferable Attack. The full proof can be found in Appendix D.

6 Transferable Attacks and Cryptography

In this section, we show that tasks with Transferable Attacks exist. To construct such examples, we use cryptographic tools. But importantly, the fact that we use cryptography is not coincidental. As a second result of this section, we show that every learning task with a Transferable Attack implies a certain cryptographic primitive. One can interpret this as showing that Transferable Attacks exist only for complex learning tasks , in the sense of computational complexity theory.

6.1 A Cryptography-based Task with a Transferable Attack

Next, we give an example of a cryptography-based learning task with a Transferable Attack. The following is an informal statement of the formal version (Theorem 7) given in Appendix F.

Theorem 2 ( Transferable Attack for a Cryptography-based Learning Task, informal ) . There exists a learning task L crypto and A such that for all sufficiently small ϵ

Moreover, L crypto is such that for every ϵ , ≈ 1 ϵ time (and O ( 1 ϵ ) samples) is enough, and Ω ( 1 ϵ ) samples (and in particular time) is necessary to learn a classifier of error ϵ .

Notably, the parameters are set so that A (the party computing x ) has a smaller circuit size than B (the party computing y ), specifically ≈ 1 /ϵ compared to Ω(1 /ϵ 2 ) . Furthermore, because of the cryptography tools used, this is a setting where a single input maps to multiple outputs, which deviates away from the setting of classification learning tasks considered in Theorem 1.

Proof (Sketch). We start with a definition of a learning task that will be later augmented with a cryptographic tool to produce L crypto .

Lines on Circle Learning Task L ◦ (Figure 2). We associate the input space { 0 , 1 } n with vertices of a 2 n regular polygon inscribed in { x ∈ R 2 | ∥ x ∥ 2 = 1 } . Let H := { h w | w ∈ R 2 , ∥ w ∥ 2 = 1 } , where h w ( x ) := sgn ( ⟨ w,x ⟩ ) . Let L ◦ be a distribution corresponding to the following process: sample h w ∼ U ( H ) , return ( U ( { 0 , 1 } n ) , h w ) . Additionally, let B w ( α ) := { x ∈ { 0 , 1 } n | | ∡ ( x, w ) | ≤ α } denote the set of points within an angular distance up to α to w .

Fully Homomorphic Encryption (FHE) (Appendix E). FHE [Gentry, 2009] allows for computation on encrypted data without decrypting it. An FHE scheme allows to encrypt x via an efficient procedure e x = FHE.ENC ( x ) , so that later, for any algorithm C , it is possible to run C on x homomorphically . More concretely, it is possible to produce an encryption of the result of running C on x , i.e., e C,x := FHE.EVAL ( C, e x ) . Finally, there is a procedure FHE.DEC that, when given a secret key sk, can decrypt e C,x , i.e., y := FHE.DEC ( sk , e C,x ) , where y is the result of running C on x . Crucially, encryptions of any two messages are indistinguishable for all efficient adversaries.

Cryptography-based Learning Task L crypto (Figure 2). L crypto is derived from Lines on Circle Learning Task L ◦ . L crypto corresponds to the following process: w ∼ U ( { w ∈ R 2 | ∥ w ∥ 2 = 1 } ) , return the distribution D w , which is an equal mixture of two parts D w = 1 2 D w CLEAR + 1 2 D w ENC . The first

hw

h

A

≈

ϵ

0. Learning Task

L

crypto

samples a distribution

D

w

=

1

2

D

w

CLEAR

+

1

2

D

w

ENC

.

1.

x

∼

U

(

{

0

,

1

}

n

)

,

b

∼

Ber

(1

/

2)

, where

U

(

{

0

,

1

}

n

)

is uniform on the cir-

cle.

2. If

b

= 0

, return

(

x, hw

(

x

))

.

3. Else, return

(

FHE.ENC

(

x

)

,

FHE.ENC

(

hw

(

x

)))

.

(size

S

A

≈

1

/ϵ

)

(size

S

B

≈

1

/ϵ

2

)

learns

h

A

learns

h

B

x

y

part, i.e., D w CLEAR , is equal to x ∼ U ( { 0 , 1 } n ) with the correct label y = h w ( x ) . The second part, i.e., D w ENC , is equal to x ′ ∼ U ( { 0 , 1 } n ) , y ′ = h w ( x ′ ) , ( x, y ) = ( FHE.ENC ( x ′ ) , FHE.ENC ( y ′ )) , 6 which can be thought of as D w CLEAR under an encryption. See Figure 2 for a visual representation. Note that we omitted the size parameter n for simplicity.

Transferable Attack (Figure 2). Consider the following attack strategy A . First, A collects O (1 /ϵ ) samples from the distribution D w CLEAR and learns a classifier h A w ′ ∈ H that is consistent with these samples. Since the VC-dimension of H is 2, the hypothesis h A w ′ has error at most ϵ with high probability. 7 Next, A samples a point x BND uniformly at random from a region close to the decision boundary of h A w ′ , i.e., x BND ∼ U ( B w ′ ( ϵ )) . Finally, with equal probability, A sets as an attack x either FHE.ENC ( x BND ) or a uniformly random point D w CLEAR = U ( { 0 , 1 } n ) . We claim 8 that x satisfies the properties of a Transferable Attack.

Since h A w ′ has a low error with high probability, x BND is a uniformly random point from an arc containing the boundary of h w (see Figure 2). The circuit size of B is upper-bounded by Ω(1 /ϵ 2 ) , meaning it can only learn a classifier with error ⪆ 10 ϵ 2 (see Lemma 3 for details). B 's can only learn (Lemma 3) a classifier of error, ⪆ 10 ϵ 2 . Taking these two facts together, we expect B to misclassify x ′ with probability ≈ 1 2 · 10 ϵ 2 ϵ = 5 ϵ > 2 ϵ, where the factor 1 2 takes into account that we send an encrypted sample only half of the time. This implies transferability .

Note that x is encrypted with the same probability as in the original distribution because we send FHE.ENC ( x BND ) and a uniformly random x ∼ D w CLEAR = U ( { 0 , 1 } n ) with probability 1 2 . Crucially, FHE.ENC ( x BND ) is indistinguishable, for efficient adversaries, from FHE.ENC ( x ) for any other x ∈ { 0 , 1 } n . This follows from the security of the FHE. Consequently, undetectability holds.

6.2 Tasks with Transferable Attacks Imply Cryptography

In this section, we show that a Transferable Attack for any task implies a cryptographic primitive .

EFID Pairs. In cryptography, an EFID pair [Goldreich, 1990] is a pair of ensembles of distributions D 0 , D 1 , that are E fficiently samplable, statistically F ar, and computationally I ndistinguishable. By

a seminal result [Goldreich, 1990], we know that the existence of EFID pairs is equivalent to the existence of Pseudorandom Generators (PRG), which can be used for tasks including encryption and key generation [Goldreich, 1990], which makes EFID pairs a useful primitive. We consider a slight modification of the standard definition of EFID pairs, where instead of defining security to hold against polynomial time adversaries we do it for a fixed size bound function. More concretely, for two size bounds S, S ′ : ◆ → ◆ we call a pair of ensembles of distributions ( D 0 , D 1 ) an ( S, S ′ ) -EFID pair if for every n (i) D 0 n , D 1 n are samplable by circuits of size S ( n ) , (ii) D 0 n , D 1 n are statistically far, (iii) D 0 n , D 1 n are indistinguishable for circuits of size S ′ ( n ) .

Tasks with Transferable Attacks imply EFID Pairs. The second result shows that any task with a Transferable Attack implies the existence of a type of EFID pair. This guarantees that any learning task with a Transferable Attack has to be computationally complex. The proof is in Appendix G.

Theorem 3 ( Transferable Attacks imply EFID pairs, informal ) . For every ϵ ∈ (0 , 1) , S A , S B : ◆ → ◆ , S A ≤ S B , every learning task L learnable to error ϵ with high confidence and circuit complexity S A if there exists TRANSATTACK ( L , ϵ, S A , S B ) then there exists an ( S A , S B ) -EFID pair.

We note that it is unclear if the existence of EFID-pairs guaranteed by Theorem 3 implies PRGs because the sampling of D 0 , D 1 requires oracle access to L . Therefore, the standard construction of PRGs from EFID pairs does not automatically transfer.

7 Tasks with Watermarks and Adversarial Defenses

As the final pair of results, we present tasks exhibiting Watermarks and Adversarial Defenses. In the first, hypothesis classes with polynomially bounded VC-dimension admit polynomial-size Adversarial Defenses against all attackers. In the second, a learning task of polynomially bounded VC-dimension admits a Watermark secure against fast adversaries. These lemmas highlight the importance of bounding the sizes of A and B . See Figure 3 for a visual summary; formal statements and proofs appear in Appendices H and I.

Lemma 1 ( Adversarial Defense for bounded VC-dimension, informal ) . There exists B such that for every n , every learning task L of VC-dimension n 9 , every sufficiently small ϵ ,

Lemma 2 ( Watermark for bounded VC-dimension against fast adversaries, informal ) . For every d , there exists a learning task L of VC-dimension d and A such that for every sufficiently small ϵ ,

References

Lukáš Adam, Rostislav Horˇ cík, Tomáš Kasl, and Tomáš Kroupa. Double oracle algorithm for computing equilibria in continuous games. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 5070-5077, 2021.

Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security 18) , pages 1615-1631, 2018.

Noga Amit, Shafi Goldwasser, Orr Paradise, and Guy Rothblum. Models that prove their own correctness. arXiv preprint arXiv:2405.15722 , 2024.

Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safetyaligned llms with simple adaptive attacks, 2024.

Cem Anil, Guodong Zhang, Yuhuai Wu, and Roger Grosse. Learning to give checkable answers with prover-verifier games. arXiv preprint arXiv:2108.12099 , 2021.

Sanjeev Arora and Boaz Barak. Computational Complexity: A Modern Approach . Cambridge University Press, USA, 1st edition, 2009. ISBN 0521424267.

Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled) fully homomorphic encryption without bootstrapping. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference , ITCS '12, page 309-325, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450311520. doi: 10.1145/2090236.2090262. URL https://doi.org/ 10.1145/2090236.2090262 .

Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras. Scalable ai safety via doubly-efficient debate. arXiv preprint arXiv:2311.14125 , 2023.

Sébastien Bubeck, Yin Tat Lee, Eric Price, and Ilya Razenshteyn. Adversarial examples from computational constraints. In International Conference on Machine Learning , pages 831-840. PMLR, 2019.

Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeffrey Wu. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning , volume 235 of Proceedings of Machine Learning Research , pages 4971-5012. PMLR, 21-27 Jul 2024. URL https://proceedings.mlr.press/v235/burns24b.html .

Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramèr, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? ArXiv , abs/2306.15447, 2023. URL https: //api.semanticscholar.org/CorpusID:259262181 .

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2023.

Jiefeng Chen, Yang Guo, Xi Wu, Tianqi Li, Qicheng Lao, Yingyu Liang, and Somesh Jha. Towards adversarial robustness via transductive learning. arXiv preprint arXiv:2106.08387 , 2021.

Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. arXiv preprint arXiv:2306.09194 , 2023.

Paul Christiano, Jacob Hilton, Victor Lecomte, and Mark Xu. Backdoor defense, learnability and obfuscation. arXiv preprint arXiv:2409.03077 , 2024.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 1310-1320. PMLR, 09-15 Jun 2019. URL https://proceedings.mlr. press/v97/cohen19c.html .

Anne Condon, Joan Feigenbaum, Carsten Lund, and Peter Shor. Probabilistically checkable debate systems and approximation algorithms for pspace-hard functions. In Proceedings of the twenty-fifth annual ACM symposium on Theory of Computing , pages 305-314, 1993.

Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the twenty-fourth international conference on architectural support for programming languages and operating systems , pages 485-497, 2019.

Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4307-4316, 2019. URL https://api.semanticscholar. org/CorpusID:102350868 .

Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling cnns with simple transformations. ArXiv , abs/1712.02779, 2017. URL https://api.semanticscholar.org/CorpusID:21929206 .

Yousof Erfani, Ramin Pichevar, and Jean Rouat. Audio watermarking using spikegram and a twodictionary approach. IEEE Transactions on Information Forensics and Security , 12(4):840-852, 2017. doi: 10.1109/TIFS.2016.2636094.

Jon Feldman, Rocco A Servedio, and Ryan O'Donnell. Pac learning axis-aligned mixtures of gaussians with no separation assumption. In Learning Theory: 19th Annual Conference on Learning Theory, COLT 2006, Pittsburgh, PA, USA, June 22-25, 2006. Proceedings 19 , pages 20-34. Springer, 2006.

Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22466-22477, 2023.

Liam Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojciech Czaja, and Tom Goldstein. Adversarial examples make strong poisons. Advances in Neural Information Processing Systems , 34:30339-30351, 2021.

Gammerman, V. Vovk, and V. Vapnik. Learning by transduction. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence , UAI'98, page 148-155, San Francisco, CA, USA, 1998. Morgan Kaufmann Publishers Inc. ISBN 155860555X.

Sanjam Garg, Somesh Jha, Saeed Mahloujifar, and Mahmoody Mohammad. Adversarially robust learning could leverage computational hardness. In Algorithmic Learning Theory , pages 364-385. PMLR, 2020.

Craig Gentry. Fully homomorphic encryption using ideal lattices. In Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing , STOC '09, page 169-178, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585062. doi: 10.1145/1536414. 1536440. URL https://doi.org/10.1145/1536414.1536440 .

Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenberg, and Ian J. Goodfellow. Adversarial spheres. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings , 2018. URL https://openreview.net/forum?id=SkthlLkPf .

Greg Gluch and Shafi Goldwasser. A cryptographic perspective on mitigation vs. detection in machine learning, 2025. URL https://arxiv.org/abs/2504.20310 .

Oded Goldreich. A note on computational indistinguishability. Information Processing Letters , 34 (6):277-281, 1990. ISSN 0020-0190. doi: https://doi.org/10.1016/0020-0190(90)90010-U. URL https://www.sciencedirect.com/science/article/pii/002001909090010U .

S Goldwasser and M Sipser. Private coins versus public coins in interactive proof systems. In Proceedings of the Eighteenth Annual ACM Symposium on Theory of Computing , STOC '86, page 59-68, New York, NY, USA, 1986. Association for Computing Machinery. ISBN 0897911938. doi: 10.1145/12130.12137. URL https://doi.org/10.1145/12130.12137 .

S Goldwasser, S Micali, and C Rackoff. The knowledge complexity of interactive proof-systems. In Proceedings of the Seventeenth Annual ACM Symposium on Theory of Computing , STOC '85, page 291-304, New York, NY, USA, 1985. Association for Computing Machinery. ISBN 0897911512. doi: 10.1145/22145.22178. URL https://doi.org/10.1145/22145.22178 .

Shafi Goldwasser, Yael Kalai, Raluca Ada Popa, Vinod Vaikuntanathan, and Nickolai Zeldovich. Reusable garbled circuits and succinct functional encryption. In Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing , STOC '13, page 555-564, New York, NY, USA, 2013. Association for Computing Machinery. ISBN 9781450320290. doi: 10.1145/2488608. 2488678. URL https://doi.org/10.1145/2488608.2488678 .

Shafi Goldwasser, Adam Tauman Kalai, Yael Tauman Kalai, and Omar Montasser. Beyond perturbations: Learning guarantees with arbitrary adversarial test examples. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.

Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. Planting undetectable backdoors in machine learning models. ArXiv , abs/2204.06974, 2022. URL https://api. semanticscholar.org/CorpusID:248177888 .

Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. Watermarking pre-trained language models with backdooring. arXiv preprint arXiv:2210.07543 , 2022.

Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987 , 2024.

Geoffrey Irving, Paul Christiano, and Dario Amodei. Ai safety via debate, 2018. URL https: //arxiv.org/abs/1805.00899 .

Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , 2023. URL https://api.semanticscholar.org/CorpusID:258557682 .

Adam Tauman Kalai, Adam R Klivans, Yishay Mansour, and Rocco A Servedio. Agnostically learning halfspaces. SIAM Journal on Computing , 37(6):1777-1805, 2008.

Michael J. Kearns and Umesh V. Vazirani. An introduction to computational learning theory . MIT Press, Cambridge, MA, USA, 1994. ISBN 0262111934.

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 17061-17084. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr. press/v202/kirchenbauer23a.html .

Jan Hendrik Kirchner, Yining Chen, Harri Edwards, Jan Leike, Nat McAleese, and Yuri Burda. Prover-Verifier Games improve legibility of LLM outputs, 2024. URL https://arxiv.org/ abs/2407.13692 .

Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. CoRR , abs/2307.15593, 2023. doi: 10.48550/ARXIV.2307.15593. URL https://doi.org/10.48550/arXiv.2307.15593 .

Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, and Thore Graepel. A unified game-theoretic approach to multiagent reinforcement learning. Advances in neural information processing systems , 30, 2017.

Peixuan Li, Pengzhou Cheng, Fangqi Li, Wei Du, Haodong Zhao, and Gongshen Liu. Plmmark: a secure and robust black-box watermarking framework for pre-trained language models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 14991-14999, 2023.

Richard J. Lipton and Neal E. Young. Simple strategies for large zero-sum games with applications to complexity theory. In Proceedings of the Twenty-Sixth Annual ACM Symposium on Theory of Computing , STOC '94, page 734-740, New York, NY, USA, 1994a. Association for Computing Machinery. ISBN 0897916638. doi: 10.1145/195058.195447. URL https://doi.org/10. 1145/195058.195447 .

Richard J Lipton and Neal E Young. Simple strategies for large zero-sum games with applications to complexity theory. In Proceedings of the twenty-sixth annual ACM symposium on Theory of computing , pages 734-740, 1994b.

Chang Liu, Jie Zhang, Han Fang, Zehua Ma, Weiming Zhang, and Nenghai Yu. Dear: A deeplearning-based audio re-recording resilient watermarking. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 13201-13209, 2023.

Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770 , 2016.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. URL https://openreview.net/forum?id= rJzIBfZAb .

Stephen McAleer, John B Lanier, Kevin A Wang, Pierre Baldi, and Roy Fox. Xdo: A double oracle algorithm for extensive-form games. Advances in Neural Information Processing Systems , 34: 23128-23139, 2021.

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically, 2024.

Erwan Le Merrer, Patrick Pérez, and Gilles Trédan. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications , 32:9233 - 9244, 2017. URL https://api.semanticscholar.org/CorpusID:11008755 .

Omar Montasser, Steve Hanneke, and Nathan Srebro. Transductive robust learning guarantees. In International Conference on Artificial Intelligence and Statistics , pages 11461-11471. PMLR, 2022.

Yuki Nagai, Yusuke Uchida, Shigeyuki Sakazawa, and Shin'ichi Satoh. Digital watermarking for deep neural networks. International Journal of Multimedia Information Retrieval , 7:3-16, 2018.

Vaishnavh Nagarajan and J Zico Kolter. Uniform convergence may be unable to explain generalization in deep learning. Advances in Neural Information Processing Systems , 32, 2019.

Ryota Namba and Jun Sakuma. Robust watermarking of neural network with exponential weighting. Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security , 2019. URL https://api.semanticscholar.org/CorpusID:58028915 .

Noam Nisan. Pseudorandom generators for space-bounded computations. In Proceedings of the twenty-second annual ACM symposium on Theory of computing , pages 204-212, 1990.

Zhenxing Niu, Yuyao Sun, Qiguang Miao, Rong Jin, and Gang Hua. Towards unified robustness against both backdoor and adversarial attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(12):7589-7605, 2024. doi: 10.1109/TPAMI.2024.3392760.

Ambar Pal and René Vidal. A game theoretic analysis of additive adversarial attacks and defenses. Advances in Neural Information Processing Systems , 33:1345-1355, 2020.

Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, and Xing Xie. Are you copying my model? protecting the copyright of large language models for eaas via backdoor watermark. arXiv preprint arXiv:2305.10036 , 2023.

Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. URL https://openreview.net/forum?id=Bys4ob-Rb .

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems , volume 20. Curran Associates, Inc., 2007. URL https://proceedings.neurips.cc/paper_ files/paper/2007/file/013a006f03dbc5392effeb8f18fda755-Paper.pdf .

Oded Regev. On lattices, learning with errors, random linear codes, and cryptography. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , pages 84-93. ACM, 2005.

Rivest, L. Adleman, and M. Dertouzos. On data banks and privacy homomorphisms. In Foundations of Secure Computation , page 169-179, New York, NY, USA, 1978. Academic Press.

Mingjie Sun, Siddhant Agarwal, and J Zico Kolter. Poisoned classifiers are not only backdoored, they are fundamentally broken. arXiv preprint arXiv:2010.09080 , 2020.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , 2014. URL http://arxiv.org/abs/1312. 6199 .

Lue Tao, Lei Feng, Jinfeng Yi, Sheng-Jun Huang, and Songcan Chen. Better Safe than Sorry: Preventing Delusive Adversaries with Adversarial Training. In Proceedings of the 35th International Conference on Neural Information Processing Systems , NIPS '21, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9781713845393.

Stuart A. Thompson Tiffany Hsu. Disinformation researchers raise alarms about a.i. chatbots. https://scottaaronson.blog/?p=6823 , 2023. Accessed: March 2024.

Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv:1704.03453 , 2017.

Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in neural information processing systems , 33:1633-1645, 2020.

Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval , pages 269-277, 2017.

J v. Neumann. Zur theorie der gesellschaftsspiele. Mathematische annalen , 100(1):295-320, 1928.

Vinod Vaikuntanathan. Computing blindfolded: New developments in fully homomorphic encryption. In Proceedings of the 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science , FOCS '11, page 5-16, Washington, DC, USA, 2011. IEEE Computer Society. ISBN 9780769543001. doi: 10.1109/FOCS.2011.98. URL https://doi.org/10.1109/FOCS.2011. 98 .

Stephan Wäldchen, Kartikey Sharma, Berkant Turan, Max Zimmer, and Sebastian Pokutta. Interpretability Guarantees with Merlin-Arthur Classifiers. In International Conference on Artificial Intelligence and Statistics , pages 1963-1971. PMLR, 2024.

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? ArXiv , abs/2307.02483, 2023. URL https://api.semanticscholar.org/CorpusID: 259342528 .

Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 51008-51025. Curran Associates, Inc., 2023a.

Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. ArXiv , abs/2305.20030, 2023b. URL https://api.semanticscholar.org/CorpusID:258987524 .

Cheng-Hsin Weng, Yan-Ting Lee, and Shan-Hung (Brandon) Wu. On the trade-off between adversarial and backdoor robustness. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 11973-11983. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/8b4066554730ddfaa0266346bdc1b202-Paper.pdf .

Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , volume 80 of Proceedings of Machine Learning Research , pages 5283-5292. PMLR, 2018. URL http://proceedings.mlr.press/v80/wong18a.html .

Yi-Hsuan Wu, Chia-Hung Yuan, and Shan-Hung Wu. Adversarial robustness via runtime masking and cleansing. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 10399-10409. PMLR, 13-18 Jul 2020. URL https://proceedings.mlr.press/v119/ wu20f.html .

Cihang Xie, Zhishuai Zhang, Jianyu Wang, Yuyin Zhou, Zhou Ren, and Alan Loddon Yuille. Improving transferability of adversarial examples with input diversity. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2725-2734, 2018. URL https: //api.semanticscholar.org/CorpusID:3972825 .

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM , 64(3):107-115, 2021.

Hanlin Zhang, Benjamin L. Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. arXiV , abs/2311.04378, 2023. doi: 10.48550/ARXIV.2311.04378. URL https://doi.org/10. 48550/arXiv.2311.04378 .

Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph. Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security , ASIACCS '18, page 159-172, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450355766. doi: 10.1145/3196494.3196550. URL https://doi.org/10.1145/3196494. 3196550 .

Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. CoRR , abs/2306.17439, 2023a. doi: 10.48550/ARXIV.2306.17439. URL https://doi.org/10.48550/arXiv.2306.17439 .

Xuandong Zhao, Kexun Zhang, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai. 2023b. URL https://api.semanticscholar.org/CorpusID: 259075167 .

Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. A recipe for watermarking diffusion models. ArXiv , abs/2303.10137, 2023c. URL https://api. semanticscholar.org/CorpusID:257622907 .

Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. ArXiv , abs/2307.15043, 2023. URL https: //api.semanticscholar.org/CorpusID:260202961 .

A Additional Methods in Related Work

This section provides an overview of the main areas relevant to our work: Watermarking techniques, adversarial defenses, and transferable attacks on Deep Neural Networks (DNNs). Each subsection outlines important contributions and the current state of research in these areas, offering additional context and details beyond those covered in the main body

A.1 Watermarking

Watermarking techniques are crucial for protecting the intellectual property of machine learning models. These techniques can be broadly categorized based on the type of model they target. We review watermarking schemes for both classification and generative models, with a primary focus on classification models, as our work builds upon these methods.

A.1.1 Watermarking Schemes for classification Models

classification models, which are designed to categorize input data into predefined classes, have been a major focus of watermarking research. The key approaches in this domain can be divided into black-box and white-box approaches.

Black-Box Setting. In the black-box setting, the model owner does not have access to the internal parameters or architecture of the model, but can query the model to observe its outputs. This setting has seen the development of several watermarking techniques, primarily through backdoor-like methods.

Adi et al. [2018] and Zhang et al. [2018] proposed frameworks that embed watermarks using specifically crafted input data (e.g., unique patterns) with predefined outcomes. These watermarks can be verified by feeding these special inputs into the model and checking for the expected outputs, thereby confirming ownership.

Another significant contribution in this domain is by Merrer et al. [2017], who introduced a method that employs adversarial examples to embed the backdoor. Adversarial examples are perturbed inputs that cause the model to produce specific outputs, thus serving as a watermark.

Namba and Sakuma [2019] further enhanced the robustness of black-box watermarking schemes by developing techniques that withstand various model modifications and attacks. These methods ensure that the watermark remains intact and detectable even when the model undergoes transformations.

Provable undetectability of backdoors was achieved in the context of classification tasks by Goldwasser et al. [2022]. Unfortunately, it is known ([Goldwasser et al., 2022]) that some undetectable watermarks are easily removed by simple mechanisms similar to randomized smoothing [Cohen et al., 2019].

The popularity of black-box watermarking is due to its practical applicability, as it does not require access to the model's internal workings. This makes it suitable for scenarios where models are deployed as APIs or services. Our framework builds upon these black-box watermarking techniques.

White-Box Setting. In contrast, the white-box setting assumes that the model owner has full access to the model's parameters and architecture, allowing for direct examination to confirm ownership. The initial methodologies for embedding watermarks into the weights of DNNs were introduced by Uchida et al. [2017] and Nagai et al. [2018]. Uchida et al. [2017] presented a framework for embedding watermarks into the model weights, which can be examined to confirm ownership.

An advancement in white-box watermarking is provided by Darvish Rouhani et al. [2019], who developed a technique to embed an N -bit ( N ≥ 1 ) watermark in DNNs. This technique is both dataand model-dependent , meaning the watermark is activated only when specific data inputs are fed into the model. For revealing the watermark, activations from intermediate layers are necessary in the case of white-box access, whereas only the final layer's output is needed for black-box scenarios.

Our work does not focus on white-box watermarking techniques. Instead, we concentrate on exploring the interaction between backdoor-like watermarking techniques, adversarial defenses, and transferable attacks. Overall, watermarking through backdooring has become more popular due to its applicability in the black-box setting.

A.1.2 Watermarking Schemes for Generative Models

Watermarking techniques for generative models have attracted considerable attention with the advent of Large Language Models (LLMs) and other advanced generative models. This increased interest has led to a surge in research and diverse contributions in this area.

Backdoor-Based Watermarking for Pre-trained Language Models. In the domain of Natural Language Processing (NLP), backdoor-based watermarks have been increasingly studied for Pretrained Language Models (PLMs), as exemplified by works such as [Gu et al., 2022] and [Li et al., 2023]. These methods leverage rare or common word triggers to embed watermarks, ensuring that they remain robust across downstream tasks and resilient to removal techniques like fine-tuning or pruning. While these approaches have demonstrated promising results in practical applications, they are primarily empirical, with theoretical aspects of watermarking and robustness requiring further exploration.

Watermarking the Output of LLMs. Watermarking the generated text of LLMs is critical for mitigating potential harms. Significant contributions in this domain include [Kirchenbauer et al., 2023], who proposed a watermarking framework that embeds signals into generated text that are invisible to humans but detectable algorithmically. This method promotes the use of a randomized set of 'green' tokens during text generation, and detects the watermark without access to the language model API or parameters.

Kuditipudi et al. [2023] introduced robust distortion-free watermarks for language models. Their method ensures that the watermark does not distort the generated text, providing robustness against various text manipulations while maintaining the quality of the output.

Zhao et al. [2023a] presented a provable, robust watermarking technique for AI-generated text. This approach offers strong theoretical guarantees for the robustness of the watermark, making it resilient against attempts to remove or alter it without significantly changing the generated text.

However, Zhang et al. [2023] highlighted vulnerabilities in these watermarking schemes. Their work demonstrates that current watermarking techniques can be effectively broken, raising important considerations for the future development of robust and secure watermarking methods for LLMs.

Image Generation Models. Various watermarking techniques have been developed for image generation models to address ethical and legal concerns. Fernandez et al. [2023] introduced a method combining image watermarking with Latent Diffusion Models, embedding invisible watermarks in generated images for future detection. This approach is robust against modifications such as cropping. Wen et al. [2023b] proposed Tree-Ring Watermarking, which embeds a pattern into the initial noise vector during sampling, making the watermark robust to transformations like convolutions and rotations. Jiang et al. [2023] highlighted vulnerabilities in watermarking schemes, showing that human-imperceptible perturbations can evade watermark detection while maintaining visual quality. Zhao et al. [2023c] provided a comprehensive analysis of watermarking techniques for Diffusion Models, offering a recipe for efficiently watermarking models like Stable Diffusion, either through training from scratch or fine-tuning. Additionally, Zhao et al. [2023b] demonstrated that invisible watermarks are vulnerable to regeneration attacks that remove watermarks by adding random noise and reconstructing the image, suggesting a shift towards using semantically similar watermarks for better resilience.

Audio Generation Models. Watermarking techniques for audio generators have been developed for robustness against various attacks. Erfani et al. [2017] introduced a spikegram-based method, embedding watermarks in high-amplitude kernels, robust against MP3 compression and other attacks while preserving quality. Liu et al. [2023] proposed DeAR, a deep-learning-based approach resistant to audio re-recording (AR) distortions.

A.2 Adversarial Defense

The field of adversarial robustness has a rich and extensive literature [Szegedy et al., 2014, Gilmer et al., 2018, Raghunathan et al., 2018, Wong and Kolter, 2018, Engstrom et al., 2017]. Adversarial defenses are essential for ensuring the security and reliability of machine learning models against adversarial attacks that aim to deceive them with carefully crafted inputs.

For classification models, there has been significant progress in developing adversarial defenses. Techniques such as adversarial training [Madry et al., 2018], which involves training the model on adversarial examples, have shown promise in improving robustness. Certified defenses [Raghunathan et al., 2018] provide provable guarantees against adversarial attacks, ensuring that the model's predictions remain unchanged within a specified perturbation bound. Additionally, methods like randomized smoothing [Cohen et al., 2019] offer robustness guarantees.

A particularly relevant work for our study is [Goldwasser et al., 2020], which considers a different model for generating adversarial examples. This approach has significant implications for the robustness of watermarking techniques in the face of adversarial attacks.

In the context of LLMs, there is a rapidly growing body of research focused on identifying adversarial examples [Zou et al., 2023, Carlini et al., 2023, Wen et al., 2023a]. This research is closely related to the notion of jailbreaking [Andriushchenko et al., 2024, Chao et al., 2023, Mehrotra et al., 2024, Wei et al., 2023], which involves manipulating models to bypass their intended constraints and protections.

A.3 Transferable Attacks and Transductive Learning

Transferable attacks refer to adversarial examples that are effective across multiple models. Moreover, transductive learning has been explored as a means to enhance adversarial robustness, and since our Definition 5 captures some notion of transductive learning in the context of Transferable Attacks, we highlight significant contributions in these areas.

Adversarial Robustness via Transductive Learning. Transductive learning [Gammerman et al., 1998] has shown promise in improving the robustness of models by utilizing both training and test data during the learning process. This approach aims to make models more resilient to adversarial perturbations encountered at test time.

One significant contribution is by Goldwasser et al. [2020], which explores learning guarantees in the presence of arbitrary adversarial test examples, providing a foundational framework for transductive robustness. Another notable study by Chen et al. [2021] formalizes transductive robustness and proposes a bilevel attack objective to challenge transductive defenses, presenting both theoretical and empirical support for transductive learning's utility.

Additionally, Montasser et al. [2022] introduce a transductive learning model that adapts to perturbation complexity, achieving a robust error rate proportional to the VC dimension. The method by Wu et al. [2020] improves robustness by dynamically adjusting the network during runtime to mask gradients and cleanse non-robust features, validated through experimental results. Lastly, Tramer et al. [2020] critique the standard of adaptive attacks, demonstrating the need for specific tuning to effectively evaluate and enhance adversarial defenses.

Transferable Attacks on DNNs. Transferable attacks exploit the vulnerability of models to adversarial examples that generalize across different models. For classification models, significant works include Liu et al. [2016], which investigates the transferability of adversarial examples and their effectiveness in black-box attack scenarios, [Xie et al., 2018], who propose input diversity techniques to enhance the transferability of adversarial examples across different models, and [Dong et al., 2019], which presents translation-invariant attacks to evade defenses and improve the effectiveness of transferable adversarial examples.

In the context of generative models, including LLMs and other advanced generative architectures, relevant research is rapidly emerging, focusing on the transferability of adversarial attacks. This area is crucial as it aims to understand and mitigate the risks associated with adversarial examples in these powerful models. Notably, Zou et al. [2023] explored universal and transferable adversarial attacks on aligned language models, highlighting the potential vulnerabilities and the need for robust defenses in these systems.

A.4 Interactive Proof Systems in Machine Learning

Interactive Proof Systems [Goldwasser and Sipser, 1986] have recently gained considerable attention in machine learning for their ability to formalize and verify complex interactions between agents, models, or even human participants. A key advancement in this area is the introduction of Prover-

Verifier Games (PVGs) [Anil et al., 2021], which employ a game-theoretic approach to guide learning agents towards decision-making with verifiable outcomes. Building on PVGs, Kirchner et al. [2024] enhance this framework to improve the legibility of Large Language Models (LLMs) outputs, making them more accessible for human evaluation. Similarly, Wäldchen et al. [2024] apply the proververifier setup to offer interpretability guarantees for classifiers. Extending these concepts, self-proving models Amit et al. [2024] introduce generative models that not only produce outputs but also generate proof transcripts to validate their correctness. In the context of AI safety, scalable debate protocols [Condon et al., 1993, Irving et al., 2018, Brown-Cohen et al., 2023] leverage interactive proof systems to enable complex decision processes to be broken down into verifiable components, ensuring reliability even under adversarial conditions.

Undetectability

Unremovability

Uniqueness

Goldwasser et al. [2022]

"

robust to some

smoothing attacks

"

(E)

Classification

Adi et al. [2018], Zhang et al. [2018]

"

(E)

%

"

(E)

Merrer et al. [2017]

"

(E)

robust to fine tunning

attacks

"

(E)

Christ et al. [2023], Kuditipudi et al. [2023]

"

%

"

Zhao et al. [2023a]

%

robust to edit

distance attacks only

"

LLMs

"

(E)

%

"

Kirchenbauer et al. [2023]

%

%

"

B Preliminaries

For n ∈ ◆ we define [ n ] := { 1 , . . . , n } . We say a boolean sequence a : ◆ → { 0 , 1 } is true with frequency α ∈ [0 , 1] if

For two sequences a, b : ◆ → R we say they agree with frequency at least α ∈ [0 , 1] if the sequence ( a ? = b ) : →{ 0 , 1 } , i.e. ( a ? = b )( n ) = a ( n )= b ( n ) , is true with frequency α .

◆ ✶ Learning. For a set Ω , we write ∆(Ω) to denote the set of all probability measures defined on the measurable space (Ω , F ) , where F is some fixed σ -algebra that is implicitly understood. For a parameter n , we denote by { 0 , 1 } n the input space and by { 0 , 1 } the output space. A model is a function f : { 0 , 1 } n →{ 0 , 1 } .

Definition 6 ( Learning Task ) . A learning task L is a family { L n } n ∈ ◆ , where for every n , L n is an element of ∆ ( ∆( { 0 , 1 } n ) ×{ 0 , 1 } { 0 , 1 } n ) .

̸

For a distribution D n ∈ ∆( { 0 , 1 } n ) and a ground truth h n : { 0 , 1 } n →{ 0 , 1 } , we define an error of f as err D n ,h n ( f ) := E x ∼D n [ f ( x ) = h ( x )] , where the index of err will often be understood implicitly and omitted in notation. For D n ∈ ∆( { 0 , 1 } n ) , h n : { 0 , 1 } n →{ 0 , 1 } we define an example oracle Ex ( D n , h n ) as an oracle that samples x ∼ D n and returns ( x, h n ( x )) .

Interaction. When Ex ( D , h ) generates ( x, h ( x )) it is encoded as an n + 1 bit-string, because x ∈ { 0 , 1 } n and the label space is { 0 , 1 } . For a message space M = {M n } n = {{ 0 , 1 } m ( n ) } n a representation class is a collection of mappings {R n } n , where for every n , R n : M n →{ 0 , 1 } { 0 , 1 } n . Thus, there is a function class corresponding to a representation, i.e., for every n there is a function class F n , which is an image of R n . Note that h n (which is the ground truth) may or may not be in F n . All function classes considered in this work have an implicit representation class and an underlying message space.

Computation. We work with the collection of Boolean circuits over the standard basis B 2 , the set of all two-bit Boolean functions. The size of a circuit C is measured by its number of gates; let | C | denote the size of C . For a circuit family C = { C n } n we say it has a circuit complexity S ( n ) if for every n , | C n | ≤ S ( n ) .

For a distribution D n over { 0 , 1 } n , and a ground truth h n : { 0 , 1 } n → { 0 , 1 } we denote by C Ex ( D n ,h n ) a circuit with some 10 number of specified input gates that are initialized with samples ( x, h ( x )) sampled from x ∼ D n . We will also by interested in interaction between circuits. When messages are exchanged between circuits we assume that there are specified input (output) gates that correspond to outgoing (ingoing) messages. Also, when a circuit is randomized we assume there are designated input gates that are initialized with random bits.

Definition 7 ( Computationally Bounded Learnability ) . For ϵ, δ : ◆ → (0 , 1) we say that a learning task L = { L n } n ∈ ◆ is learnable to error ϵ with confidence 1 -δ and with circuit complexity S : ◆ → ◆ by a function class F = {F n } n ∈ ◆ (with a corresponding representation class R ), or ( ϵ, δ, S, F ) -learnable in short, if there exists a circuit family C = { C n } n ∈ ◆ with complexity S ( n ) such that for every sufficiently large n , with probability 1 -δ over the choice of ( D n , h n ) ∼ L n , C Ex ( D n ,h n ) n computes an m ( n ) bit message m f n ∈ M n such that R n ( m f n ) ∈ F n has error at most ϵ , i.e. for every sufficiently large n

We often abuse the notation and use f n to denote both m f n as well as R n ( m f n ) .

C Formal Definitions

Definition 8 ( Watermark ) . Let L = { L n } n be a learning task, and F = {F n } n a function class. Let S A , S B , q : ◆ → ◆ , ϵ ∈ ( 0 , 1 2 ) , l, c, s ∈ (0 , 1) , s < c , where S B ( n ) bounds the circuit size of B n , and S A ( n ) the circuit size of A n , q ( n ) the number of queries, ϵ the risk level, c probability that uniqueness holds, s probability that unremovability and undetectability holds, l the learning probability.

We say that a family of circuits A WATERMARK = { A WATERMARK n } n with complexity S A ( n ) implements a watermarking scheme for L with frequency α , denoted by

if the following is true with frequency α over parameter n . An interactive protocol in which first ( D n , h n ) ∼ L n and then A WATERMARK n computes ( f, x ) , f : { 0 , 1 } n →{ 0 , 1 } , x ∈ ( { 0 , 1 } n ) q ( n ) , and B n outputs y = B n ( f, x ) , y ∈ { 0 , 1 } q ( n ) , where f is sent using the representation R n , satisfies the following

Correctness ( f has low error). With probability at least l

Uniqueness (models trained from scratch give low-error answers). There exists a circuit B n of size S A ( n ) such that with probability at least c

Unremovability (fast B n give high-error answers). For every circuit B n of size at most S B ( n ) with probability at most s

Undetectability (fast B n cannot detect that they are tested). On average over ( D n , h n ) ∼ L n , distributions D q ( n ) n and x ∼ A WATERMARK n are s 2 -indistinguishable for a class of circuits B n of size at most S B ( n ) , i.e., for every circuit B n of size at most S B ( n ) returning one bit,

Definition 9 ( Adversarial Defense ) . Let L = { L n } n be a learning task, and F = {F n } n a function class. Let S A , S B , q : ◆ → ◆ , ϵ ∈ ( 0 , 1 2 ) , l, c, s ∈ (0 , 1) , with s < c , where S A ( n ) bounds the circuit size of A n , and S B ( n ) the circuit size of B n , q ( n ) the number of queries, ϵ the error parameter, c the completeness, s the soundness, and l the learning probability.

We say that a family of circuits B DEFENSE = { B DEFENSE n } n with complexity S A ( n ) implements an adversarial defense for L with frequency α , denoted by

if the following is true with frequency α over parameter n . An interactive protocol in which first ( D n , h n ) ∼ L n , B DEFENSE n computes f : { 0 , 1 } n → { 0 , 1 } , A n replies with x = A n ( f n ) , x ∈ ( { 0 , 1 } n ) q ( n ) , and B DEFENSE n outputs b = B DEFENSE n ( f, x ) , b ∈ { 0 , 1 } , satisfies the following:

Correctness ( f n has low error). With probability at least l

Completeness (natural inputs are not flagged as adversarial). When x ∼ D q ( n ) n , with probability at least c

Soundness (adversarial inputs are detected). For every circuit A n of size at most S A ( n ) , with probability at most s

Definition 10 ( Transferable Attack ) . Let L = { L n } n be a learning task and F = {F n } n a function class. Let S A , S B , q : ◆ → ◆ , ϵ ∈ ( 0 , 1 2 ) , and c, s ∈ (0 , 1) , with s < c , where S A ( n ) bounds the circuit size of A n , and S B the circuit size of B n , q ( n ) the number of queries, ϵ the error parameter, c the transferability probability, and s the undetectability probability.

We say that a family of circuits A TRANSFATTACK = { A TRANSFATTACK n } with complexity S A ( n ) implements a transferable attack for L with frequency α , denoted by

if the following is true with frequency α over parameter n . An interactive protocol in which first ( D n , h n ) ∼ L n , A TRANSFATTACK n computes x ∈ ( { 0 , 1 } n ) q ( n ) , and B n outputs y = B n ( x ) , y ∈ ( { 0 , 1 } ) q ( n ) , satisfies the following:

Transferability (fast provers return high-error answers). For every circuit B n of size at most S B ( n ) , with probability at least c

Undetectability (fast provers cannot detect that they are tested). On average over ( D n , h n ) ∼ L n , distributions x ∼ D q ( n ) n and x := A TRANSFATTACK n are s 2 -indistinguishable for every circuit B n of size at most S B ( n ) , i.e.,

D Main Theorem

Before proving our main theorem we recall a result from Lipton and Young [1994a] about simple strategies for large zero-sum games.

Game theory. A two-player zero-sum game is specified by a payoff matrix G . G is an r × c matrix. MIN, the row player, chooses a probability distribution p 1 over the rows. MAX, the column player, chooses a probability distribution p 2 over the columns. A row i and a column j are drawn from p 1 and p 2 and MIN pays G ij to MAX. MIN tries to minimize the expected payment; MAX tries to maximize it.

By the Min-Max Theorem, there exist optimal strategies for both MIN and MAX. Optimal means that playing first and revealing one's mixed strategy is not a disadvantage. Such a pair of strategies is also known as a Nash equilibrium. The expected payoff when both players play optimally is known as the value of the game and is denoted by V ( G ) .

We will use the following theorem from Lipton and Young [1994a], which says that optimal strategies can be approximated by uniform distributions over sets of pure strategies of size O (log( c )) .

Theorem 4 (Lipton and Young [1994a]) . Let G be an r × c payoff matrix for a two-player zero-sum game. For any η ∈ (0 , 1) and k ≥ log( c ) 2 η 2 there exists a multiset of pure strategies for the MIN (row player) of size k such that a mixed strategy p 1 that samples uniformly from this multiset satisfies

where G max , G min denote the maximum and minimum entry of G respectively. The symmetric result holds for the MAX player.

We are ready to prove our main theorem.

Theorem 5. Let ϵ ∈ ( 0 , 1 2 ) , δ ∈ ( 0 , 1 48 ) , S : ◆ → ◆ . For every learning task L = { L n } n learnable to error ϵ with confidence 1 -δ and circuit complexity O ( √ S ( n ) log( S ( n )) ) and for every family √

of function classes F = {F n } n , every query bound q ( n ) such that S ( n ) log( S ( n )) = Ω( m ( n ) + q ( n ) · n ) at least one of the three

exists with frequency 1 3 .

Proof. Let ϵ ∈ (0 , 1 2 ) and q : ◆ → ◆ be a query bound. Let L be a learning task learnable to error ϵ with confidence 1 -δ and complexity S ( n ) .

We will consider every n separately and show that for every n , one of the three schemes exists. This automatically implies that one of the schemes exists with frequency at least 1 3 .

be a set of s ( n ) -sized circuits computing ( f, x ) . Recall that the execution of a A n ∈ C W ( n ) proceeds by first sampling from Ex ( D n , h n ) and providing these samples as inputs to A n and then running A n to obtain m + q · n bits. The first m bits are interpreted as a representation of f (according to R n ), and the following consecutive blocks of n bits each are interpreted as q elements of { 0 , 1 } n . Similarly, let C D ( n ) be a set of s ( n ) -sized circuits accepting as input ( f, x ) and outputting ( y , b ) ,

where y ∈ { 0 , 1 } q , b ∈ { 0 , 1 } . Formally, this is a set of circuits with up to s ( n ) input gates and q +1 output gates. We interpret C W ( n ) as candidate algorithms for a watermark, and C D ( n ) as candidate algorithms for attacks on watermarks.

For every n define a zero-sum game G n between A n ∈ C W ( n ) , B n ∈ C D ( n ) . The payoff is given by

where A n tries to minimize and B n maximize the payoff.

Then the number of possible circuits is bounded by

because every internal gate of a circuit is one of AND, OR, and NOT, and is connected to 2 gates out of at most s ( n ) choices.

Applying Theorem 4 to G n with η = 2 -5 we get two probability distributions, p over a multiset of pure strategies in C W and r over a multiset of pure strategies in C D that lead to a 2 -5 -approximate Nash equilibrium. The size k ( n ) of the multisets is bounded

Next, observe that the mixed strategy corresponding to the distribution p can be represented by a circuit of size

because we can create a circuit that is a collection of k ( n ) circuits corresponding to the multiset of p , where each one is of size s ( n ) with additional gadgets of size O (log( k )) activating the corresponding gate depending on the randomness determining a strategy. This implies that p can be implemented by a S ( n ) -sized circuit. The same holds for r . Let's call the strategy corresponding to p , A n Nash , and the strategy corresponding to r , B n Nash .

Consider cases:

Case G ( A NASH n , B NASH n ) ≥ 19 24 . Define B DEFENSE n to work as follows:

Simulate the circuit of size O ( √ S ( n ) log( S ( n )) ) L n that learns f , such that P ( D n ,h n ) ∼ L n , f ← L Ex ( D n,hn ) n [ err ( f ) ≤ ϵ ] ≥ 1 -1 48 .

Send f to A n .

Receive x from A n .

Simulate ( y , b ) := B NASH n ( f, x ) .

Return b ′ = 1 if b = 1 or d ( f ( x ) , y ) > 3 ϵ · q ( n ) and b ′ = 0 otherwise,

,

where d ( · , · ) is the Hamming distance. B DEFENSE n can be implemented by circuit of size O ( S ( n )) because it simulates a circuit of size O ( √ S ( n ) log( S ( n )) ) , then simulating B NASH n of size S ( n ) , and computing a predicate d ( f ( x ) , y ) > 3 ϵq , which can be done in size log( q ( n )) .

We claim that we have:

Assume towards contradiction that completeness or soundness of B DEFENSE n as defined in Definition 9 does not hold.

If completeness of B DEFENSE n does not hold, then

Let us compute the payoff of A n , which first runs f ← L Ex ( D n ,h n ) n (where L n is the learning circuit) and sets x ∼ D q , in the game G n , when playing against B NASH n

where the contradiction is with the properties of Nash equilibria.

Assume that A n breaks the soundness of B DEFENSE n , which translates to

Let A ′ n first simulate f ← L Ex ( D n ,h n ) n , then runs x ← A n ( f ) , and returns ( f, x ) . We have

where the contradiction is with the properties of Nash equilibria. Thus equation (2) holds.

Case G n ( A NASH n , B NASH n ) < 19 24 . Consider B n that returns ( f ( x ) , b ) for a uniformly random b . We have

because when x ∼ D q n and err ( f ) ≤ ϵ the probability that err ( x , y ) ≤ 2 ϵ and b = 0 is 1 2 , and similarly when x ← A NASH n then the probability that b = 1 is equal 1 2 . The assumption that G n ( A Nash n , B n ) < 19 24 and properties of Nash equilibria imply that P ( D n ,h n ) ∼ L n , f ← A Nash n [ err ( f ) ≤ ϵ ] ≥ 10 24 .

This implies that correctness holds for A Nash n with l = 10 24 .

Next, assume towards contradiction that unremovability of A NASH n does not hold, i.e., there is B n running in time o ( √ S ( n ) / log( S ( n )) ) such that P [ err ( x , y ) ≤ 2 ϵ ] > 19 24 . Consider B ′ n that on input ( f, x ) returns ( B n ( f, x ) , 0) . Then by definition of G n , G n ( A NASH , B ′ n ) > 19 24 , which is a contradiction .

Next, assume towards contradiction that undetectability of A NASH n does not hold, i.e., there exists B n such that it distinguishes x ∼ D q n from x ← A NASH n with probability higher than 19 24 . Consider B ′ n that on input ( f, x ) returns ( f ( x ) , B n ( f, x )) . 11 Then by definition of G n , G n ( A NASH n , B ′ n ) > 19 24 , which is a contradiction .

There are two further subcases. If A NASH n satisfies uniqueness then

If A NASH n does not satisfy uniqueness , then, by definition, every succinctly representable circuit B n of size o ( √ S ( n ) / log( S ( n )) ) satisfies err ( x , y ) ≤ 2 ϵ with probability at most 21 24 . Consider the following A n . It computes ( f, x ) ← A Nash n , ignores f and sends x to B n . By the assumption that uniqueness is not satisfied for A NASH n transferability of Definition 5 holds for A n with c = 3 24 . Note

that B n in the transferable attack does not receive f but it makes it no easier for it to satisfy the properties. Note that undetectability still holds with the same parameter. Thus

E Fully Homomorphic Encryption (FHE)

We include a definition of fully homomorphic encryption based on the definition from Goldwasser et al. [2013]. The notion of fully homomorphic encryption was first proposed by Rivest, Adleman and Dertouzos Rivest et al. [1978] in 1978. The first fully homomorphic encryption scheme was proposed in a breakthrough work by Gentry in 2009 Gentry [2009]. A history and recent developments on fully homomorphic encryption is surveyed in [Vaikuntanathan, 2011].

E.1 Preliminaries

We say that a function f is negligible in an input parameter λ , if for all d > 0 , there exists K such that for all λ > K , f ( λ ) < λ -d . For brevity, we write: for all sufficiently large λ , f ( λ ) = negl ( λ ) . We say that a function f is polynomial in an input parameter λ , if there exists a polynomial p such that for all λ , f ( λ ) ≤ p ( λ ) . We write f ( λ ) = poly ( λ ) . A similar definition holds for polylog ( λ ) . For two polynomials p, q , we say p ≤ q if for every λ ∈ , p ( λ ) ≤ q ( λ ) .

◆ When saying that a Turing machine A is p.p.t. we mean that A is a non-uniform probabilistic polynomial-time machine.

E.2 Definitions

Definition 11 (Goldwasser et al. [2013]) . A homomorphic (public-key) encryption scheme FHE is a quadruple of polynomial time algorithms (FHE.KEYGEN, FHE.ENC, FHE.DEC, FHE.EVAL) as follows:

FHE.KEYGEN (1 λ ) is a probabilistic algorithm that takes as input the security parameter 1 λ and outputs a public key pk and a secret key sk .

FHE.ENC ( pk, x ∈ { 0 , 1 } ) is a probabilistic algorithm that takes as input the public key pk and an input bit x and outputs a ciphertext ψ .

FHE.DEC ( sk, ψ ) is a deterministic algorithm that takes as input the secret key sk and a ciphertext ψ and outputs a message x ∗ ∈ { 0 , 1 } .

FHE.EVAL ( pk, C, ψ 1 , ψ 2 , . . . , ψ n ) is a deterministic algorithm that takes as input the public key pk , some circuit C that takes n bits as input and outputs one bit, as well as n ciphertexts ψ 1 , . . . , ψ n . It outputs a ciphertext ψ C .

Compactness: For all security parameters λ , there exists a polynomial p ( · ) such that for all input sizes n , for all x 1 , . . . , x n , for all C , the output length of FHE.EVAL is at most p ( n ) bits long.

Definition 12 ( C -homomorphism, Goldwasser et al. [2013] ) . Let C = { C n } n ∈ N be a class of boolean circuits, where C n is a set of boolean circuits taking n bits as input. A scheme FHE is C -homomorphic if for every polynomial n ( · ) , for every sufficiently large security parameter λ , for every circuit C ∈ C n , and for every input bit sequence x 1 , . . . , x n , where n = n ( λ ) ,

̸

where the probability is over the coin tosses of FHE.KEYGEN and FHE.ENC.

Definition 13 ( Fully homomorphic encryption ) . A scheme FHE is fully homomorphic if it is homomorphic for the class of all arithmetic circuits over GF (2) .

Definition 14 ( Leveled fully homomorphic encryption ) . A leveled fully homomorphic encryption scheme is a homomorphic scheme where FHE.KEYGEN receives an additional input 1 d and the resulting scheme is homomorphic for all depthd arithmetic circuits over GF (2) .

Definition 15 ( IND-CPA security ) . A scheme FHE is IND-CPA secure if for any p.p.t. adversary A ,

We now state the result of Brakerski, Gentry, and Vaikuntanathan [Brakerski et al., 2012] that shows a leveled fully homomorphic encryption scheme based on a standard assumption in cryptography called Learning with Errors [Regev, 2005]:

Theorem 6 ( Fully Homomorphic Encryption, definition from Goldwasser et al. [2013] ) . Assume that there is a constant 0 < ϵ < 1 such that for every sufficiently large ℓ , the approximate shortest vector problem gapSVP in ℓ dimensions is hard to approximate to within a 2 O ( ℓ ϵ ) factor in the worst case. Then, for every n and every polynomial d = d ( n ) , there is an IND-CPA secure d -leveled fully homomorphic encryption scheme where encrypting n bits produces ciphertexts of length poly ( n, λ, d 1 /ϵ ) , the size of the circuit for homomorphic evaluation of a function f is size ( C f ) · poly ( n, λ, d 1 /ϵ ) and its depth is depth ( C f ) · poly (log n, log d ) .

F Existence of Transferable Attacks

Learning Theory Preliminaries. For the next lemma, we will consider a slight generalization of learning tasks to the case where there are many valid outputs for a given input. This can be understood as the case of generative tasks. More concretely, we assume that for the input space X n the output space is Y n instead of { 0 , 1 } . It will always be the case that X n and Y n are equal to { 0 , 1 } p ( n ) for some polynomial p . For a distribution D n over X n we call a function h : X n ×Y n →{ 0 , 1 } an error oracle if the error of a function f : X n →Y n is defined as

where the randomness of expectation includes the potential randomness of f . The example oracle Ex provides access to samples ( x, y ) ∈ X n ×Y n , where x ∼ D n and y ∈ Y n is some y such that h ( x, y ) = 0 .

The following learning task will be crucial for our construction.

Definition 16 ( Lines on a Circle Learning Task L ◦ ) . We define L ◦ = { L ◦ n } n . For every n we define X n = { 0 , 1 } n and associate X n with vertices of a 2 n regular polygon inscribed in the unit circle { x ∈ R 2 | ∥ x ∥ 2 = 1 } . The output space is {-1 , +1 } for all n . Let H := { h w | w ∈ R 2 , ∥ w ∥ 2 = 1 } , where h w ( x ) := sgn ( ⟨ w,x ⟩ ) . For every n , let L ◦ n be the distribution corresponding to the following process: sample h w ∼ U ( H ) , return ( U ( X n ) , h w ) . Note that H has VC-dimension equal to 2 so L is learnable to error ϵ with O ( 1 ϵ ) samples for every n and every ϵ .

Moreover, for n ∈ ◆ define B w n ( α ) := { x ∈ X n | | ∡ ( x, w ) | ≤ α } . Lemma 3 ( Learning lower bound for L ◦ ) . Let n ∈ ◆ . Let L n be a learning algorithm for L ◦ n (Definition 16) that uses K samples and returns a classifier f : X n →{-1 , +1 } . Then

̸

Proof. Let n ∈ ◆ . Consider the following algorithm A . It first simulates L n on K samples to compute f . Next, it performs a smoothing of f , i.e., computes

Note that if err ( f ) ≤ η for a ground truth h w then for every x ∈ X n \ B x n (2 πη ) we have f η ( x ) = h w ( x ) . This implies that A can be adapted to an algorithm that with probability 1 finds w ′ such that | ∡ ( w,w ′ ) | ≤ err ( f ) .

Assuming towards contradiction that the statement of the lemma does not hold it means that there is an algorithm using K samples that with probability 3 100 locates w up to angle 1 2 K .

Consider any algorithm A using K samples. Probability that A does not see any sample in B w n (2 πη ) is at least

which is bigger than 1 -3 100 if we set η = 1 2 K . But note that if there is no sample in B w n (2 πη ) then A cannot locate w up to η with certainty. This proves the lemma.

Lemma 4 ( Boosting for L ◦ ) . Let η, ν ∈ (0 , 1 4 ) , n ∈ ◆ , L n be a learning algorithm for L ◦ n that uses K samples and outputs f : X n →{-1 , +1 } such that with probability δ

̸

where H is as defined earlier { h w | w ∈ R 2 , ∥ w ∥ 2 = 1 } . Then there exists a learning algorithm L ′ n for L ◦ n that uses max ( K, 9 η ) samples such that with probability δ -1 1000 returns f ′ such that

̸

Proof. Let n ∈ ◆ . L ′ n first draws max ( K, 9 η ) samples Q and defines g : X n →{-1 , +1 , ⊥} as follows, g maps to -1 the smallest continuous interval containing all samples from Q with label -1 . Similarly g maps to +1 the smallest continuous interval containing all samples from Q with label +1 . The intervals are disjoined by construction. Unmapped points are mapped to ⊥ . Next, L ′ n simulates L n with K samples and gets a classifier f that with probability δ satisfies the assumption of the lemma. Finally, it returns

̸

Consider 4 arcs defined as the 2 arcs constituting B w n (2 πη ) divided into 2 parts each by the line { x ∈ R 2 | ⟨ w,x ⟩ = 0 } . Let E be the event that some of these intervals do not contain a sample from Q . Observe that

By the union bound with probability δ -1 1000 , f satisfies equation (5) and E does not happen. By definition of f ′ this gives the statement of the lemma.

Theorem 7 ( Transferable Attack for a Cryptography based Learning Task ) . There exists a learning task L = { L λ } λ and a function class F = {F λ } λ such that for every ϵ : ◆ → ◆ where 1 /ϵ ( λ ) is lower-bounded by a sufficiently large polynomial and upper-bounded by some polynomial the following holds.

L is ( ϵ, δ = 1 10 , S = 10 3 ϵ 1 . 3 , F ) -learnable.

L is not ( ϵ, δ = 1 10 , S = 1 ϵ , F ) -learnable

There exists a circuit family A = { A λ } λ such that

Proof. The learning task is based on L ◦ = { L ◦ n } n from Definition 16.

Setting of Parameters for FHE. Observe that by assumption of the lemma p ≤ 1 /ϵ ≤ r , for some polynomial r , and a polynomial p that we will define later. Let FHE be a fully homomorphic encryption scheme from Theorem 6. We will use the scheme for constant leveled circuits d = O (1) . Let s ( n, λ, d ) be the polynomial bounding the size of the encryption of inputs of length n with λ security as well as bounding the size of the circuit for homomorphic evaluation, which is guaranteed to exist by Theorem 6. Let β ∈ (0 , 1) and p be a polynomial such that

which exist because s is a polynomial.

We define n ( λ ) := ⌊ p 1 /β ( λ ) ⌋ 12 for the length of inputs in the FHE scheme. Observe that for every λ

Learning Task. The learning task will be parametrized by λ , i.e. L = { L λ } λ .

Let λ ∈ ◆ . We define D λ := {D (pk,sk) λ } (pk,sk) , H λ := { h (pk,sk,w) λ } (pk,sk,w) (for D (pk,sk) λ and h (pk,sk,w) λ to be defined later), where they are indexed by valid public/secret key pairs of the FHE and w ∈ { x ∈ R 2 | ∥ x ∥ 2 = 1 } . Let L λ be defined as corresponding to the following process: sample ( pk,sk , w ) ∼ FHE.KEYGEN (1 λ ) × U ( { x ∈ R 2 | ∥ x ∥ 2 = 1 } ) , return ( D (pk,sk) λ , h (pk,sk,w) λ ) .

For a valid (pk,sk) pair we define D (pk,sk) as the result of the following process: x ∼ U ( { 0 , 1 } n ( λ ) ) , with probability 1 2 return (0 , x, pk ) and with probability 1 2 return (1 , FHE.ENC ( pk , x ) , pk ) , where the first element of the triple describes if the x is encrypted or not. Formally, in the case that the first element of the triple is 0 one needs to add a padding of size s ( n ( λ ) , λ, d ) -n ( λ ) so that descriptions have the same size in both cases. 13

̸

Note 1 ( Ω( 1 ϵ ) -sample learning lower bound. ) . By construction any learner using K samples for L λ (for any λ ) can be transformed (potentially computationally inefficiently) into a learner using K samples for L ◦ n ( λ ) (Defnition 16) that returns a classifier of the same error. This, together with a lower bound for learning from Lemma 3 proves point 2 of the lemma.

For a valid (pk,sk) pair and w ∈ { x ∈ R 2 | ∥ x ∥ 2 = 1 } we define h (pk,sk,w) (( b, x, pk ) , y ) as a result of the following algorithm: if b = 0 return ✶ h w ( x )= y , otherwise let x DEC ← FHE.DEC ( sk , x ) , y DEC ← FHE.DEC ( sk , y ) and if x DEC , y DEC = ⊥ (decryption is succesful) return ✶ h w ( x DEC )= y DEC and return 1 otherwise.

Definition of A (Algorithm 1). A λ draws N ( λ ) samples Q = { (( b i , x i , pk ) , y i ) } i ∈ [ N ] for N ( λ ) := 900 ϵ ( λ ) .

Next, A λ chooses a subset Q CLEAR ⊆ Q of samples for which b i = 0 . It trains a classifier f w ′ ( · ) := sgn ( ⟨ w ′ , ·⟩ ) on Q CLEAR by returning any f w ′ consistent with Q CLEAR . This can be done in time

by keeping track of the smallest interval containing all samples in Q CLEAR labeled with +1 and then returning any f w ′ consistent with this interval.

Note 2 ( O ( 1 . 3 ) -time learning upper bound. ) . First note that A λ learns well, i.e., with probability at

1 ϵ least 1 -2 ( 1 -ϵ ( λ ) 100 ) 900 ϵ ( λ ) ≥ 1 -1 1000 ,

̸

Algorithm 1 TRANSFATTACK ( Ex ( D λ , h λ ) , ϵ, λ )

1: Input: Access to the example oracle Ex ( D λ , h λ ) , where ( D λ , h λ ) ∼ L λ , error level ϵ : ◆ → ◆ , and the security parameter λ .

2: N := 900 /ϵ ( λ ) , q := 16 /ϵ ( λ )

3: Q = { (( b i , x i , pk ) , y i ) } ∈ ∼ ( D λ ) N ( λ

i [ N ] ) ▷ N ( λ ) i.i.d. samples from D λ

4: Q CLEAR = { (( b, x, pk ) , y ) ∈ Q : b = 0 } ▷ Q CLEAR ⊆ Q of unencrypted x 's

5: f w ′ ( · ) := sgn ( ⟨ w ′ , ·⟩ ) ← a line consistent with samples from Q CLEAR ▷ f w ′ : X n →{-1 , +1 }

6: { x ′ i } i ∈ [ q ( λ )] ∼ U ( ( X n ( λ ) ) q ( λ ) )

7: S ∼ U (2 [ q ( λ )] )

8: E BND ; = ∅

9: for i ∈ [ q ( λ ) -| S | ] do

x BND ∼ U ( B w ′ (2 π ( ϵ ( λ ) + ϵ ( λ ) )))

10: n ( λ ) 100 ▷ x BND is close to the decision boundary of f w ′

11: E BND := E BND ∪ { FHE.ENC ( pk , x BND ) }

12: end for

13: x := { (0 , x ′ i , pk ) | i ∈ [ q ( λ )] \ S } ∪ { (1 , x ′ , pk ) | x ′ ∈ E BND }

14: Return x

Moreover, f w ′ ( x ) can be implemented by a circuit C f w ′ that compares x with the endpoints of the interval. This can be done by a constant leveled circuit. Moreover C f w ′ can be evaluated with FHE.EVAL in time

where the last inequality follows from equation (7). This proves point 1 of the lemma.

Next, A λ prepares x as follows. It samples q ( λ ) = 16 ϵ ( λ ) points { x ′ i } i ∈ [ q ] from { 0 , 1 } n ( λ ) uniformly at random. It chooses a uniformly random subset S ⊆ [ q ( λ )] . Next, A λ generates q ( λ ) -| S | inputs using the following process: x BND ∼ U ( B w ′ n ( λ ) (2 π ( ϵ ( λ ) + ϵ ( λ ) 100 ))) ( x BND is close to the decision boundary of f w ′ ), return FHE.ENC ( pk , x BND ) . Call the set of q ( λ ) -| S | points E BND. A λ defines: x := { (0 , x ′ i , pk ) | i ∈ [ q ] \ S } ∪ { (1 , x ′ , pk ) | x ′ ∈ E BND } .

The running time of this phase is dominated by evaluations of FHE.EVAL, which takes

where the first inequality follows from equation (7). Taking the sum of equation (8) and equation (10) we get that A λ can be implemented by a circuit of size 10 3 ϵ 1 . 3 ( λ ) .

A λ Constitutes a Transferable Attack. Now, consider B λ of size S B ( λ ) = 1 ϵ 2 ( λ ) . By the assumption S B ( λ ) ≤ r ( λ ) , which implies that the security guarantees of FHE hold for B λ .

Weclaim that x is indistinguishable from D (pk,sk) λ for B λ . Observe that by construction the distribution of ratio of encrypted and not encrypted x 's in x is identical to that of D (pk,sk) λ . Moreover, the distribution of unencrypted x 's is identical to that of D (pk,sk) λ by construction. Finally, by the INDCPA security 14 of FHE and the fact that the size of B λ is bounded by some polynomial in λ , FHE.ENC ( pk , x BND ) is distinguishable from x ∼ X n , FHE.ENC ( pk , x ) with advantage at most negl ( λ ) . Thus undetectability holds with near perfect soundness s = 1 2 + negl ( λ ) .

Next, we claim that B λ can't return low-error answers on x .

Assume towards contradiction that with probability 5

̸

▷ S ⊆ [ q ( λ )] a uniformly random subset

We can apply Lemma 4 to get that there exists a learner using 1 100 ϵ 2 ( λ ) + 9 ϵ ( λ ) ≤ 1 90 ϵ 2 ( λ ) samples that with probability 4 100 returns f ′ such that

̸

Applying Lemma 3 to equation (12) we know that

which is a contradiction. Thus equation (11) does not hold and in consequence, using equation (9), with probability 1 -6 100

̸

where crucially x is sampled from U ( B w ′ n ( λ ) ) and not U ( B w n ( λ ) ) . By Fact 1 we know that | S | ≥ q ( λ ) 3 with probability at least

Using the setting of q ( λ ) = 16 ϵ ( λ ) and applying the Chernoff bound and the union bound we get from equation (13) that with probability at least 1 -1 10 the error err ( x , y ) is larger than 2 ϵ ( λ ) .

Note 3. We want to emphasize that it is crucial (for our construction) that the distribution has both an encrypted and an unencrypted part.

As mentioned before, if there was no D CLEAR then A λ would see only samples of the form

and would not know which of them lie close to the boundary of h w , and so it would not be able to choose tricky samples. A λ would be able to learn a low-error classifier, but only under the encryption. More concretely, A λ would be able to homomorphically evaluate a circuit that, given a training set and a test point, learns a good classifier and classifies the test point with it. However, it would not be able to, with high probability, generate FHE.ENC ( x ) , for x close to the boundary as it would not know (in the clear) where the decision boundary is.

If there was no D ENC then everything would happen in the clear and so B would be able to distinguish x 's that appear too close to the boundary.

Fact 1 ( Chernoff-Hoeffding ) . Let X 1 , . . . , X k be independent Bernoulli variables with parameter p . Then for every 0 < ϵ < 1

and

Also for every δ > 0

G Transferable Attacks Imply Cryptography

G.1 EFID Pairs

The typical way in which security of EFID pairs is defined, e.g., in [Goldreich, 1990], is that they should be secure against all polynomial-time algorithms. However, for the case of pseudorandom generators (PRGs), which are known to be equivalent (in the standard definition) to EFIDs pairs, more granular notions of security were considered. For instance, in [Nisan, 1990] the existence of PRGs secure against adversaries running in time bounded by a fixed, in contrast to all, polynomial, was studied. In a similar spirit, we consider EFID pairs that are secure against adversaries with fixed circuit complexity bounds.

Definition 17 ( Total Variation ) . For two distrbutions D 0 , D 1 over a finite domain { 0 , 1 } n we define their total variation distance as

Definition 18 ( EFID pairs ) . For parameters η, δ : ◆ → (0 , 1) and circuit complexity bounds S, S ′ : ◆ → ◆ we call a pair of ensembles of distributions ( D 0 = {D 0 n } n , D 1 = {D 1 n } n ) over domain X = {X n } n an ( S, S ′ , η, δ ) -EFID pair if for every n

The circuit complexity of sampling D 0 and D 1 is at most S ,

For every n , △ ( D 0 n , D 1 n ) ≥ η ( n ) ,

For every n , D 0 n , D 1 n are δ ( n ) -indistinguishable for circuits with complexity S ′ ( n ) .

Observe that Definition 18 is a generalization of the standard definition. Indeed, for every EFID pair ( D 0 , D 1 ) according to the standard definition there exists an inverse polynomial function η and a polynomial S such that for all polynomials S ′ there exists a negligible function δ such that ( D 0 , D 1 ) is an ( S, S ′ , η, δ ) -EFID pair.

G.2 Transferable Attacks imply EFID pairs

Theorem 8 ( Tasks with Transferable Attacks Imply EFID pairs ) . For every ϵ ∈ (0 , 1) , q ∈ ◆ , S A , S B : ◆ → ◆ such that S A ≤ S B , every learning task L learnable to error ϵ with confidence p and circuit complexity S A , every c, s ∈ (0 , 1) if

exists with frequency 1 3 then there exist S ′ A , S ′ B : ◆ → ◆ that agree with S A and S B respectively with frequency 1 3 and there exists

Proof. Let ϵ, S A , S B , q, c, s, p, L be as in the assumption of the theorem. Additionally let A = { A n } n be a family of circuits certifying that a Transferable Attack exists with frequency 1 3 for L .

For every n , define D 0 n := D q n , where we recall that q is the number of samples A n sends in the attack. Define D 1 n to be the distribution of x := A n . Note that x ∈ ( X n ) q .

Let a : ◆ →{ 0 , 1 } be a sequence certifying that a Transferable Attack exists with frequency 1 3 . Let n be such that a ( n ) = 1 . Observe that D 0 n , D 1 n are samplable with circuit complexity S A ( n ) because A n complexity is bounded by S A ( n ) . Secondly, D 0 n , D 1 n are s 2 -indistinguishable for S B ( n ) -sized adversaries by undetectability of A n . Finally, the fact that D 0 n , D 1 n are statistically far follows from transferability . Indeed, the following procedure accepting input x ∈ ( { 0 , 1 } n ) q is a distinguisher:

Run the learner (the existence of which is guaranteed by the assumption of the theorem) to obtain f .

y := f ( x ) .

If err ( x , y ) ≤ 2 ϵ return 0 , otherwise return 1 .

If x ∼ D 0 = D q then err ( f ) ≤ ϵ with probability p . By Fact 1 and the union bound we also know that err ( x , y ) ≤ 2 ϵ with probability p -e -ϵq 3 and so, the distinguisher will return 0 with probability p -e -ϵq 3 . On the other hand, if x ∼ D 1 = A we know from transferability of A n that every algorithm running in time S B ( n ) will return y such that err ( x , y ) > 2 ϵ with probability at least c . By the assumption that S B ( n ) ≥ S A ( n ) we know that err ( x , f ( x )) > 2 ϵ with probability at least c also. Consequently, the distinguisher will return 1 with probability at least c in this case. By the properties of total variation this implies that △ ( D 0 n , D 1 n ) ≥ 1 2 ( p + c -1 -e -ϵq 3 ) .

We define a pair of families of distributions ̂ D 0 , ̂ D 1 and functions S ′ A , S ′ B as follows. For every n such that a ( n ) = 1 we define ̂ D 0 n = D 0 n , ̂ D 1 = D 1 n , S ′ A ( n ) = S A ( n ) , S ′ B ( n ) = S B ( n ) . For every n sich that a ( n ) = 0 we define ̂ D 0 n = D 0 k for the smallest k > n such that a ( k ) = 1 , and S ′ A ( n ) = S A ( k ) And analogously for ̂ D 1 n and S ′ B .

Simple verification yields that ̂ D 0 n , ̂ D 1 n is an ( S ′ A , S ′ B , 1 2 ( p + c -1 -e -ϵq 3 ) , s 2 ) -EFID pair.

Note 4 ( Setting of parameters ) . Observe that if p ≈ 1 , i.e., it is possible to almost surely learn f in time S A such that err ( f ) ≤ ϵ , c is a constant, q = Ω( 1 ϵ ) then η in the parameters for the EFID is a constant and so △ ( D 0 , D 1 ) is a constant.

Note 5. We want to emphasize that our distinguisher crucially uses the error oracle in its last step. So it is possible that it is not implementable for all circuit complexity bounds!

H Adversarial Defenses exist

Our result is based on [Goldwasser et al., 2020]. Before we state and prove our result we give an overview of the learning model considered in [Goldwasser et al., 2020]. The authors give a defense against arbitrary examples in a transductive model with rejections. In contrast, our model does not allow rejections, but we do require indistinguishability.

H.1 Transductive Learning With Rejections.

In [Goldwasser et al., 2020] the authors consider a model, where a learner L receives a training set of labeled samples from the original distribution ( x D , y D = h ( x D )) , x ∼ D N , y D ∈ {-1 , +1 } N , where h is the ground truth, together with a test set x T ∈ ( { 0 , 1 } n ) q . Next, L uses ( x D , y D , x T ) to compute y T ∈ {-1 , +1 , ⊔ ⊓} q , where ⊔ ⊓ represents that L abstains (rejects) from classifying the corresponding x .

Before we define when learning is successful, we will need some notation. For q ∈ ◆ , x ∈ ( { 0 , 1 } n ) q , y ∈ {-1 , +1 , ⊔ ⊓} q we define

̸

̸

̸

which means that we count ( x, y ) ∈ { 0 , 1 } n ×{-1 , +1 , ⊔ ⊓} as an error if h is well defined on x , y is not an abstantion and h ( x ) = y .

̸

Learning is successful if it satisfies two properties.

If x T ∼ D q then with high probability err ( x T , y T ) and ⊔ ⊓ ( y T ) are small.

For every x T ∈ ( { 0 , 1 } n ) q with high probability err ( x T , y T ) is small. 15

The formal guarantee of a result from Goldwasser et al. [2020] are given in Theorem 9. Let's call this model Transductive Learning with Rejections (TLR).

Note the differences between TLR and our definition of Adversarial Defenses. To compare the two models we associate the learner L from TLR with B in our setup, and the party producing x T with A in our definition. First, in TLR, B does not send f to A . Secondly, and most importantly, we do not allow B to reply with rejections ( ⊔ ⊓ ) but instead require that B can 'distinguish' that it is being tested (see soundness of Definition 9). Finally, there are no apriori time bounds on either A or B in TLR. The models are similar but a priori incomparable and any result for TLR needs to be carefully analyzed before being used to prove that it is an Adversarial Defense.

H.2 Formal guarantee for Transductive Learning with Rejections (TLR)

Theorem 5.3 from Goldwasser et al. [2020] adapted to our notation reads.

Theorem 9 ( TLR guarantee (Goldwasser et al. [2020]) ) . For any N ∈ ◆ , ϵ ∈ (0 , 1) , h ∈ H and distribution D over { 0 , 1 } n :

where ϵ ∗ = √ 2 d N log (2 N ) + 1 N log ( 1 ϵ ) and f = REJECTRON ( x D , h ( x D ) , x T , ϵ ∗ ) , where f : { 0 , 1 } n → {-1 , +1 , ⊔ ⊓} and d denotes the VC-dimension on H . REJECTRON is defined in Figure 2. in [Goldwasser et al., 2020].

REJECTRON is an algorithm that accepts a labeled training set ( x D , h ( x D )) and a test set x T and returns a classifier f , which might reject some inputs. The learning is successful if with a high probability f rejects a small fraction of D N and for every x T ∈ { 0 , 1 } nN the error on labeled x 's in x T is small.

H.3 Adversarial Defense for bounded VC-dimension

We are ready to state the main result of this section.

Lemma 5 ( Adversarial Defense for bounded VC-dimension ) . Let {H n } n be a family of hypothesis classes such that there exists a polynomial p such that for every n , H n has a VC-dimension bounded by p ( n ) . There exists a family of circuits B = { B n } n such that for every L satisfying for every n that the support of the marginal of L n is contained in H n , i.e., the ground truth sampled from L are always in H , such that for every sufficiently small ϵ ,

Note that, by the PAC learning bound, this is a setting of parameters, where B has enough time to learn a classifier of error ϵ . By slightly abusing the notation, we write S A = ∞ , meaning that the defense is secure against all adversaries regardless of their running time.

Proof. The proof is based on an algorithm from Goldwasser et al. [2020].

Construction of B . Let ϵ ∈ (0 , 1) , n ∈ N , d ( n ) be the VC-dimension of H n and

Let q := N . First, B , draws N labeled samples ( x FRESH , h ( x FRESH )) . Next, it finds f ∈ H consistent with them and sends f to A . Importantly this computation is the same as the first step of REJECTRON.

̸

Next, B receives as input x ∈ ( { 0 , 1 } n ) q from A . B . Let ϵ ∗ := √ 2 d N log (2 N ) + 1 N log ( 1 ϵ ) . Next B runs f ′ = REJECTRON ( x FRESH , h ( x FRESH ) , x , ϵ ∗ ) , where REJECTRON is starting from the second step of the algorithm (Figure 2 [Goldwasser et al., 2020]). Importantly, for every x ∈ { 0 , 1 } n , if f ′ ( x ) = ⊔ ⊓ then f ( x ) = f ′ ( x ) . In words, f ′ is equal to f everywhere where f ′ does not reject.

Finally B returns 1 if ⊔ ⊓ ( f ′ ( x )) > 2 3 ϵ , and returns 0 otherwise.

B is a Defense. First, by the standard PAC theorem, with probability at least 1 -ϵ , err ( f ) ≤ ϵ 2 . This means that correctness holds with probability l = 1 -ϵ .

Note that with our setting of N ,

Theorem 9 guarantees that

if x ∈ D q then with probability at least 1 -ϵ ,

which in turn implies that with the same probability B returns b = 0 . This implies that completeness holds with probability 1 -ϵ .

for every x ∈ ( { 0 , 1 } n ) q with probability at least 1 -ϵ ,

To compute soundness we want to upper bound the probability that err ( x , f ( x )) > 2 ϵ 16 and b = 0 . By construction of B if b = 0 then ⊔ ⊓ ( f ′ ( x )) ≤ 2 ϵ 3 , which means that with probability at least 1 -ϵ

This translates to soundness holding with s = ϵ .

REJECTRON can be implemented by a circuit of size polynomial in N and makes O ( 1 ϵ ) calls to an Empirical Risk Minimizer on H (that we assume can be implemented by a circuit of size polynomial in d ), which implies the promised circuit complexity.

I Watermarks exist

Lemma 6 ( Watermark for bounded VC-dimension against fast adversaries ) . There exists a family of hypothesis classes {H d } d such that for every d , H d has VC-dimension d and a family of distributions {D d } d such that for every ϵ ∈ ( 10000 d , 1 8 ) there exists a family of circuits A = { A d } d and a family of function classes F for which the following conditions hold. For every learning L = { L d } d that for every d samples D d always and h d ∈ H d ,

Note that the setting of parameters is such that A can learn (with high probability) a classifier of error ϵ , but B is not able to learn a low-error classifier within its allotted circuit size S B . This contrasts with Lemma 5, where B has a sufficiently large circuit size to learn. This is the regime of interest for Watermarks, where the scheme is expected to be secure against B with limited circuit complexity.

Proof. Let D be the uniform distribution over [ N ] for N = 100 d 2 , where recall that [ N ] = { 1 , . . . , N } . Let H be the concept class of functions that have exactly d +1 's in [ N ] . Note that H has VC-dimension d . Let h ∈ H be the ground truth.

Construction of A . A works as follows. It draws n = O ( d ϵ ) samples from D labeled with h . Let's call them x TRAIN . Let

A takes a uniformly random subset A w ⊆ A of size q . It defines sets

A computes f consistent with the training set { ( x, +1) : x ∈ A ′ } ∪ { ( x, -1) : x ∈ B ′ } . A samples S ∼ D q . It defines the watermark to be x := A w with probability 1 2 and x := S with probability 1 2 .

A sends ( f, x ) to B . A can be implemented with circuit complexity O ( d ϵ ) .

A is a Watermark. We claim that ( f, x ) constitutes a watermark.

It is possible to construct a watermark of prescribed size, i.e., find a subset A w of a given size, only if | A | ≥ q . The probability that a single sample from D is labeled +1 is d N , so by the Chernoff bound (Fact 1) | A | , | B | > dn 2 N ≥ q with probability 1 -1 100 , where we used that n = O ( d ϵ ) , N = 100 d 2 , q = O ( 1 ϵ ) .

Correctness. Let h ′ ( x ) := h ( x ) if x ∈ [ N ] \ A w and h ′ ( x ) := -h ( x ) otherwise. Note that h ′ has exactly d -q +1 's in [ N ] . By construction, f is a classifier consistent with h ′ . By the PAC theorem we know that with probability 1 -1 100 , f has an error at most ϵ wrt to h ′ (because the hypothesis class of functions with at most d +1 's has a VC dimension of O ( d ) ). h ′ differs from h on q points, so

with probability 1 -1 100 , which implies that correctness is satisfied with l = 1 -1 100 .

Distinguishing of x and D q . Note that the distribution of A w is the same as the distribution of a uniformly random subset of [ N ] of size q (when taking into account the randomness of the choice of h ∼ U ( H ) ). Observe that the probability that drawing q i.i.d. samples from U ([ N ]) we encounter repetitions is at most

because q < d 100 < √ N 10 . This means that 1 100 is an information-theoretic upper bound on the distinguishing advantage between x = A w and D q .

Moreover, B has access to at most t samples and the probability that the set of samples B draws from D t and A w have empty intersection is at least 1 -1 100 . It is because it is at least (1 -t N ) t ≥ (1 -1 √ ) √ N/ 10 ≥ 1 -1 100 , where we used that t < √ N 10 . 17

Note that by construction f maps all elements of A w to -1 . The probability over the choice of F ∼ D q that F ⊆ h -1 ( {-1 } ) , i.e., all elements of F have true label -1 , is at least

The three above observations and the union bound imply that the distinguishing advantage for distinguishing x from D q of B is at most 4 100 and so the undetectability holds with s = 8 100 .

Unremovability. Assume, towards contradiction with unremovability , that B can find y that with probability s ′ = 1 2 + 6 100 satisfies err ( x , y ) ≤ 2 ϵ . Notice, that err ( A w , f ( A w )) = 1 by construction.

Consider an algorithm A for distinguishing A w from D q . Upon receiving ( f, x ) it first runs y = B ( f, x ) and returns 1 iff d ( y , f ( x )) ≥ q 2 . We know that the distinguishing advantage is at most 1 2 + 4 100 , so

But also note that

Combining the two above equations we get a contradiction and thus the unremovability holds with s ′ = 1 2 + 6 100 .

Uniqueness. The following B certifies uniqueness . It draws O ( d ϵ ) samples from D , let's call them x ′ TRAIN and trains f ′ consistent with it. By the PAC theorem err ( f ′ ) ≤ ϵ with probability at least 1 -1 100 . Next upon receiving x ∈ { 0 , 1 } nq = [ N ] q it returns y = f ′ ( x ) . By the fact that x is a random subset of [ N ] of size q by the Chernoff bound, the union bound we know that err ( x , y ) = err ( x , f ′ ( x )) ≤ 2 ϵ with probability at least 1 -2 100 over the choice of h . This proves uniqueness .

J Future Directions

Below we provide some interesting technical and conceptual future directions.

J.1 Alternate Viewpoint for Task Complexity

Here we briefly note a connection to the Platonic Representation Hypothesis (PRH) Huh et al. [2024], which posits that as models grow in capacity, their learned representations become increasingly similar-hence properties (and failures) transfer more readily across models. Theorem 3 in our work shows that transferable attacks arise only when the underlying learning task is computationally hard (in the EFID sense). If PRH's convergence of representations holds, one should indeed expect greater transferability of adversarial examples; our result then suggests that such transferability is an indicator of computational complexity of the task. In this view, a (plausibly) necessary condition-and a partial explanation-of PRH is that frontier models are solving increasingly difficult problems, which in turn induces representation similarities and transfer. Making these connections precise could be a promising direction for future work.

J.2 Practical Aspects of our Main Theorem

At a fundamental level, families of Boolean circuits are Turing complete, meaning they can simulate any algorithm that a Turing machine can. This makes them expressive enough to capture any computable algorithm and a natural abstraction for studying the inherent properties of learning tasks, independent of specific algorithms. The existence result can guide practical efforts by informing where to search. A key part of our proof is the formulation of a zero-sum game between a "watermarking agent" and a "defense agent," where the game's value indicates which of the three properties exists. Moreover, an optimal strategy in this game corresponds to an actual implementation of a watermark, defense, or transferable attack.

Though finding a Nash equilibrium in such a game seems computationally challenging-since the action spaces involve all circuits of a given size-recent iterative algorithms for large-scale games Lanctot et al. [2017], McAleer et al. [2021], Adam et al. [2021] offer promising approaches. These methods work by evaluating only parts of the game at each step, discovering good strategies over time. Recall that our model captures examples like Pal and Vidal [2020].

In practical settings, circuits can be replaced with standard ML models (e.g., deep neural networks). This opens the door to algorithms that (1) determine whether a given task admits a defense, watermark, or transferable attack, and (2) produce an implementation accordingly. In summary, our theory shows that for a given task, one only needs to set up the appropriate loss and apply these iterative algorithms to extract the desired property. We believe this represents an exciting future direction at the intersection of large-scale algorithmic game theory and AI security.

J.3 Beyond Classification

Inspired by Theorem 2, we conjecture a possibility of generalizing our results to generative learning tasks. Instead of a ground truth function, one could consider a ground truth quality oracle Q , which measures the quality of every input and output pair. This model introduces new phenomena not present in the case of classification. For example, the task of generation , i.e., producing a high-quality output y on input x , is decoupled from the task of verification , i.e., evaluating the quality of y as output for x . By decoupled, we mean that there is no clear formal reduction from one task to the other. Conversely, for classification, where the space of possible outputs is small, the two tasks are equivalent. Without going into details, this decoupling is the reason why the proof of Theorem 1 does not automatically transfer to the generative case.

This decoupling introduces new complexities, but it also suggests that considering new definitions may be beneficial. For example, because generation and verification are equivalent for classification tasks, we allowed neither A nor B access to h , as it would trivialize the definitions. However, a modification of the Definition 8 (Watermark), where access to Q is given to B could be investigated in the generative case. Interestingly, such a setting was considered in [Zhang et al., 2023], where access to Q was crucial for mounting a provable attack on 'all' strong watermarks. As we alluded to earlier, Theorem 2 can be seen as an example of a task, where generation is easy but verification is hard the opposite to what Zhang et al. [2023] posits. Furthermore, recent work Gluch and Goldwasser [2025] proved that, in the context of adversarial robustness and safety, there exist generative learning tasks for which verification (or detection , i.e., identifying adversarial examples) is strictly harder than generation (or mitigation , i.e., allocating additional inference time to correct outputs of the base model).

We hope that careful formalizations of the interaction and capabilities of all parties might give insights into not only the schemes considered in this work, but also problems like weak-to-strong generalization [Burns et al., 2024] or scalable oversight [Brown-Cohen et al., 2023].

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Claims are based on the theoretical derivations in the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Our paper focuses on classification tasks, and in Appendix J we discuss the challenges of extending these ideas to generative learning tasks.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: See preliminaries, technical overview, and the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: No experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: No experiments.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: No experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: No experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: No experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Guidelines were read and paper does conform.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Discussion of broader impact is in introduction, related literature and in the reflections at the end of the paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Citations are provided throughout the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets are released.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: No research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: LLMs were used to edit the text.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper shows that for all learning tasks, at least one of the following three algorithms exists, Watermark, Transferable Attacks, and Adversarial Defenses. They give rigorous definitions of these algorithms by formulating them as two-player interaction protocols, and prove their main result based on circuit complexity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.\tThis paper provides comprehensive analysis of Watermarks, Adversarial Defenses and Transferable Attacks under a cryptographic perspective.\n\n2.\tRelationship between Transferable Attacks and complex learning tasks based on cryptography has been well-studied.\n\n\nWeaknesses:\n\n1.\tThe definitions are very obscure, it’s hard for readers to understand. There are no explanations of why these definitions are reasonable, and no discussions on the relationship between these definitions and the well-known concepts of Watermarking, Transferable Attacks and Adversarial Defenses. For example, in the definition of Watermark, what is $B_n$? Is it a decoder that try to obtain the injected watermarked messages? In the definition of Adversarial Defense, why soundness means adversarial inputs are detected? Does this definition align more with the adversarial detection rather than defense? Also in the definition of Transferable Attacks, why it is defined in this way? I don’t think this align well with the well-known transferable attacks, that an attack can be transferable across models or datasets. Overall, these definitions are abstract, it is very hard for readers to connect them with real-world scenarios. I suggest the authors to provide more discussions, explanations of these definitions, and their potential alignment with well-known definitions of these aspects.\n\n2.\tAll these definitions and results are based on the $F_2$ field. This may be not practical for modern deep networks and embeddings.\n\n3.\tAuthors say this paper focus on classification tasks. However, not any experimental results can validate the soundness of their definitions and theorems. I suggest the authors to conduct some simple experiments on MNIST, or at least some hand-wavy evaluation on certain human-crafted datasets (with some multivariate Gaussian distributions I guess, as often used in other theoretical works). It is worthy to note that, NeurIPS is a conference consider more on practicality, even for a theoretical paper, some simple experiments are welcome and necessary.\n\nQUESTIONS:\n1.\tWhat is the relationship between the Undetectability of Watermark in your definition and the pseudorandomness of error-correcting codes[a]?\n\n2.\tCan these results be expanded to continuous data space?\n\n3.\tIn Section 3.3, the probability is $1/2+\\gamma$, why $\\gamma$ be $[0,1]$?\n\n\n\nRef:\n\n[a] Christ and Gunn. Pseudorandom Error-Correcting Codes. 2024.\n\nLIMITATIONS:\nThey talked about some limitations on extending these ideas to generative tasks. But more perspective should be considered. (See weaknesses and questions, like $F_2$ field, potential misalignment of definitions.)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends prior work exploring provable properties of learning tasks and tradeoffs between backdoor attacks and adversarial robustness. It proves that, for any learning task, a watermark, adversarial defense, or transferable task must exist. This result is rigorously derived through a series of proofs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- Strong mathematical formalism of complex ideas. Proofs are communicated clearly and concisely. \n- Establishes novel results on transferable attacks and their connection to the computational complexity of the learning task. Side note on this - the statement on lines 257-258 (\"One can interpret this as showing that Transferable Attacks exist\nonly for complex learning tasks, in the sense of computational complexity theory\") made me think of the Platonic Representation Hypothesis (PRH) paper (https://arxiv.org/pdf/2405.07987), which claims that increasingly complex models will become more similar (e.g. more transferable properties between the models). A key limitation of the PRH paper is that it does not back up this claim, but rather derives it from intuition. I wonder if your result here about transferability as a result of task complexity could provide some theoretical basis for this claim. Interesting future work, perhaps. \n\nWeaknesses:\n- Paper claims that result holds for any learning task, but proofs and definitions appear specific to classifiers. Claims should be adjusted appropriately. \n- Paper lacks discussion to connect findings to any real-world impacts. The theoretical result is interesting, but I finished the paper wondering why the result matters.\n\nQUESTIONS:\n- It seems the only type of watermark considered in this paper is a backdoor-based watermark. If this is the case, why don't the authors generalize their result to showing that \"for every learning task, at least one of the three must exist: a backdoor, an adversarial defense, or a transferable attack\"? There are many proposed types of watermarks now, particularly for LLMs (lines 993-1026 of supplementary), so making a broad claim about \"watermarks\" when in fact what is meant is a backdoor was confusing. \n\n- In Figure 1, who ``owns'' the model? It seems like model ownership transfers back and forth between Alice and Bob -- in \"watermark\" setting, Alice owns the model and proves ownership by embedding a strong watermark; in the \"defense\" setting, Bob owns the model and is defending it; and in the \"transferable\" setting, Bob again owns the model and is defending it. A simple but helpful change to the model would be updating the protocol so one player is always the attacker and one is always the defender. \n\n- Can these results generalize beyond classifiers?\n\nLIMITATIONS:\nThe paper provides an interesting theoretical result but fails to connect this result to important real-world use cases. The paper would be greatly improved by the inclusion of a discussion setting that contextualizes the theoretical finding and helps the reader understand its significance.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn black-box settings, verifying a model is done through watermarking, which requires embedding a backdoor during training. This inherently conflicts with adversarial robustness because it sacrifices the model vulnerability for model verifiability. Thus, there’s a fundamental trade-off between verifiability and robustness. While previous works explored this trade-off (either empirically or theoretically) only between the two components (watermark vs. adversarial defense), this paper extends this by adding a third component, ‘transferable attacks’. It formally proves that for any discriminative learning task, at least one of the three phenomena: watermark, adversarial defense, or transferable attack must exist, revealing a deeper impossibility. This is proven using three mathematical notions: game theory, circuit complexity, and probabilistic proof. For demonstration, the authors construct a learning task with certain assumptions (encrypted data from the attacker, limited resource bound for the defender), showing that transferable attacks actually occur in practice.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**\n\n-\tThe authors propose a generalizable theoretical framework that applies to any discriminative learning task, given a resource bound.\n\n-\tThe authors provide a helpful illustration to better understand each scenario (watermark, adversarial defense, and transferable attack).\n\n-\tThe authors utilize a diverse set of ideas taken from game theory, circuit complexity, and cryptography to support their claims.\n\n### **Weaknesses**\n\n-\tThe term ‘transferable attack’ used in this paper is defined differently than in the usual sense, which is very confusing. As far as I know, the term “transferability” in model security refers to an adversarial input crafted using one model (surrogate) being effective to another model (target). On the other hand, “transferable attack” in this paper refers to a set of queries that can fool all possible defenders within a given resource bound. Even though the authors note the difference in the footnote of section 1.1, I carefully suggest using a different naming to avoid confusion, considering that the definition of “transferable attack” is crucial for understanding the paper.\n\n-\tThe clarity of the writing and the overall structure is difficult to follow. For example, it’s hard to grasp the main proposal or argument of the paper even after reading the abstract and the introduction. What is the main argument made in the paper? Why is it important to analyze these components (watermarks, adversarial defense, transferable attack) using a theoretical framework? In my understanding, the paper is trying to show the inherent limitation of a general learning task where it’s impossible for a model to be robust and verifiable at the same time, and if one ability is not dominant then there must exist a universally transferable attack.  However, this is not clear even after several readings of the manuscript. Also, since the paper assumes deep familiarity with computational learning theory, complexity theory, and cryptographic primitives, providing a high level introduction to these notions in advance would be better for a broader ML audience.\n\n-\tThe technical contribution of the paper is limited. Although a thorough analysis is always necessary and meaningful, the analysis made in this paper does not seem to have a new finding or enhance the explainability of the problem in any aspect. What is the main takeaway of this paper? What can the readers learn from this?\n-\tThe paper is entirely theoretical. While that is acceptable if the contribution is foundational (as it arguably is), the lack of even toy experimental demonstrations or case studies makes the concepts less tangible. No attempt is made to show how this framework might be instantiated in modern neural network architectures or datasets.\n\nQUESTIONS:\nPlease address the technical contribution and experimental result issues.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a unified theoretical framework to study the trade-offs between backdoor-based watermarking, adversarial robustness, and a novel third axis: transferable attacks. By modeling the interaction between a verifier and a prover as a game, the authors prove that for any learning task, at least one of these mechanisms must exist. They formalize each mechanism, demonstrate examples of transferable attacks using fully homomorphic encryption, and link the existence of transferable attacks to the existence of cryptographic primitives (EFID pairs). Overall, this paper introduces transferable attacks into the trade-off between watermarks and defences, providing a new perspective compared to existing works.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n1. The paper addresses a significant research problem by exploring the relationship between backdoor-based watermarking and adversarial robustness. It innovatively introduces the concept of transferable attacks as a novel perspective to analyze this relationship.\n\n2. The Lines-on-Circle combined with Fully Homomorphic Encryption example provides the first rigorous demonstration that transferable attacks can succeed using substantially less computational power compared to what is required for defence.\n\n- Weaknesses\n1. The work is exclusively theoretical. Without empirical validation using real-world models or datasets, the practical implications and manifestations of the proposed trilemma in real deep learning systems remain unclear.\n\n2. The analysis is conducted in the idealized Boolean-circuit world. It remains unclear how (or whether) the same argument can scale to high-dimensional, structured data such as images, text, or graphs.\n\n3. The exemplar task relies on FHE, which is far removed from common computer-vision or NLP pipelines. It remains open whether similar transferable attacks can be instantiated without heavyweight cryptography.\n\n4. The notation in lines 206 and 217 seems incomplete or unclear. It should likely be corrected to \\mathbf{x} \\in ({0,1}^{n})^q rather than \\mathbf{x} \\in {0,1}^{nq}.\n\nQUESTIONS:\nQuestion\n1. The input space is defined as {0,1}^n, but most current classification tasks operate on continuous spaces (e.g., \\mathbb{R}^n). Can this framework be extended to more realistic or complex input spaces? \n\n2. Why are Definitions 3 and 4 considered complementary? Intuitively, it seems that adversarial defence and transferable attacks are complementary. Or perhaps the complementarity is between defence and the union of transferable attacks and watermarks?\n\n3. In Definition 3, it is stated that there exists a B_n such that err(x,y)< 2\\epsilon. What if there are two such B_n​, or three, or up to n−1different B_n satisfying this condition? Do these cases need to be analyzed within your framework?\n\n4. Intuitively, if a model is adversarially robust, then adversarial examples (transferable attacks in this paper) should not exist. This seems to imply that for any model, either adversarial robustness holds or transferable attacks exist. In that case, why is watermarking needed? Could you clarify how watermarking fits into this framework and how it relates to the intuitive dichotomy between robustness and transferable attacks?\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a unified framework for analyzing the trade-offs between backdoor-based watermarks, adversarial defenses, and transferable attacks in machine learning. The authors formalize these concepts as interactive protocols between two players, modeled as families of circuits to account for their computational resources. The key contribution is a theorem stating that for any learning task, at least one of these three properties must exist. This result is proven by formulating the interaction as a zero-sum game between the players and analyzing its Nash equilibrium. \n\nThe paper also establishes a connection between transferable attacks and cryptographic primitives, specifically showing that the existence of transferable attacks implies the existence of EFID pairs of distributions. This connection suggests that tasks admitting transferable attacks must be computationally complex. Additionally, the paper provides concrete examples of learning tasks that support watermarks and adversarial defenses, particularly for hypothesis classes with bounded VC dimension. These examples demonstrate how different security properties can exist or coexist depending on the characteristics of the learning task and the available computational resources.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths** \n\nS1. The paper demonstrates technical depth by employing diverse theoretical tools. It leverages game theory (zero-sum games and Nash equilibria) to prove its main theorem, uses cryptographic concepts (EFID pairs) to analyze transferable attacks, applies computational complexity theory to model resource-bounded adversaries, and connects to learning theory (VC-dimension) in analyzing watermarks and defenses. \n\nS2. The main theorem, stating that at least one of the three properties (watermark, adversarial defense, or transferable attack) must exist for any learning task, appears to be a significant contribution. It provides fundamental insights into the nature of machine learning security and the inherent trade-offs involved.  \n\n**Weaknesses** \n\nW1. The definitions in Section 4 don't clearly connect to established notions of watermarking and adversarial defense, nor do they fully justify the various properties introduced. This makes it harder for readers to relate the new framework to existing knowledge in the field. \n\nW2. The paper could benefit from more discussion on the practical implications of its findings for real-world machine learning systems and security practices.  \n\nW3. While not a major weakness, the paper focuses solely on classification tasks. The authors acknowledge this limitation and discuss potential generalizations to generative modeling tasks in Appendix E. \n\nW4. The paper could benefit from a discussion of whether there are any real-world scenarios that aren't captured by the theory. Additionally, the authors could justify more thoroughly their choice to model Alice and Bob as circuits and discuss potential limitations of this approach. \n\n**Minor** \n\n- Incorrect cite style \n\n- Fig 1: the caption refers to “adversaries” and “defenders”, but it’s not clear whether this refers to Alice or Bob. (I believe Alice corresponds to the adversary in all cases).\n\nQUESTIONS:\nThe paper presents a strong theoretical framework, but the practical implications of the results are not thoroughly discussed. Could the authors provide more guidance on how practitioners in machine learning security should interpret and apply these results? Specifically: a) How might the trade-offs identified in the paper influence the design of secure machine learning systems in practice? b) Can the authors provide examples or scenarios where their framework could inform decision-making in real-world ML security situations?\n\nLIMITATIONS:\nThe authors have addressed limitations by acknowledging their focus on classification tasks. However, two additional limitations could be discussed: \n\n- The implications of modeling Alice and Bob as circuits and whether this accurately represents real-world adversaries and defenders. Are there scenarios where this model might break down? \n\n- The paper introduces new definitions for watermarks, adversarial defenses, and transferable attacks. It would be beneficial to discuss potential limitations of these definitions, especially in relation to “working definitions” used in the field.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}