Blameless Users in a Clean Room: Defining Copyright Protection for Generative Models
Authors: Aloni Cohen
Venue: NeurIPS 2025 spotlight ()
Keywords: copyright
generative models
law
definitions
AI and law
differential privacy
Primary area: theory
TL;DR: This paper revisits the question of provable copyright protection for generative models: identifying limitations of prior work, proposing new definitions, and describing conditions under which differential privacy suffices.

Abstract: Are there any conditions under which a generative model’s outputs are guaranteed not to infringe the copyrights of its training data? This is the question of "provable copyright protection" first posed by Vyas, Kakade, and Barak [ICML 2023]. They define _near access-freeness (NAF)_ and propose it as sufficient for protection. This paper revisits the question and establishes new foundations for provable copyright protection---foundations that are firmer both technically and legally. First, we show that NAF alone does not prevent infringement. In fact, NAF models can enable verbatim copying, a blatant failure of copy protection that we dub being _tainted_. Then, we introduce our _blameless copy protection framework_ for defining meaningful guarantees, and instantiate it with _clean-room copy protection_. Clean-room copy protection allows a user to control their risk of copying by behaving in a way that is unlikely to copy in a counterfactual "clean-room setting." Finally, we formalize a common intuition about differential privacy and copyright by proving that DP implies clean-room copy protection when the dataset is _golden_, a copyright deduplication requirement.

Blameless Users in a Clean Room:
Defining Copyright Protection for Generative Models
Aloni Cohen
Department of Computer Science University of Chicago
aloni @uchi edu
Abstract
Are there any conditions under which generative model's outputs are guaranteed not to infringe the copyrights of its training data? This is the question of "provable copyright protection" first posed in [VKB23]. They define near uccess-freeness (NAF) and propose it as sufficient for protection. This paper revisits the question and establishes new foundations for provable copyright protection-foundattinn that are firmer both technically and legally. First, we show that NAF alone does not prevent infringement. fact, NAF models can enable verbatim copying, a blatant failure of copy protection that we dub being tainted. Then, we introduce our blameless copy protection framework for defining meaningful guarantees, and instantiate it with clean-room noom copy protection. Clean-rood copy protection allows a user to control their risk of copying by behaving in a way that unlikely to copy in counterfactual "clean-room room setting." Finally, formalize common intuition about differential privacy and copyright by proving that DP implies clean-room copy protection when the dataset is golden, copyright deduplication requirement.
Introduction
A user of generative model is worried about unwitting copyright infringement. She is worried that the model's 's outputs might resemble copyrighted works the training data through no fault of her own, exposing her to legal liability. The user would like some assurance that this won't ton't happen. We ask: What assurances can the model provider give, and under what conditions?
Ideally, the generative model would never reproduce copyrighted work. That's unrealistic. Typical models can be prompted to generate copyrighted output: print the following LCG24]. It's also unnecessary. In the example, copying is clearly the user's fault. Instead, we should protect blameless users from inadvertent infringement. If the model reproduces copyrighted work, it should be because the user induced it to (or was very unlucky). This guarantee should satisfy careful, honest user. As long as the user's 's use of the model does not itself cause the infringement, consciously subconsciously, then the result is unlikely infringe.
[VKB23,
Our goal is to formalize such a guarantee- in a mathematically and legally rigorous way- and to study its feasibility. We'ne 're after "provable copyright protection" at deployment time, first studied by Vyas, Kakade, and Barak [VKB23]. They propose near access-freeness (NAF) as an answer.
Our contributions This paper establishes new foundations for provable copyright protection- foundations that are firmer both mathematically and legally than prior work.
1. We prove that near access-free (NAF) models enable verbatim copying. NAF is the first (and only other) attempt at a mathematical definition intended to offer "provable copyright protection"
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

for generative models [VKB23]. The proof leverages NAF's lack of protection against multiple prompts or data-dependent prompts.
2. We define tainted models, which enable users to reproduce verbatim training data despite knowing nothing about the underlying dataset, blatant failure of copy protection (Section 5). A meaningful definition of provable copyright protection must exclude tainted models. NAF does not.
3. We introduce a framework for defining copy protection guarantees, called blameless copy protection (Section 6). It protects blameless those who don't tt themselves induce infringement- from unwitting copying.
4. We define clean-room copy protection (lceaa) first instantiation of our frame- work (Section 7), drawing inspiration from clean-room design. A training algorithm is (k, -)clean if, for every user who copies in (counterfactual) "clean-room environment" with probability S (, the probability of copying in the real world is S K. Clean-room copy protection lets users choose their tolerance for risk K and then tune accordingly. Clean-room copy protection also excludes tainted models under mild assumptions.
5. We prove that differential privacy (DP) implies clean-room copy protection, formalizing common intuition about DP and copyright (Section 8). This holds when the dataset is golden, a copyright deduplication requirement. Thus, DP provides way to bring copyrighted expression "into the clean room" " without tainting the model.
This paper does not offer a legal analysis of where liability would lie under existing law. Instead,our paper develops mathematical framework for preventing unwitting infringement-wiir where holding the user liable would be unjust-and proposes a rule for assigning fault when copying does inevitably occur (Section 9).
Related work There is a lot of recent work on copyright questions for generative AI [CLG 23, LCG24, HLJ 23], and the first generation of cases working their way through the courts (e.g.. New York Times v Microsoft). But most are only tangentially related to goal of stating and proving formal guarantees against copyright infringement.
Vyas, Kakade, and Barak were the first to propose mathematical property- -nea access-freeness (Nme) at "preventing deployment-time time copyright infringement" [VKB23]. Elkin-Koren Hacohen, Livni, and Moran argue that copyright cannot be "reduced to privacy" [EKHLM24]. Referring to both NAF and DP by umbrella term "algorithmic stability". they argue that the sort of provable guarantees that [VKB23] and this paper seek do not capture copyright's 's complexities. Li, Shen, and Kawaguchi propose and empirically evaluate an attack called VA3 against NAF [LSK24]. They provide good evidence that the CP-k algorithm of [VKB23] may not prevent infringement, but some uncertainty remains (see Appendix B.3). We discuss these three papers at length. There is also work on new NAF algorithms [GAZ+ 24, CKOX24).
Scheffler, Tromer, and Varia [STV22] give and a procedure for adjudicating disputes. In contrast, we treat substantial similarity as a black box. That differential privacy might protect against infringement has been suggested in [BLM20. HLJ+ VKB23, EKHLM24, CKOX24]. Livni, Moran, Nissim, and Pabbaraju [LMNP24] study the problem of attributing credit to inputs of an algorithm that influence its output, motivated by copyright. They define a condition under which a given input need not be credited (with credit required otherwise). Under their definition, a differentially private algorithm never has to credit its inputs manifestation of the common intuition that differential privacy prevents copying which we discuss in Section 8.
cemplexity-theoreiie account of substantial similarity.
1.1 Notation
W is the domain of works protected by copyright law. For example, W might be the space of possible images, texts, or songs. We assume W is discrete. We usually use w E W to denote an individual work, along with c, y. E W as described in the following. We denote by C W the set of all in-copyright works those currently under copyright protection- and denote a particular copyrighted work by e E C. A dataset = (wi,..., wx) E W is a list of works with multiplicities allowed. We sometimes abuse notation and treat /) as set. A conditional generative model (model) p is mapping from a prompt x to probability distribution p(-lær) over W which samples y E W with probability z)). A training algorithm Train maps dataset D to a model p. A user is algorithm which

---

uses Bluck-box access to model along with auxiliary input aux E 10, U fLL} and outputs work W (see Section 4).
Symbol
Definition
W; w,y.2
Domain of works (c.g., images, texi); individual works Dataset D (w',,...; Wn)
D
Train
Training algorithm mapping datasets D to models p Conditional generative model; the probability generating y on prompt æ Auxiliary information given to the user (usually aux = ideas(w)) User taking input aux and access to and outputting a work <- u (aux)
p: p(y ())
aux
T3 r(B; aux)
Real-world distribution; the probability of event E E in the probability experiment p Train(D), (D), u ua(x)
C: c
Set of in-copyright works; an in-copyright work Set of works substantially similar to w Non-copyrightable ideas associated with w' derivative work ut represented as edges graph C co-popyrghht works from which some work (Cp) / no work (C p) in D stems Sub-dataset of works in D that don't Io stem from c Clean-rov room distribution, defined like but replacing D with scrub(D,,)
SubSim(w)) (w) ideas(w)
"w' stems from w"
Cd: C_D scrub(D,e)
T-c; TEe aux)
Table 1: Common symbols and their definitions. Bolded terms are exogenously (legally) defined.
2 The legal-technical interface
To use legal concepts within a mathematical formalism, we assume some wrogenously-define functions reflecting and operationalizing those concepts. They the interface between math and law, enabling mathematical study of legal concepts without formalizing the concepts themselves. Three legal concepts are needed for this paper: substantial similarity, ideas, and access. This section defines functions SubSim and ideas for the first two. Section 7.1 operationalizes access.
To prove copying, plaintiff (copyright holder) must prove two things: substantial similarity and access. The former requires the defendant work's 's to substantially similar to the original, with vague tests varying by jurisdiction. Access requires the defendant to have had reasonable opportunity to view the original (e.g., it's online). Independent creation of substantially similar works is allowed. The function SubSim defines what it means for one work to be substantially similar to another.
Definition 2.1 (SubSim). For a work w E W, the set (u)) S W contains all works w/ that are substantially similar to We assume substantially similar to itself: SubSim(W); For set of works W C W, we define SubSim () = Uwew SubSim(w)) (w).
Copyright does not protect ideas, only original expression. For example. the text and images in a cookbook may be copyrighted. but not the method of cooking a dish. To be afforded protection, there must be a minimum of creativity beyond the ideas. The function ideas defines the non-copyrightable ideas of a work, opposed their expression. We represent these ideas a string which one view as encoding a set or any other data type.
Definition 2.2 (ideas). For a work w E W, the string ideas(w) E ULL1)" is (some represen- tation of) the ideas contained in w. For a set of works W C W, we define the set of all their ideas W).
ideas(W) {ideas(w):
We never actually compute SubSim nor ideas. This is feature: it means we can be agnostic specific instantiation. One might view them as capturing the "true" legal concept. Or one may take more practical view. For example, interpreting SubSim(w) judge or jury. However the reader wishes to interpret these functions is fine (up to any assumptions stated where used). In particular, our results hold even for SubSim(wi (w) {w) and ideas(w) I. In contrast, a model provider (or data provider) would need to actually work with the copyright dependency graph in Section We discuss this limitation in Section 9.
the
reflecting the opinion of typical
3 Near access-free models do not provide provable copyright protection
Near access-freeness (NAF) a mathematical definition for "provable copyright protection" " [VKB23]. This section describes near access-freeness and its limitations. We prove that models can enable
3

---

verbatim copying while still satisfying NAF. While NAF provides protection against single prompt that independent of the training data, makes no guarantees against many prompts (composi- tion) [L.SK24], nor single prompt derived from non-copyrightable ideas (not expression). We emphasize that our focus is definitions for provable copyright protection, where NAF falls short. Practically, NAF may still provide useful protection in many natural settings. For space, we defer algorithms, proofs, and additional discussion to Appendix B.
3.1 NAF Background
NAF informally Near access-freeness (NAF) is the first (and only other) attempt a mathematical definition intended to offer "provable copyright protection" for generative models [VKB23]. Moti- vated by the legal requirement of access, NAF is meant show "that the defendant's work is close to a work which was produced without access to the plaintiff's ss work" [VKB23]. NAF is closely related to DP. But surprisingly, any training algorithm Train be used in black-box way to construct NAF model without the extra noise that is the hallmark of DP (Theorem 3.3).
The definition of NAF requires generative model p trained using a copyrighted work to be close to a "safe" model safe trained without any access to c. How close is governed by a parameter k: 2 smaller k means closer models. A corollary of the NAF requirement is that for any prompt z, the probability p generates something substantially similar to is at most 2 - greater than safe doing so. We heuristically expect the latter probability to be miniscule, if safe and are independent of c. If so, this bounds the probability that p's output infringes on prompt z.
Known limitations of NAF Prior works discuss and critique the NAF framework in three ways: questioning NAF's technical effectiveness [LSK224), its legal applicability (LCGG24 EKHLM244, and its real-world practicality [HI,I+ EKHLM24, LCG24, CKOX24).
Technically, Li. Shen, and Kawaguchi propose and empirically evaluate an attack called VA3 against NAF [LSK24]. See Appendix B.3 for a full discussion. They show that a black-box attacker, can reliably induce a model to produce outputs that infringe on a target work c*. They also give a white-box prompt writing algorithm for diffusion models that greatly improves performance. VA3 provides good evidence that the CP-k algorithm of [VKB23] may not prevent infringement. Some uncertainty remains because the algorithm implemented in VA3 deviates from the original. The paper leaves open whether k-NAF (with fixed k) prevents copyright infringement, formalizes a flawed attack model, and avoids the underlying definitional questions almost entirely.
Legally, Elkin-Koren et al. [EKHLM24] and Lee, Cooper, and Grimmelman [LCGG24 argue that NAF wrong on the law. See Appendix B.4 for a detailed discussion of these arguments.
Practically, the main concern is that deduplicating data the way needed to make NAF effective is infeasible [EKHLM24, LCG24, HLJ+23) It is unrealistic to produce enough clean training data to pretrain foundation models, advances in deduplication notwithstanding. Moreover, NAF learning algorithms are computationally expensive and may lag in performance [CKOX24]. These concerns are justified and also apply to DP. Still, there may be settings where golden data is practical (Sec. 9), say nothing of the value of theory.
3.2
NAF definition and the Copy Protection (CP) algorithm
Near access-freente (NAF) is defined with respect to a function safe. The safe function maps a copyrighted data point e a generative model safe, E P trained without access to c. An example sharded-safe is in Appendix B (Alg. 1). Thm. 3.3 presents the main feasibility result: any training algorithm Train can be used as a black-box to construct an NAF model CP. short for copy protection.
Definition 3.1 (Max KL divergence). For distributions p. q. Amax (pllq) = maxxyeSupp()) log 2(v)
q(v)
Definition 3.2 (kgF-NAF [VKB23]). Fix a set and function safe : + P. generative model
p is ku- -ner access-free (k,-NAF) on prompt E X with respect to C and safe for every E C, maxe (p(-lx: safe, xr) S lig. A model p is k-NAF with respect to and safe if for all x X,
is -AF for some hg < k.

---

NAF bounds the probability that p's output copies relative to the probability under safe. If is kg -N-N safe on prompt z with respect C and safe, then for any E C:
p(SubSim(c) | I 2h (SbbS((() a:).
(1)
Hence, bounding k prevents copying c, assuming safe; (SbSSi((b)))))) is negligibly small (in |c]- Theorem 3.3 (Copy Protection algorithm [VKB23]). Let p be the model returned by CP (Algorithm in Appendix B). and qı and q2 be the models returned by sharded-safee Let k S log dTv (qi(-)z) (2(+1=2)) Then p is -x-NAF for a with respect to and sharded-safe.
3.3 Failures of NAF models
We now describe two ways NAF fails to prevent copying, leveraging its lack of protection against multiple prompts (i.e., composition) and data-dependent prompts. In each case, a user reproduces training data e D) verbatim. In Section 3,3.1, the model generates e when prompted with ideas(c) a prompt that depends on e but not on any copyrightable expression thereof. In Section 3.3.2, the user issues fixed sequence of prompts, recovering & bits of the dataset with each query.
3.3.1 CP can regurgitate training data
We show that CP-the main NAF algorithm of [VKB23]- fail to protect against copying. Using prompt containing copyrightable expression, user can cause the NAF model returned by CP to regurgitate copyrighted training data. This based on an observation of Thomas Steinke [Ste23].
Observe that CP is black-box transformation from an underlying training algorithm Train. Thus, CP really family of algorithms one per Train. The following theorem states that there exists Train" such that the resulting NAF algorithm CP" fails in the following way. On prompt æ = ideas(c) c) reproduces training datum verbatim. We defer the proof of (a slight generalization) of this theorem to Appendix B.2. It uses the training algorithm Train* = Trainw (Alg. 3) described in Example 5.2.
Theorem 3.4. For model training algorithm Train", let CP" be the -g-NAF algorithm from Theo- rem 3.3, and let p" -CP" For all ideas, there exists training algorithm Train" such that for all datasets D and for all works e. E D whose ideas are distinct D (i.e., Yw E D, w + e: ideas(w) + ideas(c)): p² (c ideas(c)) 1.
3.3.2 NAF does not prevent full reconstruction
Our next theorem shows that k-NAF may allow training data to be reconstructed, even if k is arbitrarily small and independent of This is because the k-NAF guarantee does not compose across a user's many queries, and each query may leak up to k bits of training data. In more detail, we give family of models that are k-NAF with respect to pure noise, yet which enable the dataset verbatim. For any 2 1. let coing(-12) be uniform over {0, 1) for all prompts æ. As generative model, (-I-)(-l-) clearly "safe" instantiation of safe, for any copyrighted work c. The proof is deferred to Appendix B.2.
to reconstruct
Theorem 3.5. Fix S W S {0, 13" For D E W let be total the length of D in bits. There exists a (deterministic) training algorithm Train (D, k) + PE.D satisfying the following.
For all D and > 0: PK.D k--AF with respect to and coine for E max(1, [k]]. There exists a user such that for all D and l > 0: u makes pLly 1/k) black-box queries to PE,D and outputs D with probability > 0.99.
4 The interaction between user and model
People interact with generative models in complex ways: refining prompts and using results to create a final product whose author is not solely human nor machine. Meaningful copyright protection must cover such cases. To that end, make the interaction between user and the model explicit.
Let be a (randomized) algorithm called the user, p be a model, and aux E (0, O (1) be an auxiliary input. In cryptography, auxiliary inputs allow one define security guarantees even when an algorithm has instance-specin specific side information. In our setting, the algorithm the user, and the side information is the non-copyrightable ideas of an in-copyright work. We denote by uP (aux) the
5

---

algorithm u run with input aux and black- box access to p. The result is a work & E W distributed as 2 f ua(au)). Together with a training algorithm Train and dataset D. this process induces probability measure r over W. Definition 4.1 (User's output distribution). For a user u, training algorithm Train, dataset D, and auxiliary information aux, we define the user's ss output distribution over W as:
r(u; aux)
Pr
w).
pe- pe-Train 26- (aux)
The probability is taken over the randomness of the algorithms Train, u, and p. Note that T depends on Train, u, and D. The notation elides these dependencies to reduce clutler.
Legally, the user's output infringes on the copyright a work с E C only if it is substantially similar to c. Preventing substantial similarity prevents infringement. This motivates our first desideratum.
Desideratum 1. We want (SubSim((); aux) to be as small as possible for as many users as possible.
By considering the user's output distribution, we require protection against multiple prompts (com- position) and data-dependent prompts (when aux contains the ideas of work in the training data, as below). This fixes the shortcomings of NAF that enabled the attacks described in Section 3.3.
5 Tainted models enable copying
This section defines what it means for generative model to be tainted. Tainted models let users copy training data of which they have no knowledge. We say training algorithm Train is tainted there is fixed algorithm that copies work from the training dataset using only trained model and unprotected ideas. The order of quantifiers is important: the user is fixed while the dataset D is arbitrary. Such a user cannot possibly be at fault-it cannot know work in all possible datasets D.
Definition 5.1 (Tainted training). We say Train is tainted with respect to ideas if there exists a user such that for all datasets D and all works w D: T r(SubSim( D); ideas(w)) > 0.99.
If the goal is to prevent copying, being tainted is as bad as it gets. Thus, we get our next desideratum. We will see that NAF fails this test but Definition 7.6 passes.
Desideratum 2. meaningful definition of provable copyright protection should bar tainted models.
As an example, we show that any training algorithm can be made tainted (Example 5.2). This is used to prove Theorem 3.5. Note E. presents a negative example: a training algorithm that always returns single fixed model is not tainted, under the mild assumption that it is possible for two works to express the same ideas so differently that no work similar to both.
Example 5.2. Algorithm 3 in the Appendix describes a tainted algorithm Trainw constructed from any training algorithm Traino in a black-box way. in words, Trainw trains a model qu Traing (D), and also builds key-value store I mapping ideas id to the set Did {w e D : (d) = id). On prompt x, the model 9kv returns a random element of [[x] = D2 if it is non-empty. Otherwise, it returns generation sampled from qu(-Iz). It is easy to see that Train is tainted. Consider the user ua(uux) that returns a sample from p(- lax).. Fix D and w E D, and let p + Train (D). By construction, the user always outputs element of Dideastay- Therefore, r (SubSim(D); id)as(w)) 2 T (Diieass(w); ideas(w)) 1.
5.1 Tainted NAF models
Desideratum 2 lays out necessary condition for provable copyright protection: exclude tainted training algorithms. NAF fails this test. This is captured by the following corollaries of Theorem B.5 (slightly generalizing Theorem 3.4) and Theorem 3.5.
Corollary 5.3 (of Thm. B.5). For any ideas, the NAF algorithm CP" given by Thm. 3.4 is tainted.
Proof. Consider the user uP (aux) that returns a sample from p(- p(-laux). Fix D and E D, and let p" + CP' (D)) By Theorem B.5, T (SubSim(D);; ideas(w)) 2 T (Dideessww) ideas(w)) = O
Corollary 5.4 (of Thm 3.5). For any k, ideas, there exists a tainted algorithm Train such that for all datasets D, the model < TDain(D) is k-NAF with respect coing.

---

Blameless copy protection: a definitional framework
Our goal is to define provable copyright protection. The previous section gives negative answer: provable copyright protection should bar tainted models. This section and those that follow give a positive answer: provable copyright protection should protect blameless users.
We can't hope to guarantee that generative model never reproduces copyrighted work, even works was not trained on. Malicious users can always induce copying. (Formally, if the set of in-copyright works C non-empty, there exists u that always infringes: 7 r(SubSim(C); L) = 1.) Instead, we wish to protect blameless whe don't themselves induce infringement- from unwitting copying.
This suggests framework for defining meaningful guarantees, which we call blameless copy protection. First, define a class of blameless users B. Second, guarantee that for blameless users, the probability of copying r(SubSim(C); aux) at most some small K.
Definition 6.1 (Blameless copy protection). Fix K. >> 0 and a class of blameless users. We say Train is (x, B) coop protective for all blameless E B and all aux, D E W C W:
r(SubSim(C); aux) < K.
The missing piece is B: What makes user blameless? Different answers will yield different versions of blameless copy protection. So Definition 6.1 is less definition than a framework for definitions. Instantiating it may require additional assumptions. As we cannot perfectly capture legal blamelessness, we should err the side of protecting more users rather than less. In Section 7, we offer a first instantiation of blameless copy protection, inspired by clean-room design. But there may be very different ways to formalize blamelessness, which we leave for future work. The cryptographic notion of extraction offers an intriguing approach: a user is blameworthy a copyright-infringe work can be efficiently extracted from the user itself.
7 Defining clean-room copy protection
This section formalizes what it means have access to copyrighted work (Section 7.1) and instantiates the blameless copy protection framework (Section 7.2) draw-dgawing inspiration from clean- room design. In copyright law, a clean room is "a process of producing product under conditions guaranteeing independent design and foreclosing the possibility of copying" [EIk90]. The clean room keeps contamination out, like keeping dust out of a semiconductor lab. The idea comes from cases involving reverse engineering. Roughly, team is given description of design specs of the product (ideas) but not the product itself (expression), and tasked with producing compatible product. There is no access, as long as the team itself was not spoiled by prior familiarity with the product or its design. As such, the outcome constructively non-infringing. Even substantial similarities will be the product of independent creation.
Clean-room design is the inspiration for counterfactual clean-room distribution where a user interacts with a model trained without access to certain in-copyright works (Definition 7.4). We quantify the blamelessness of user as the probability /3 that the user-in the clean-room distribution- would have produced something that is substantially similar to the excluded works (Definition 7.5). Clean-room room copy protection provides a corresponding bound on the probability of copying any work in the real distribution (Define 7.6), where x depends on B.
7.1 Scrubbing a dataset removes access
A clean room is setting where user does not have access to particular copyrighted work. To pin this down, we first operationalize the legal concept of access.
The copyright dependency graph We need a way to say whether a work w/ is a derivative of another work w copyright-relevant way. Such dependencies directed, with later works stemming from earlier works. The copyright dependency graph encodes this relation. It is assumed to capture the legal concept of access in the following sense: For purposes of copyright law, dataset D only gives access to a work w if there is some w' E D that stems from w.
Definition 7.1 (Copyright dependency graph). The copyright dependency graph is a directed gruph G (W, F)) whose edges reflect dependencies relevant for copyright. If (w, w') ) E F,, we say that w' stems from w. We assume that every stems from itself: (w, () E E for all w.
7

---

It is useful to refer to the set of all copyrighted works for which access is or isn't implied by access to dataset D. We denote these sets by Cp and C- respectively. For the sake of copyright analysis. access to D does not constitute access any copyrighted c. C-
Definition 7.2 (CD and C-D). We define CD {c E Ew E ()( E E} as the set of copyrighted works from which any work in D stems. We denote its complement C-D = C CD.
The scrub function We define the function scrub to operationalize the legal concept of access. It removes from a dataset any work that stems from a target copyrighted work c. That is, any w for which - w is edge in the copyright dependency graph. The result is For the sake of copyright analysis, access to scrub D, c) does not constitute to access to C.
dataset scrub D, C D.
Definition 7.3 (scrub). Fix copyright dependency graph G = (W, E), dataset D and work The dataset scrub ( DD, c) = (w E D (c, (c,w) & F.) is the sub-dataset of D of works not stemming from c.
Observe that Cp = {c E C scrub(D,c) c) * D} and C-D = fco C: scrub(D,c) =D}
7.2 Clean training algorithms: copy protection for blameless users a clean room
This section gives clean-room inspired formulation of copy protection. Informally, we say that training algorithm is (k, -cllenn if for all users u: either (i) u would have copied in a clean-rood room setting with probability at least 3; or (ii) copies in the real world with probability at most k. Making this precise, we define the user's clean-rvom output distribution (Definition 7.4), blameless users in the clean room (Definition 7.5). and (k. ---e-en----ooonn copy protection (Definition 7.6).
7.2.1 The user's clean-room distribution
We define a user's clean-room distribution, counterfactual its true distribution т. For a copyrighted work c, we denote by T-e the user's output distribution in clean room where the model doesn't 't depend on c. The only difference from the user's woaldwolld output distribution 7 (Definition 4. that the model is trained on scrub( D, c) instead of D).
Definition 7.4 (User's clean-roow distribution). For user u. training algorithmm Train, dataset D, work C, and auxiliary information aux, we define the user's 'er's clean-room distribution for as:
(w: aux)
Pr
[2 = w].
pe ---Trrnnnscru(DD (.c))) zt-u uaux)
This is closely related to NAF, The model Train(scrub(D,cc) is NAF's safe model safe, for an appropriately defined function safe. The clean-room distribution T-c is the distribution over outputs that results when the user given aux interacts with safe..
7.2.2 Clean-room blamelessness and copy protection
We postulate that blameless users can control their risk of copying when using a model trained in clean room. Given 3 > 0, blameless user can guarantee that they copy work that was "outside the clean room"" with probability at most B. This should hold even if the user is exposed to e in some other than through the model. (For a weaker assumption, see Note E.3.) People are constantly exposed to copyrighted works, yet we somehow manage to avoid copying every day. Using a model trained without access to shouldn't tttl change that.
We call these users 3-blameless in a clean room, or 3-blameless for short. Definition 7.5 formalizes this idea. Recall that C-D is the set of copyrighted works from which no element of D stems (Definition 7.2). Thus, the set of works "outside the clean room" C_ U (c).
Definition 7.5 (Blameless in the clean room (8-blameless)). For 0 < B < 1, user is 8-blameless in the clean room (8-blameless) with respect to D, C, Train for all c C and all aux:"
SubSim (C aux
S 3.
Instantiating our framework (Definition 6.1), we now define clean training algorithms as those that provide copy protection users who are blameless in the clean room.
'qquivaeettly (D, ideas) {u : Ve E C, Vaux, 7-e (SubSim id) $ 3).
8

---

Definition 7.6 (Clean-room copy protection: (k. )leeann) For K. 3 > 0, we say Train is (k, -lcenn for all C C C. D E W all users u who are 3-blameless (w.r.. C, D. Train), and all aux:
r(SubSim(C); aux) S
If this only holds for datasets D in an admissible set D _ W we say Train is (fc, ()-lean for D. Remark 7.7 (How small is 3?). We think of B3 as the probability that "truly blameless" user (whatever that means) nevertheless produces something substantially similar to a copyrighted work. There is limit to how small this can be. Even monkey
typewriter- truly blameless if anyone is has a non-zero 3. We conjecture that B xs 10 or 10 9 is achievable by a conscientious user.
7.2.3 Clean-room copy protection is not tainted
By Desideratum 2, a good definition of provable copy protection should exclude tainted training algorithms. Theorem D.1 shows that clean-room copy protection passes this test, under a reasonable assumption on ideas and SubSim. Roughly, the assumption is that there exist at least n distinct ideas each of which can be expressed in m > n ways satisfying a strong dissimilarity property: that for any distinct and w', the set of works substantially similar to both w and w' is empty.
8 Differential privacy's protection against copying
Many works have suggested that differential privacy [DMNS06] (DP) should protect against copying the training data are deduplicated [BLM20, HLJ 23, VKB23, EKHLM24, CKOX24, LMNP24]. This section turns the suggestion into a theorem. For background on DP, see Appendix C.
At a high level, we show that DP training blameless users from copyright infringement (in the sense of Definition 7.6) if the training dataset is golden, a copyright-dedupliene
condition defined below. If so, the probability of copying is at most = e 3ND. where is a user's 's probability of copying in the clean room and ND [Cp] is the number of copyrighted works to which D grants access. To guarantce the risk is at most k, the user sets B Rs k/e" ND. The linear dependence on Np is not ideal, but may sometimes be good enough or improved (Note E.4.)
This allows user choose their appetite for copyright risk x and then act to set 3 accordingly. Stephen King writing his next bestseller has much lower risk tolerance (x 10-9) than Joe Schmoe writing a wedding toast (A = 10 10-1). 1). Most users are somewhere between. For 8 = 5 and No 200, 000 Stephen King needs 3 Rs 10 while Joe Schmoe only needs 3 R 10-6, If /3 RJ 10-15 is too small (Remark 7.7), Stephen King can instead forgo using the LLM altogether.
A dataset D is golden if at most one item E D stems from any copyrighted work c.² That item could be itself or a derivative work. No protected original expression appears in more than one element of golden dataset. For example, it can contain one copy or parody of the Abbey Road album cover (not both), and many parodies of the co--yrrgt Mona Lisa. Note E.5 explains why golden datasets will typically remain golden as the set of in-copyright works evolves over time.
Definition 8.1 (Golden dataset). Let G (W, E') be a copyright dependency graph, and C C W be the set of in-copyright works. A dataset D is golden (with respect to G. C) for all E C there is at most one w E such that (c, w) E E.
Theorem 8.2. Let Train be (e, ddiffeerretl private > 0, 2 Let ND = [Cp). Then Train is (x, -)-eaan for golden datasets for all 3 2 0 and k 2 (ef ND + 1)3 + NP
Proof. Fix S W, D, and user u that is 3-blameless (Def. 7.5). We must show that for all aux, rSSubSim(C); aux) I k. Because D is golden, [D] scrub D, c) < 1. Hence, datasets scrub D, c) and D are either equal or neighboring for all c E C. Applying DP and post-processing, we have T(E) e (E) + ó for all events E and all c E C.
r (SSubSimCC); aux) < T (SubSim(C-D); aux) + Σ
r(SubSim(c); aux)
(union bound)
cECp
< N + Σ (SubSim((;; aux)
(DP; Prop. 8.3 below)
cECp
$ ND + 1)B + ND8 <
(8- (B-blameless)
"The term is borrowed from [VKB23] who use only informally and with different meaning.

---

In the proof above, we need to bound the probability r(SubSim(C_D): aux) of similarity with some work in C_D D in the real distribution т. Blamelessness only gives us a bound in the clean-room distribution T-c- Proposition 8.3 shows that the latter implies the former. Proposition 8.3. Let be B-blameless w.r.t D. C. Train. For all aux: T (C_n: aux) S B.
Proof. By definition of C.n. scrub D, c) = D and hence T- = T. Also, C- n = C- n U {c}. Thus, r(C (C=-p;aux) S T-o (SubSim (C-n U {c]) aux) S B, with the last inequality by blamelessness. m
9 Discussion: provable copyright protection in practice?
What if copying occurs? People copy without generative models, and will continue copying with them. One way to evaluate the usefulness of copyright-mitiguin measure is to consider what happens when copying does occur. Who should pay?
It is hard to assign culpability on the basis of NAF alone, as tainted NAF models can induce user acting appropriately to infringe. Clean-room copy protection gives a clearer theoretical account of who is culpable if training is differentially private. The provider is culpable if the data wasn't 't golden; the user is culpable if not blameless; possibly both are culpable, or neither. However, this test is impossible to apply as blamelessness cannot in general be checked.
Even so, a simple indemnification rule is possible: The model provider pays for infringement if the copied work appears too often in the training data. This is question of fact that legal process and forensic experts are well-suited to resolve. Experts for each party can examine the training data and testify before the finder of fact. Our indemnification rule gives users what they want. Users are indemnified when there is any possibility that the model is blame, and they can control their risk tolerance by tuning 3. The rule also gives model providers what they want. Besides offering a valuable protection for users, the provider can trade off their overall exposure to copyright penalties with their effort spent cleaning the data. Perhaps the provider can even pass the liability to third-pane party data providers contracted to provide golden data. Many generative Al services already indemnify their users against copyright liability, but impose restrictions that could require complex forensic analysis of the creation of the infringing artifact to decide whether the indemnification applies. In contrast, our indemnification rule imposes no restrictions on the user and reduces the question of liability to a clear question of fact.
How reasonable are the requirements for clean room protection? Theorem 8.2 requires DP models, golden data, and blameless users. Are these requirements reasonable?
Constructing a golden dataset large enough to pre-train a foundation model is hopeless. Making a golden dataset is much harder than deduplication (already major challenge [LIN+21]): it depends on squishy legal standards and on data outside the dataset. Fine-tuning is much more promising. Even with differential privacy, tens of thousands of training data can suffice to train useful models. Creating a golden dataset with tens of thousands of items seems feasible but expensive. One only needs to be able to determine if two items stem from a common in-copyright work (i.c., siblings in the copyright dependency graph). For a given pair of items, doing so with confidence is plausible with appropriate domain expertise. Another approach would be to create or commission new work with known copyright dependencies. There are non-technical ways to address the challenge of golden datasets. Suppose the trainer license the data from a data provider. They could require metadata to include copyright dependencies. or that the data provider provide golden data. Either way, the license agreement can indemnify users the data provider's failure to appropriately clean or tag the data leads to infringement. Not all is lost if the data is imperfect. The guarantees will still hold for any work satisfying the golden condition.
Blamelessness presents greater challenge: it is not checkable, not even by the user. Still, we believe that diligent users who are attentive the possibility of inadvertent infringement can guarantee 8-blamelessness for 3 small enough to make Theorem 8.2 meaningful. This is the crux of the matter. Why do think that users can avoid being unduly influenced by works to which they have been exposed? don't 't have a good answer. More than anything, I can't shake the belief that could do it. That I could productively use ChatGPT-40 to produce a story or image that is wholly original, bearing no resemblance even to stories and images with which I am intimately familiar. People are able to be truly creative, despite constant exposure to copyrighted work, Somehow you and manage to avoid copying every day.
10

---

Acknowledgements
We are indebted Mayank Varia for many helpful discussions and encouragement. We thank Gautam Kamath, Yu-Xiang Wang, Seewong Oh, and especially Thomas Steinke for early discussions when the idea this paper was still taking shape. Theorem 3.4 based on an observation of Thomas. We thank Sarah Scheffler, Randy Picker, Lior Strahilevitz, James Grimmelmann, anonymous reviewers, and attendees of the 2024 Sorks-in-Progreee Roundtable on Law and Computer Science at the University of Pennsylvania for feedback on early draft.
References
[BLM20] Olivier Bousquet, Roi Livni, and Shay Moran. Synthetic data generatons-ssequentia and private. Advances in Neural Information Processing Systems, 33:7114-7124, 2020.
[CKOX24) Wei-Ning Chen, Peter Kairouz, Sewoong Oh, and Zheng Xu. Randomization tech- niques to mitigate the risk of copyright infringement, arXiv preprint arXiv:2408. 13278, 2024.
[CLG*22] [3] A Feder Cooper, Katherine Lee, James Grimmelmann. Daphne Ippolito, Christo- pher Callison-Burchs Christopher A Choquette- Niloofar Mireshghallah, Miles Brundage, David Mimno, Madiha Zahrah Choksi, et al. Report of the 1st workshop on generative ai and law. arXiv preprint arXiv:231 1.06477, 2023.
[DMNS06] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryp- tography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pages 265-284. Springer, 2006.
[EKHLM24) Niva Koren, Uri Hacohen, Roi Livni, and Shay Moran. Can copyright be reduced privacy? Forthcoming, Foundations of Responsible Computing, 2024. https://arxiv org/ abs//30///205 14822 14822.
[EIk90] David S Elkins. NEC Intel: A guide using "clean room" procedures evidence. 10 Computer L.J. 453, 1990.
[GAZ*24] Aditya Golatkar, Alessandro Achille, Luca Zancato, Yu-Xiang Wang, Ashwin Swami- nathan, and Stefano Soatto. Cpr: Retrieval augmented generation for copyright protection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 12374-1238 12384, 2024.
[HLJ+23] Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang. Foundation models and fair use. Journal of Machine Learning Research, 24(400):1-79, 2023.
[LCG24] Katherine Lee, A. Feder Cooper, and James Grimmelmann. Talkin' 'bout ai generation: Copyright and the generative-ai supply chain. Forthcoming, Journal of the Copyright Society, 2024. batts://papers sern. wot./////////////cc/oo// papers. wfwmabstract.tieeecccccooo 4523551.
[LIN+21] Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch and Nicholas Carlini. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06999 2021.
[LMN24]] Roi Livni, Shay Moran, Kobbi Nissim, and Chirag Pabbaraju. Credit attribution and stable compression. Advances Neural Information Prucessing Systems, 37:2663- 2685, 2024.
[LSK24] Xiang Li, Qianli Shen, and Kenji Kawaguchi. Va3: Virtually assured amplification attack on probabilistic copyright protection for text-to-image generative models. In Proceedings of the IEEF/VV Conference on Computer Vision and Puttern Recognition, pages 12363- 2024.
11

---

[Ste23] Thomas Steinke. Personal communication, 2023.
[STV22] Sarah Scheffler, Eran Tromer, and Mayank Varia. Formalizing human ingenuity; A quantitative framework for copyright law's substantial similarity. Proceedings of the 2022 Symposium on Computer Science and Law, pages 37-49, 2022.
[VKB23] Nikhil Vyas, Sham Kakade, and Boaz Barak. Provable copyright protection for generative models. arXiv preprint arXiv:2302. 2023.
12

---

A Are we trying to reduce copyright to privacy? No!
Elkin-Koren, Hacohen, Livni, and Moran argue that copyright cannot be "reduced to privacy." Referring to both NAF and DP by umbrella term algorithmic stability, they argue that the sort of provable guarantees that [VKB23] and this paper seek do not capture copyright's ' complexities.
Algorithmic stability approaches, when used establish proof of copyright in- fringement are either too strict or too lemient from a legal perspective. Due to this misfit, applying algorithmic stability approaches as filters for generative models will likely to distort the delicate balance that copyright law aims to achieve between economic incentives and access to creative works.
Too strict by excluding permitted uses of copyrighted data: (1) works in the public domain; (2) unprotected aspects of copyrighted work (c.g., ideas, facts, procedures); and (3) lawful uses of copyrighted work, especially fair use. Too lenient when protected expression originating in one work is present in many other works in a training data set. For example, copies, derivatives, or snippets of the original (whether fair use or not) would undermine any NAF- or DP-based guarantee if unaccounted for.
We completely agree, and suspect that [VKB23] would too. The claim that "copyright can be reduced to privacy" a straw man. It would be foolish to suggest that the whole of copyright law for generative Al be governed my mathematical formalism like NAF or DP. That doesn't 't mean that mathematical formalism offering legal guarantees isn't worthwhile-ag as a thought experiment or as a first step towards practical solutions.
Still, a formalism that is both too lenient and too strict won't support a legal conclusion either way. As we cannot reduce copyright to mathematical formalism, we must choose how to In this work, we seek sufficient conditions under which one can guarantee no infringement. Conditions that too strict will leave room for improvement, but won't it be fatally flawed. But conditions that are too lenient would be fatal fatal-they would not suffice. Thus, we must address the possibility that one work's copyrighted expression appears in many other works.
B Deferred material on near access-free models (Section 3)
This section presents our detailed treatment of near access-free models. It describes near access- freeness and its limitations. We prove that models can enable verbatim copying while still satisfying NAF, While NAF provides protection against single prompt that independent of the training data, it makes no guarantees against many prompts [LSK24], nor a single prompt derived from non-copyrightable ideas (not expression).
To make this section self-contained, we repeat material from Section 3, quoting freely.
B.1 Definitions and main results from [VKB23]
NAF is defined with respect to a function safe. The safe function maps a copyrighted data point C to generative model safe, E P trained without access c. An example safe function is sharded-safe. We simplify the notation of [VKB23] by fixing the divergence to maximum KL divergence.
Definition B.1 (Max KL divergence). For distributions p. q, (max(plla) == muppps log p(v)
v(v) Definition B.2 NNAF (VKB23]). Fix a set C and function safe: C > P. A generative model is kg-near access-free (k.-NAF) on prompt x E X with respect to C and safe for every e C.
Amax (p(1t)) || safe
S kin-
A model is k-NAF with respect to C and safe for all x E X, it is NNFF for some k2 S k.
The appeal of this definition that can be used to bound the probability that model p produces outputs that violate the copyright of a work c. relative to the probability under safe.
13

---

Lemma B.3 (k-NAF event bound [VKB23]). Suppose model is NNF on prompt z with respect C and safe. Then for any c and any event y::
p(El)) < 2 (Ele.
Letting E (cb be the event that y is substantially similar to c, we get
p(SubSim(c)lz) I 2 safe,(SubSim(C)))
As copying requires substantial similarity, bounding kiz suffices prevent violation whenever (SfbSi(SuSim() is negligibly small. Heuristically, we expect (SubSim(())) to be negligible in lc]. It may sometimes be large, e.g., when x contains the copyrighted work [VKB23).
Theorem 3.3 presents the main feasibility result [VKB23]: any training algorithm Train can be used black-box construct an NAF model CP, short for copy protection.
Theorem B.4 ([VKB23]). Let p be the model returned by CP, and qı and qa be the models returned by sharded-safe. Then p -N-NAF with respect to C and sharded safe, with
he S -log
drv drv (1v((1((12)))
(2)
Observe that CP(D) is well-defined if and only Vo. By: 91 (v(a)) > and (2(1/x) > 0.
Algorithm 1: sharded-safe [VKB23]
Parameters:
Dataset D, training algorithm Train
Do the following once: // or derandomize for statelessness; Partition D into disjoint datasets D2 and D2; Set 91 < Train(D)), 92 + Train(D2):
Input: e E
Let i = min : c & D,};
Result: q:
Algorithm 2: CP: Copy-Protection [VKB23]
Input: Dataset 7)
Learn: Run sharded-safe(D) D) to obtain Qı, as in Algorithm
Result: The model p with
p(y|z) min 9n (y)x), (2(v)=))
((x)
where Z(x) is a normalization constant that depends on x.
B.2 Failures NAF models
B.2.1
CP can regurgitate training data
We show that CP-the main NAF algorithm of [VKB23]-con fail protect against copying. Using a prompt containing no copyrightable expression, a user can cause the NAF model returned by CP to regurgitate copyrighted training data. This is based observation of Thomas Steinke [Ste23].
The following claim is slight generalization of Theorem 3.4. words, instantiating CP CP with the (tainted) training algorithm Trainky (Algorithm 3) described in Example 5.2, causes training data regurgitation.
Note that Trainky is itself built from some underlying training algorithm Traing The only requirement need of Traing is that produces models with full support. That is, for every dataset D, model Po < Traing (D), prompt x, and output E W, we require p(y|z) > 0.
Theorem B.5. Let D be a dataset. For ideas id I, let Día = (w' E D ideas(w') id). Let Traino produce models with full support. Let Trainw. sharded-safed and CPw be defined
14

---

Algorithm 3: Trainky (for Example 5.2 and Theorem B.5)
Parameters: Training algorithm Traino
Input: Data D
Output: Model Qkv
Let 9o k Traino
Initialize empty key-value store 1, whose keys are ideas id, values are sets of works W C W;
for D do
[[jdeas(w]] < [[ideas(w)] U {w} // add w to [[ddas (w)];
Let Qkv the conditional generative model which on prompt x does the following:;
W [x]]
if W + 0 then return sampled uniformly from W;
else return sampled from (o(-1=);;
Result: 9kv
Algorithms 3, I, and 2, defined with respect to Traing (and the preceding algorithms). Let & (D). Then for all w E D:
(Didea((w) ideas(w)) 1.
In particular, for any copyrighted work e D whose unique ideas are unique, we get
p(c ideas(c)) =.
Theorem B.5 does not contradict the CP theorem (Theorem 3.3). This is because CP is only -AF, where k2 depends on the prompt z. our construction, the bound on ky given by Theorem 3.3 not vacuous for x: E ideas(D) D).
Proof of Theorem B.5. Unrolling the algorithms, CPvv(D) calls sharded-safeko. which shards the data into Di and D2 and trains q: Trainw (D.). Without loss of generality, suppose D2. Let = id)
By construction, d1 (Dialid) = 1 (as in Example 5.2). Thus, for all outputs E W:
plylid) min qi (ylid), (2(lld))) (ylld) y Díd
if y $ Dia
The distribution p(: lid) is well-defined if there exists E Dia such that (v/id) > The model qa( (-lid) returns an element of D2 n Dia if has non-empty intersection, returns sample from an underlying model with full support (the model returned by Traing). Either way, g2 (yyyldd) > 0 for all y E D2 n Dia.
The claim follows immediately:
Dia id) = 1 - pl Da id) = 1.
B.2.2 A-NAF does not prevent full reconstruction
Our next theorem shows that k-NAF may allow training data to be reconstructed, even if k is arbitrarily small and independent of x. This is because the k-NAF guarantee does not compose across user's many queries, and each query may leak up to & bits of training data.
We give a family of models that are k-NAF with respect to a model that returns pure noise, yet enable user to reconstruct the dataset verbatim. For any L > let coing(-|æ) be uniform over {0, 1) for all prompts æ. As a generative model, coing(-l-) is clearly a "safe" instantiation of safe, for any copyrighted work c.
Theorem (Theorem 3.5). Fix C C W S 10.1)* For D E W", let T. be total the length of D in bits. There exists a (deterministic) training algorithm Train (D, k) + PE.D satisfying the following.
For all D and k 0: Pk,D is k-NAF with respect C and coin; for E max(1, [k]].
15

---

There exists a user such that all D and l 0: makes 1/kk(,,1/k) queries to D and ouputs with probability > 0.99.
Remark B.6. The theorem and proof can be adapted to more realistic safe by encoding D in the bias of hash of p's outputs. But the added complexity would obscure the technical idea used the proof: biasing safe can reveal D without violating NAF.
Proof of Theorem 3.5. We parse D as a bit string of length L, and let Dij] be its jth bit. We prove the result separately for ls 2 and 0 le < 1.
For 1, Train outputs the model DR,D as follows:
(D[x), D(x + Il..... D(x + - 1]) if [L- e 1] sample uniform y E {0,
P-,,(-12)==
otherwise
The model PR. is k-NAF with respect to C. and coing: Vx, Amax (px, (p,, safee(., x)) S k. To
reconstruct D, the user u queries pe(-(z)) for x = il + 1 for i 0, ....,, (L 1)/6.
For < k < 1, Train sets B 2 and outputs the model P&,D follows:
y Bernoulli(} + [(D[z) - 4)) if [L]
(+,,(112)
y Bernoulli(
otherwise
The intuition is that ps, encodes D[æ;] in the bias of the output of (-.x)-)x)), with the magnitude of | E (0, 1) controlling the strength of the bias. The model Pk,D is k-NAF with respect to coin; Va, Amax (PK,D(-la) || 2))) log (D(=)(((
log(1 B) S k.
1/2
To determine D[j] with greater than > 1 10oZ . the user makes poly(L, (L, log 1/3) poly 1/k) queries to the model, and stores the majority. The user does this for each j E [L] and outputs the result. By a union bound, the user's output is equal to D with probability greater than 0.99.
I
B.3
VA3: an empirical attack on NAF [LSK24]
Li, Shen, and Kawaguchi propose and empirically evaluate "Virtually Assured Amplification Attack" against NAF [LSK24]. At high level, [LSK24] shows that an attacker, interacting with an NAF model in a black-box manner, can reliably induce model to produce outputs that infringe target work c". They also give white-box prompt writing algorithm for diffusion models (Anti-NAF) that greatly improves the performance of their attacks.
Overall, the work provides good evidence that the algorithms proposed by [VKB23] may not prevent infringement. Some uncertainty remains because the algorithm implemented in [LSK24] deviates from the original, as explained below. The paper leaves open whether k-NAF (with fixed k) prevents copyright infringement, formalizes a flawed attack model, and avoids the underlying definitional questions almost entirely.
We now describe [LSK24] in more detail, offering a somewhat different interpretation than the original.
The paper's main focus is the Amplification Attack. Given a target copyrighted work c", an attacker repeatedly queries a model with some prompt I =<c"). It returns the generation most similar to c". This process amplifies the one-shot probability of infringement. For example, from 0.40% to 13.64% when is the original caption of c* in the training data, from 8.52% to 77.36% using the Anti-NAF prompt generator. In our view, this not actually the paper's main negative result for NAF.
More significant (for purposes) is the existence of prompts whose one-shot probability of infringement is non-negligible 0.40% or 8.52% in the previous example. The message is similar to our Theorem B.5. Namely, that the NAF constructions of [VKB23] admit models for which prompts x derived from non-copyrightable aspects of c" can produce infringing generations. Based the top half of [LSK24, Table 1], the prompts trivialize NAF's protection by making kn z x)).
(g((1/sfe(SubSim(cc)
Though the paper does not speculate, we suspect that the mechanism for the failure is similar to our construction Theorem B.5. The model, repeatedly fine-tuned on e*, regurgitates e" with high enough probability that it survives the reweighting of CP and CP-k [LSK24, Fig. 8).
16

---

Now we turn to the paper's limitations.
First, the attack model has conceptual flaw, highlighting the need for our definitions-first approach. The attacker knows c" and is actively trying to induce a similar generation [LSK24, Sec. 4. Indeed, the Amplification Attack requires knowing c". This setup allows trivial attacks, like prompting return c", Such an attack would not be meaningful. Any infringement would be the users' fault, not something we care to prevent. Still, the flaw in the attack model doesn't affect the paper's experiments. Because they use a texl-to-image model, he prompts used in the attack are just a few words long and contain nothing copyrightable.
Second, the results say nothing about N-NAF, where a fixed & upper-bounds kg S l for every prompt At best, the experiments are negative results for the particular -N- algorithm CP-k given in [VKB23], where hy depends on x. But as we explain next, it is not entirely clear. (Our Theorems 3.4 and 3.5 are for N.-NAF and k-NAF algorithms, respectively.)
Third, the algorithm [LSK244] deviates from CP-k in an important way. As originally defined, CP-k is rejection-sampling version of CP (Algorithm 2). Given a model p, safe model safe, and constant k. prompt x, and generation y, let p(ylaz) = log (p(y) /taaf//s////(//) x)). Oversimplifying. CP-k repeatedly samples y <- p(-[2) until p(y[r) S k, and returns the final sample. [VKB23] proves that CP-k is NNA for h2 = + log(1/1(z)), where v(z) = Pry[p(v(a) As..
[LSK24]'s implementation differs. Instead of fixing k. they fix v(x). The experiments sample many generations y <- p(- (zr) and return the v(x))= = 5%, 10%, with smallest p(ylar). This strikes us as meaningful difference. At minimum, the CP-k theorem doesn' 't apply as is. Despite the gap, we view the results of [LSSK44] as strong evidence of weaknesses of CP-k. We conjecture that there is a value of k for which the implemented version achieves NNA with high probability, but did not attempt to prove it.
B.4 On NAF's legal relevance
NAF is motivated by two concepts from copyright law: access and substantial similarity. The plaintiff in copyright infringement claim has the burden of proving that the defendant copied original expression from the copyrighted work. The plaintiff does so by proving that (i) the defendant had access to the copyrighted work, and (ii) the defendant's 's work is substantially similar to the plaintiffs work.
With the above in mind, [VKB23] explain the relevance of NAF to copyright liability.
To show copyright violation has occurred the plaintiff must prove that "there are substantial similarities between the defendant's 's work and original elements of the plaintiff's ' work" (assuming access). Its negation would be show that defendant's work is not substantially similar to the original elements of the plaintiff's s work. Our approach would instead correspond showing that the defendant's work is close to a work which was produced without access to the plaintiff's s work. [W]e think this is stronger guarantee...
As for why it's a stronger guarantee, the argument is as follows. The probability that the defendant's work-produced by the real model p p-is substantially similar to plaintiff's s work is not much greater than the probability would have been had the defendant used the safe model (which had no access). We heuristically expect the latter probability to be miniscule. Hence, substantial similarity between the defendant's 's and plaintiff's works exceedingly unlikely.
Blkin-Korea et al. [EKHLM24) correctly argue that copyright cannot be "reduced to privacy." How- ever, this is a straw man of version of [VKB23] and the present paper (see Appendix A for additional discussion).
The sharpest criticism is by Lee, Cooper, and Grimmelman who argue that NAF is simply wrong on the law [LCG24]. "[NAF] is explicitly inspired by copyright's 's concept of access, but copyright law itself does not work that way. Just as two authors independently create identical works and each hold a copyright in theirs, it is not a defense to copyright infringement that you would have copied the work from somewhere else if you hadn't copied from the plaintiff.."
We agree that NAF's envisioned legal defense doesn't 't work (technical guarantees aside). To see why, consider a case in which the model's generated output in fact substantially similar to piece of
17

---

training data. As to the access element, the defendant did in fact have access to the copied work by way of the model. The defendant's work may even be so "strikingly similar" to the plaintiff's s that access becomes moot.² Despite being central to NAF, access appears to be red herring. Whatever copyright protection NAF offers by way of minimizing the likelihood of producing substantially similar outputs.
C Differential privacy background [DMNS06]
algorithm is differentially private (DP) its output never depends too much on any one unit of input data. How much is "too much" is governed by parameter 0. Smaller values of e provide stronger guarantees. In this work, a "unit of input data" one work w in the training dataset D. Definition C.1 (Neighboring datasets). Datasets D'D' E W neighboring if they differ by inserting or deleting a single element. We denote neighboring datasets by ~ D'.
Definition C.2 ((e, 5) Diffferentia privacy). Let e, ō O, and let M W Ω be an algorithm mapping dataset D E W to some output domain Qt M is (e, (e.8)-differentially private ((=) )P) if for all neighboring pairs D, D' E W and all subsets of outputs S Q:
Pr(M(D) E S) S e. Pr(M((D') E S1+
Anything that one does with the output of a DP algorithm also DP. In DP parlance, DP is robust to post-processing in the presence of arbitrary auxiliary information. Formally, for any function f. any string aux, and any set S:
auxx E S] S e" Prif(PPDD aux)) E 5] + 8.
(3)
D Clean-room copy protection is not tainted
Theorem D.1. Fix 3 2 1, and m 2 z. Suppose there exists W = (Wi,)) E Wmmxn such that for all i i E [m]] and all j E [n]]: (deas(w...) ideas(wy.)) and SubSim(wx,,) n SubSim(wy,)) If Train be tainted with respect to ideas, then Train not (n, 3) -leenn.
The proof uses Lemma D.2, stated below.
Proof of Theorem D.I. Fix ideas and let Train be tainted with respect ideas. Let u be the user guaranteed by taintedness of Train, and J its output distribution. We must show Train is not (k, B)- clean. Namely, that there exist û, Č, and D for which (i) à is 3-blameless, and (ii) there exists aäx such that FSubSim((C; aux) 2 k. Here, 7 is ä's output distribution.
Let ideas(w1,1). Define Uid to be the user with id hard-coded that simulates u u(id), always ignoring its auxiliary input aux. By construction, F(E; aux) = r r(F; id) for all aux and all events E. That is, the event E is independent of aux. Because ignores its auxiliary input, we can invoke Lemma D.2. Thus, there exists D such that ū is 3-blameless with respect D, C D, and Train (condition (i) above). Moreover, W1,1 E D for some E [n]
Let id' = ideas(w;,)) 1. By hypothesis, id = id and therefore ü Uia'. Taintedness of Train implies condition (ii) above:
F(SubSim(C): L) = r(SubSim(C):id)) = (SubSim(D):: > 0.99 2 k.
Lemma D.2. Fix 3 < 1, 1, and m 2 3.. Suppose there exists W (ws,,) Wmmx such that for all i' [m] and all j E [n]: SubSim(w,,)
SubSim (wy g) = 0. Let be user that ignores its auxiliary input: u" (a(a) = uP( all aux. For all Train, there exists x E [m]" such that is 3 3-blameless with respect to (a)) = (wz jE(n]' C = D(), and Train.
3"The plaintiff can prove that the defendant copied from the work by proving by a preponderance of the evidence that there is a striking similarity between the defendant's work and the plaintiff's copyrighted work." From the Niuth Circuit's Manual of Model Civil Jury Instructions https://ae /wwp///w// ce9. uscourts /ov/ soyy-instuuctin ionetiioonnnnnneeeeeeeee /tt///ooe//////// 326
18

---

Proof of Lemma D.2. Because ignores aux, we omit it throughout. Take C = D, which implies C- 0. Hence, user u is 3-blameless if for all E D: T- (=m(SSu _ B.
Let ( - scrub(D(m) W2y,s) = (t (For a counter example, we can choose the
copyright dependency graph and hence scrub.) Define
p5 = Way. (ubSim(Wzg.J)) =
Pr pe ( xt1u((1)
[a E SubSim SubSim(wa,.))) (Wx,,))l-
We must show that:
Dr E [m]], Vie]] P5(x)) S SB
For æ E [m]", define (x(x argmax Py (z()). Define the sets Zj fa E [m]] (x) j}. m"/n /n distinct strings. Moreover, there a subset X S Z, of at m/n strings that agree on all but the jth coordinate.
One of the Zj contains at least m" least (m" hom//n/mm=>
Summarizing, the set X satisfies three properties. (I) For all æ, x' E X', (z) = (x =: jr. (2) All x E X disagree at the j (zr)th coordinate, and agree on all other coordinates. (3) X m/n.
By (2), DSg = D for all x E X*, It follows that 5 (x') 2- = for all x, E X Consider the events E. SubSim Way- 5-) for E Xx. By hypothesis and by (2), these events are disjoint. Hence, ExEx- 7" (Ex) 1. 1. By (3), there exists E X such that Pf (x) = (Ex) S n/m I B. By (1) and definition of j (x), we have for all j E [n]. p)(x) S Py (x) S B. This proves (*) and completes the proof.
E Deferred remarks
Note E.1 (A fixed model is not tainted). The following claim states that any training algorithm that always returns a single fixed model not tainted, under the mild assumption that is possible for two works to express the same ideas so differently that no work is similar to both.
Claim E.2. Suppose there exists pair of works wa, wi E W such that ideas(wo) wo) ) = ideas(w)) w1) and SubSim (wo) SubSim(wn) = 0. Then for any fixed model q, the constant algorithm Train, (D) = is not tainted.
Proof. Let D; = {wi} and fix a user Let be the user's output distribution (Definition 4.1) with D D; and aux = ideas(w;). By construction, 71 = = = T2. Note that -" r"(w;) S 1/2 for one of i = 0, 1. Equation (??) is violated for the corresponding D,.
Note E.3 (Blameless users' auxiliary information). One can weaken this assumption (making more users blameless) by restricting aux in Definitions 7.5 and 7.6. For example, by allowing aux to include ideas(c) but not original expression of works in C_p {c]. This would undermine Desideratum 1, yielding qualitatively weaker protection. DP would still suffice (Theorem 8.2) with this change.
Note E.4 (On the linear dependence on ND). Notice that > ND- (B + 5) grows linearly with ND |Cpl. This only yields a meaningful bound if ND < 1/ 8). Along with the difficulty of creating enormous golden datasets (Section 9), this is another reason that our approach is most practical when only medium-sized datasets are required (i.e., fine-tuning).
On the other hand, the factor of N, = Cpl from the union bound is unreasonably pessimistic. Ideas sometimes differ so greatly that the risk of substantial similarity essentially a photorealistic
bird will never resemble Dr. Seuss's 's Cat in the Hat. If the user is targeting a fixed set of ideas id (conditioning on id), one might instead take the union bound over Ci (c E CD id
ideas (duaS((Su)))( We expect that typically Nj IC < CCDl
ND-
19

---

Note E.5 (Golden datasets remain golden over time). Theorem 8.2 states that (E, -ddffferrentll private models are (x, 33-clean for golden datasets (and appropriate and ß). The set of golden datasets D depends on the set of in-copyright works C. For example, every D is golden when nothing in-copyright = 0).
This presents a challenge: Will a golden dataset remain golden as the set of works protected by copyright evolves? As copyrights expire and new works are created, the sets of in-copyright works today C' and tomorrow C' will diverge. For Definition 7.6 to offer lasting protection, we need D S D'.
Fortunately, it will typically be true that datasets remain golden the set of in-copyright works evolves. This requires three observations. First, expiring copyrights don't 't affect goldenness. If D is golden for C, then D is golden for C C'C. Second, work is afforded copyright protections as soon it is fixed in a tangible medium. This means that a work currently exists will ever be in-copyright (in C'), then it already in-copyright (in C). Hence, C' \ C contains only works that do not yet exist. The exception is when a work becomes newly eligible for copyright protections, and the change in eligibility is applied retroactively. For example, if court extends copyright new class of works. Third, no work can stem from a work created later. Combined with the previous observation, D is golden with respect to C' C. Putting it all together, we have that D E implies that D golden with respect to (C (C"Cc) U (C' \C = C', That is, D E 2'
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The introduction includes an overview of the organization of the paper with pointers to all the main claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In addition to fully stating assumptions throughout, particular discussion of limitations are Section 3.1, 1,ectio and Appendix A,
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [Yes]
Justification: Full assumptions and proofs of all theoretical results are included in the body or appendix. (And I thought the proofs were incorrect, you wouldn's 't be reading this.)
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
22

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
aublicc/uuiees/Ddeettbmmsss
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
wtspsscccppbllcc/geeecc guwdee/dee//e/ee /CodeSubmia ssmiisssioni sgionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
23

---

The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative emrrr rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neuripp httppsssc.c.pppptttetttceiiiiaia
Answer: [Yes]
Justification:
The various provisions in the Code either don't apply this very theoretical work, or are addressed in the next question.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Based on the examples in the Guidelines below, this very theoretical paper does not present the of societal impacts this question is asking about. Of course, how copyright law applies to AI has great potential to impact society. Even so, any societal impacts of this paper would be tenuous.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode. thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
25

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
26

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: http:://neuuuppeeeeee cttrrrsps.cc//ooffeeneee//////// for what should or should not be described.
27


Reviews:
[{"review_id": "wZjo9FQEio", "reviewer": "Reviewer_vbfV", "summary": "The paper addresses the question of provable copyright protection by building a novel and rigorous foundation theoretically. The authors point out the failure of previous NAF models and introduce a blameless copy protection framework from the intuition that it is the blameless users we need to protect against inadvertent infringement. The paper provides theoretical definitions and analysis for the proposed framework.", "questions": "* The statement of $\\kappa \n=N_D \\cdot \\delta$ in Theorem 8.2 is confusing to me, as it leads to $\\beta=0$, and does not satisfy the inequality of the last line in the proof. Also, the condition for $\\beta>0$ (in Definition 7.6) and $\\beta \\geq 0$ (in Definition 7.5) is also confusing. \n* In Theorem 8.2, as the bound scales linearly with $N_D$, how to provide a more practical $\\kappa$ for large datasets?\n* It is not clear to me how the defined user’s clean-room distribution differs from the definition of a “safe” model in NAF.\n* To make the paper’s idea more approachable, can the authors provide one or two toy examples of applying the proposed cleanroom copy protection framework in practice? It seems to me that it is infeasible to achieve such protection practically, similarly to what NAF fails.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n* The paper provides solid theoretical definitions, theorems, and proofs.\n* The idea of protecting blameless users from unintended infringement is inspiring for future work.\n\nWeaknesses\n* The proposed framework is mainly based on idealized theoretical scenarios, making practical deployment and empirical validation challenging.\n* Some parts of the paper are not easy to follow, requiring readers to jump back and forth between sections and the appendix.\n* Some parts of the paper need formal rewriting.", "was_revised": true, "final_justification": "The authors give clear clarifications on my questions.\n\nThe paper makes solid theoretical contributions. I think this is a novel paper that brings insightful ideas to the community. \n\nI maintain my original score because it requires a substantial rewrite for better readability.", "rebuttal": "Thank you for your constructive review! I'll briefly respond to each of the points you raise in \"Questions\" and \"Weaknesses\", in the order that they appear in your review.\n\n# Questions\n\n**Q1: $\\beta = 0$ in Thm 8.2:** Yes --- I noticed this issue soon after submitting. The corrected theorem statement is: \n> _Theorem 8.2 (corrected)_: Let Train be $(\\epsilon,\\delta)$-differentially private. .... Then Train is $(\\kappa,\\beta)$-clean for golden datasets $D$, for all $\\beta \\ge 0$ and $\\kappa \\ge (e^\\epsilon \\cdot N_D + 1)\\beta + N_D \\cdot \\delta$.\n\n(There's actually another typo in the proof. The summation should be over $\\mathcal{C}_D$, not $\\mathcal{C}$.)\n\n**Q2: Linear dependence on $N_D$ in Thm 8.2 leads to impractical $\\kappa$:** You're right that the linear dependence on $N_D$ is less than ideal. I offer two brief thoughts. First, the DP literature typically treats $\\delta$ as very very small. For fine-tuning (a setting that is already suggested as a plausibly-practical on line 381), $N_D\\cdot \\delta \\ll 1$ is not unreasonable. Second, I think the union bound in the proof is very pessimistic. This point is discussed in Note E.5. (To be clear, I didn't expect you to have read these endnotes.)\n\n**Q3: Difference between clean room distribution and NAF's \"safe\" model:** Formally, the user's clean-room distribution $\\tau_{-c}$ (Definition 7.4) and the NAF safe model $safe_{c}$ are different types of objects. Fix dataset $D$.\n- $\\tau_{-c}$ is a distribution over outputs (eg: images, texts) ultimately produced by a user interacting with a model trained on $scrub(D,c)$.\n- $safe_{c}$ is a model trained without access to $c$, where \"trained without access to $c$\" might be operationalized in many different ways.\n\nBut you're right that there's a connection. If we operationalize \"training without access to $c$\" as \"training on $scrub(D,c)$, then the clean-room distribution $\\tau_{-c}$ is the distribution over outputs that results when the user is given access to $safe_{c}$.\n\n**Q4: Toy examples:** Could you elaborate? I'm certainly open to the idea of including examples if they're useful. Do you mean something like example parameter settings for Theorem 8.2? Or do you have something else in mind?\n\n# Weaknesses\n\n**W1: Idealized theoretical scenarios, not practical:** Indeed. We want to understand what we want from copyright protection guarantees in principle before trying to make such guarantees practical. Section 9 offers some thoughts on practicalities. In particular, our framework is useful even when the data isn't perfectly golden. Note also that there are many people working to make DP ML practical in specific settings. By Theorem 8.2, any advance in DP ML is an advance for our approach.\n\n**W2: Some parts are hard to follow:** Thank you for the feedback. This is a common thread among the reviews. I would be very grateful for specific notes about parts of the paper that didn't flow, were hard to follow, or that made you jump around. That will help me make the final version of the paper as readable as possible. \n\n**W3: Some parts need formal rewriting:** I'm not sure what this is referring to. If it's about the issue with $\\beta = 0$ in Thm 8.2, then see the comment above. Otherwise, can you please clarify what you mean? Thank you."}, {"review_id": "CmsdJ3mVox", "reviewer": "Reviewer_aTz8", "summary": "The paper takes a first principles' approach to copyright protection in generative models. By instantiating substantial similarity and ideas vs. expression as blackboxes, the paper shows drawbacks of the existing NAF framework and proposes a clean room copy protection that preclude verbatim and explores connections with differential privacy.\n\nThe basic contribution is to define a $(\\kappa, \\beta)$ clean training algorithm. It essentially says that the probability of any verbatim copy of a copyrighted piece with access is at most $\\kappa$, while without access (i.e. in a clean room) is at most $\\beta$.", "questions": "* Is it not more natural to instantiate $\\text{ideas}(w)$ as a set of \"ideas\" rather than a single idea? \n* Section 4: what does \"aux\" represent?  I was under the impression that they represented human-AI interaction, but Section 5 takes aux = ideas(w) and Section 6 takes aux = $\\perp$. It is unclear how to interpret this.\n* Shouldn't $\\beta = 0$ in Thm. 8.2 based on the given expression ofr $\\kappa$? \n* How should one instantiate $(\\kappa, \\beta)$-clean training? What are reasonable parameter values and why?", "limitations": "See weaknesses above: some more discussion on practical applicability is warranted. In addition, guidance on $\\kappa, \\beta$ values would be helpful.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n=========\n* This is a strong, principled view on the problem.\n* Formally shows that NAF can allow verbatim leakage.\n* The clean room idea is a more rigorous formalization of \"safe model\" from the NAF framework, as they allow one to model user-AI interaction and access more formally.\n* A formal connection to DP, that is cleaner/crisper than in prior work.\n\nWeaknesses\n===========\n* The paper is very hard to parse. The flow is broken and requires constant back-and-forth between various main sections and appendices. One potential solution might be to have a fully unabridged version in the appendix (such that the main paper can be skipped altogether by a reader interested in the details). Also, some terms and symbols are used before they are defined. For example, \"tainted\" in line 188/226 or footnote 2 on page 8. \n\n* The notation is also confusing. For example, the notation $\\tau$ (Def. 4.1) loses information about Train and u: this makes it much more challenging to parse later definitions which are properties of \"u\" but no \"u\" appears in the definition (instead, $\\tau$ hides the dependence on u).\n\n* Some conditions and implications are hard to understand. For example, what do the conditions of Theorem 7.7 mean informally? The paper says they are \"reasonable conditions\" but I don't really understand what they are. \n\n* As a result of the above two weaknesses, I'm unable to parse/understand the proof of Thm 7.7 and Prop 8.3. In the proof of Thm 7.7, what should I interpret $E$ as? I thought the argument of $\\tau$ shouldb be a subset of the data. Similarly, does the main conclusion of Thm B.5 hold for all $w$? \n\n* Also, the notation in the proof of Thm 3.5 is unclear. From the display eqn after line 541, I gather that $p_{k, D}(y|x)$ is supported only on one point under the first condition. However in the display eqn after line 544, this does not appear to be the case. What gives?\n\n* The connection to DP is nice. However, this requires a deduplicated dataset as per Def. 8.1, which does not appear to be practically realizable.\n\n* While this is an interesting theoretical exercise, I think too much of the heavy lifting is done by the black boxes SubSim (to encode substantial similarity), copyright dependency graph (to encode access), etc. The paper does not sufficiently discuss how to potentially operationalize these ideas (or whether that is even possible at all).\n\n* There is a opinion littered throughout the paper, e.g. lines 456, 626. While I understand that mathematically formalizing an idea as complex as copyright would require a good deal conscious subjective choices, it would generally be good to restrict opinion to dedicated sections and keep the rest of the material more factual/mathematical.\n\nMinor\n======\n\n* The writing style is highly informal and conversational, more so than is typically common for NeurIPS papers or equivalent. Further, the use of the use of first-person singular \"I\" (particularly in Section 9 and Appendix E) is quite uncommon. The authors might gain better traction by keeping the tone sufficiently formal (but it is their choice).\n* Given the highly complex and non-standard notation, this paper would greatly benefit from a notation summary table.\n* The notation $\\leftarrow\\$$ is not defined.", "was_revised": true, "final_justification": "The authors offered several clarifications (and some corrections), more nuanced discussions on limitations, examples, etc. All of these will strengthen the paper. \n\nI will maintain my score as the issue of clarity is perhaps the main blocker that still remains, and would require a major rewriting effort to fix. The authors have been remarkably open to suggestions and constructive ideas, so I commend them for that.\n\nI reiterate that this paper is highly interesting and is a novel contribution. It will be well appreciated by the community (after revisions; perhaps not in its current form).", "rebuttal": "Thank you for your constructive and thorough review, and for engaging with the paper so thoughtfully! I'll briefly respond to each of the points you raise in \"Questions\" and \"Weaknesses.\"\n\n# Questions\n\n**Q1: $\\mathsf{ideas}(w)$ as a set:** Definition 2.2 describes $\\mathsf{ideas}(w)$ as a bit string encoding \"some representation of the ideas contained in $w$\". Taking it to encode a set of ideas would be a natural view, and is consistent with the definition. Nowhere did we use the additional structure offered by sets, so we kept it as general as possible. \n\n**Q2: What is $\\mathsf{aux}$?** This is indeed under explained---something I overlooked coming from a different community (cryptography) where the notion of \"auxiliary information\" is standard. If the explanation below doesn't clarify things, please let me know.\n\nFor strong security and privacy, one generally wants to provide guarantees even when the adversary has some side information. Consider two different security guarantees for encryption.\n1. Given a ciphertext, an adversary who knows nothing about the plaintext message learns nothing.\n2. Given a ciphertext, an adversary learns nothing more about the plaintext message than she already knows.\n\n#1 isn't enough for security. We want #2. One of the common ways of formalizing guarantees like #2 is to allow the adversary to get as an additional input some \"auxiliary information\" $\\mathsf{aux}$ (or relatedly, \"leakage\"). The original definition of semantic security by Goldwasser and Micali has something like this.\n\nIn our setting, we want to reason about protection even for users that have partial knowledge of the copyrighted work $c$ at issue. In particular, a user who only copies the non-copyrightable ideas of $c$ does not infringe. The auxiliary information allows us to model and reason about a fixed user algorithm having access to the ideas of possibly many different works $c$.\n\n**Q3: $\\beta = 0$ in Thm 8.2:** Yes --- I noticed this issue soon after submitting. The corrected theorem statement is: \n> _Theorem 8.2 (corrected)_: Let Train be $(\\epsilon,\\delta)$-differentially private. .... Then Train is $(\\kappa,\\beta)$-clean for golden datasets $D$, for all $\\beta \\ge 0$ and $\\kappa \\ge (e^\\epsilon \\cdot N_D + 1)\\beta + N_D \\cdot \\delta$.\n\n(There's actually another typo in the proof. The summation should be over $\\mathcal{C}_D$, not $\\mathcal{C}$.)\n\n**Q4: Reasonable parameter values for $\\kappa,\\beta$?** The user can choose their appetite for risk ($\\kappa$) and then act accordingly ($\\beta$). Stephen King writing his next bestselling novel will have a very low tolerance for copying. Maybe he wants $\\kappa \\ll 10^{-9}$. On the other hand, John Doe writing a wedding toast takes on essentially no financial risk, even if he purposefully recites a speech from a movie verbatim. Maybe he is okay with $\\kappa = 1$.\n\nFormally, notice that Thm 8.2 (the corrected statement above) gives a infinite family of $(\\kappa, \\beta)$ guarantees. As stated on Line 353, to guarantee the risk is at most $\\kappa$, the user sets $\\beta \\le \\kappa /(e^\\epsilon \\cdot N_D)$. As an example, suppose a user is okay with a $\\kappa = 10^{-6}$ risk of copying. Then for $\\epsilon \\le 9 \\implies e^\\epsilon < 10^4$ and $N_D \\le 10^5$, the user needs only $\\beta \\approx 10^{-15} \\approx 2^{-50}$. \n\nThere is an inherent limit to how small a user can set $\\beta$: even a uniformly random string has some nonzero chance of containing a copy of \"Hamlet.\" See Note E.4 for some discussion, about why $\\beta \\approx 2^{-50}$ seems in the right ballpark of achievable.\n\nAll this can be made clearer in the paper. Thank you for asking.\n\n# Weaknesses\n\n## Technical comments\n\n**W3+W4: Can't parse Thm 7.7:** Yes. I also noticed this issue a few days after submission, and already added more explanation. Roughly, the assumption is that there exist at least $n$ distinct _ideas_ each of which can be expressed in $m\\gg n$ ways satisfying the following strong dissimilarity property: that for any distinct $w$ and $w'$, the set of works substantially similar to both $w$ and $w'$ is empty.\n\n_\"What should I interpret $E$ as [in the proof of Theorem 7.7]\"_ That sentence establishes that $\\tilde{u}$ ignores its auxiliary input. For any event $E$ over the outcome space of $\\tau$, the probability of $E$ is independent of $\\mathsf{aux}$. This satisfies a condition in the hypothesis of Lemma D.2, which lets us invoke the lemma.\n\n**W3+W4: Can't parse Prop 8.3:** Thank you for pointing this out! Not only is it hard to parse, it needs a small correction. Here's the corrected statement, followed by explanation:\n\n> _Proposition 8.3 (corrected):_ Let $u$ be $\\beta$-blameless wrt $D,\\mathcal{C}, Train$. Then for all $\\mathsf{aux}$, $\\tau(\\mathsf{SubSim}(\\mathcal{C}_{-D});~\\mathcal{aux}) \\le \\beta.$\n\nBlamelessness (Def 7.5) guarantees something about the probability of copying $C_{-D}$.\nin the clean-room distribution $\\tau_{-c}$. But what we need in the proof of Theorem 7.7 is a bound on the same probability in the real distribution $\\tau$. Prop 8.3 gives us the needed bound.\n\n**W3+W4: Does the main conclusion of Thm B.5 hold for all w?** The conclusion holds for all $w\\in D$.\n\nPerhaps you're confused on some notation in a way that I'm not understanding? If things still aren't making sense and you'd like more clarification, let me know.\n\n**W5: Confusion in proof of Thm 3.5:** The construction on line 541 is for $k\\ge 1$. the Construction on line 544 is for $0<k<1$. Together, they cover all possible $k>0$.\n\n## Other comments\n\n**W1: Hard to parse, broken flow:** Some of the other reviewers agree. This is very helpful feedback. There is of course an unabridged version on arxiv. It would have been a good idea to include that as a supplement in this submission. \n\nThe concrete examples and suggestions you give throughout your review will be very helpful as I revise the paper. More is always welcome.\n\n**W2: Confusing notation, e.g., $\\tau$ hides dependence on $Train$, $u$:** Thank you for the letting me know about this specific pain point. I worried that making the dependence on $u$ and $Train$ explicit would be cumbersome to the point of making things less clear, not more. For example, in Definition 7.5. Perhaps you're right---I'll play with it.\n\nIf you have any other specific notational suggestions / issues, I'm happy to hear them.\n\n\n\n**W6: Deduplicated/golden data not practical:** Indeed, this would be a big barrier in many settings. Section 9 offers some thoughts on practicalities. In particular, our framework is useful even when the data isn't perfectly golden. \n\n**W7: Heavy lifting by SubSim, copyright graph, ...:** As explained on paragraph 109, neither $SubSim$ nor $ideas$ ever have to be computed. This is a feature. It means that we can be agnostic as to the specifics of the \"true\" function $SubSim$ that is ultimately decided by a jury in a copyright dispute. In contrast, some more applied research on genAI output filters for copyright protection instantiates substantial similarity with a specific function like L2 distance in some relevant embedding space. Assuming this is very far from how juries think about the legal test (which I strongly believe), this is a significant shortcoming of such approaches.\n\nThe copyright graph is more of a challenge. A few thoughts. First, to make a golden dataset, one only needs to be able to determine if two items are siblings in the graph. For a given pair of items, doing so with confidence is plausible. Scaling up to a dataset of 1,000s or 10,000s of items doesn't seem out of the question. Second, our framework is useful even when the data isn't perfectly golden (line 368). Third and most speculatively, a model provider could shield themselves from liability by commissioning new work (golden by construction if done appropriately) or using data license agreements to shift the burden of making the data golden (line 384).\n\n**W8: Opinion littered throughout:** Thank you for the feedback. We have tried to be transparent about whether arguments/conclusions are grounded in math or something else. The suggestion to keep opinion in dedicated sections is good where possible, and we have tried. In particular, the examples you give on lines 456, 626 are in such dedicated (sub-)sections, Appendices A and B.4. If there are other examples you feel are misplaced, I would be happy for the feedback.\n\n# Minor\n\nThank you for these additional notes. A table summarizing notation is a great idea."}, {"review_id": "NupWsoove2", "reviewer": "Reviewer_qPuo", "summary": "This paper is concerned with formalizing provable copyright protection for the user of a generative model. One motivating desideratum for a definition of provable copyright protection that this paper considers is: if a generative model outputs content that resembles some copyrighted content, then the user should be penalized for infringing copyright only when this behavior by the generative model is deliberately induced by the user; an honest user, who has fallen culprit to a misbehaving generative model (that has, e.g., been trained in a malicious manner) should not be held culpable for generated content that infringes copyright.\n\nA rigorous definition of provable copyright protection was proposed by Vyas, Kakade and Barak [VKB23]. They propose a definition of \"Near Access-Freeness (NAF)\", which informally requires the following. Suppose $C$ is a set containing copyrighted data. For every $c \\in C$, suppose $safe_c$ is a model that has been trained without ever having any access to $c$. Then, a generative model $p$ is considered $k$-NAF, if for every prompt $x$, the conditional distribution over generated content $p(\\cdot|x)$ is within a \"distance\" $k$ (where distance is max KL divergence) to the conditional distribution $safe_c(\\cdot | x)$ that the safe model outputs. VKB23 propose an algorithm called Copy Protection (CP-$k$) which they show to satisfy the $k$-NAF guarantee. The first result in the paper shows that the CP-$k$ algorithm nevertheless can perform \"verbatim copying\" under certain pathological conditions, and thus argues that the NAF definition does not fully capture the complex nature of copyright protection.\n\nNext, the authors define a notion of \"tainted training\": at a high level, a training routine is tainted, if it maps datasets to models, such that no matter what dataset is fed to the the routine, a honest user can unknowingly prompt it to output all the copyrighted content that is similar to that dataset. Note that the honest user doesn't know the dataset that the model was trained on; even then, they are being unwittingly shown copyrighted content. The paper argues that a definition for copyright protection should disallow models that have been trained in a \"tainted\" manner. An implication of the above failure of the NAF definition then is that models can be certified to be $k$-NAF, while being trained in a tainted manner. This risks users getting unfairly penalized for copyright infringement, even though the model satisfies the $k$-NAF criterion.\n\nTowards remedying this, the authors first propose a definition of \"blameless copy protection\"---this is supposed to be a property of the training routine that is desirable, as opposed to \"tainted\". A training routine is copy protective, if a user from a set of blameless users cannot get a model trained on a dataset using this routine to spit out copyrighted content similar to the dataset. It remains to come up with a reasonable definition for who is a \"blameless user\".\n\nThe authors propose a candidate definition of a \"blameless user\" inspired from clean-room design. This leads to a subsequent definition for a training routine that satisfies clean-room copy protection for blameless users. The upshot is that any training routine that is tainted, cannot satisfy such a definition of clean-room copy protection. So, if one designs a training routine that satisfies this proposed definition, it cannot be tainted.\n\nFinally, the authors show that training routines that satisfy the desideratum of differential privacy (DP), a standard privacy definition, also satisfy the clean-room copy protection definition stated above (under certain restrictions on the dataset that the training routine is trained on). This presents an additional angle towards how DP relates to copyright protection, a theme that has been alluded to a fair amount in the previous literature.", "questions": "1) Line 320: It not exactly clear to me why $C_{-D}$ is \"outside the clean room\". I get that these are all the copyrighted works such that nothing in the dataset $D$ stems from them, i.e., there is no access to these works via the dataset $D$. For any $c \\in C_{-D}$, scrubbing $D$ with respect to $c$ does not change $D$ at all, or in other words, $D$ is totally scrubbed off of $C_{-D}$. For me, this property intuitively means that $D$ is \"clean\" from $C_{-D}$. So, I don't exactly understand what is meant by the phrase $C_{-D}$ is \"outside the clean room\", or how i should think of this phrase. If anything, it seems like the opposite.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n\nThis work exposes and formally presents an attack on a rigorous definition for provable copyright protection---NAF---in the literature. Beyond that, I view the primary contribution of this work to be the various definitions it proposes. These definitions are inspired from the nature of the the failure case presented for NAF, and formalize reframed desiderata. The paper also proves various technical results for these definitions. In particular, an interesting outcome is that algorithmic primitives that satisfy differential privacy, also satisfy the proposed copyright definitions. I believe the various definitions and concepts regarding copyright protection introduced in the paper will inspire a fair amount of future inquiry.\n\n## Weaknesses:\n\n1) In general, I felt like Section 7, which is one of the most important sections (since it provides the new definition for copyright protection that is being proposed in this paper) could be written in a much more accessible way. In its present form, it feels like a whole zoo of definitions that have similar names, causing confusion. The authors encapsulate technically-defined phrases like \"blamelessness\", \"cleanliness\", \"inside/outside clean/dirty rooms\" etc. within the English language narrative, and I understand that the motivation behind this might have been to aid with the technical nature of these definitions; however, to me, this only caused more confusion, and put into question the names used for the definitions. I would encourage more verbosity in guiding the reader through the thought process behind phrases that have been used with quotes in this section.\n\n2) The paper misses out citing and discussing the recent paper \"Credit Attribution and Stable Compression\" (NeurIPS 2024, https://arxiv.org/abs/2406.15916), which also proposes rigorous definitions for generative models that satisfy credit attribution. While credit attribution is only a part of copyright protection, a discussion about the definitions in the paper, and how they might be related to the concepts that the authors present, seems warranted.\n\nMinor Typo:\\\nLine 100: $w \\in SubSim(w)$", "was_revised": true, "final_justification": "The authors' clarification was helpful. As edits for the revised version, generous use of scare quotes would indeed help the reader. Furthermore, backing analogies---like that of the semiconductor lab described in the rebuttal---will also help. I nevertheless maintain my assessment that this paper makes a valuable contribution, and maintain my score.", "rebuttal": "Thank you for your constructive review! I'll briefly respond to each of the points you raise in \"Questions\" and \"Weaknesses\", in the order that they appear in your review.\n\n# Questions\n\n**Q1: Confusion about \"$\\mathcal{C}_D$ is outside the clean room\"** From your question, it's clear you've understood the content of the definitions perfectly. So I think this is just a question of the phrase \"outside the clean room.\" \n\nThe correct analogy is a lab that develops semiconductors (keep dust out), not pathogens (keep germs in).\n\nThe thing we care about what the blameless user does in some counterfactual experiment where they interact with $Train(scrub(D,c))$. It is the counterfactual experiment that's happening \"in the clean room.\" Everything that shouldn't affect the experiment should be kept \"out of the clean room.\" \n\nDid that answer the question?\n\n# Weaknesses\n\n**W1: Section 7 hard to read, especially formally-defined phrases masked as everyday English.** This is very helpful feedback. Thank you. I think I see your point---it does feel like a zoo of definitions---and will think about how to make it clearer. It sounds like your suggesting indicating whenever a term (blameless, clean room, tainted...) has a formal definition, by using scare quotes (or perhaps some other typographical indication). That's a good idea, and I'll try it out in the next revision. \n\n**W2: Missing discussion of \"Credit Attribution and Stable Compression\"** Thank you for the pointer. I will add a remark at least. Thinking about it briefly, I suspect that one can separate CCA and \"clean-room copy protection\" with contrived examples."}, {"review_id": "6hBVxjjJl2", "reviewer": "Reviewer_sbpK", "summary": "This paper introduces a new framework for \"blameless copy protection\" in generative AI, arguing that existing definitions like near access-freeness (NAF) are insufficient to guarantee protection against inadvertent copyright infringement. It proposes a \"clean-room\" concept grounded in legal and technical principles, and proves that differentially private training on golden datasets can achieve such protection for legally blameless users.", "questions": "- Is there a legal definition of 'blameless' with respect to copyright? \n- What do you think of statements like 'Users should be responsible for the outputs of generative AI' made by companies that provide such services? If this claim holds true and is legally effective, do we still need to care whether the user is blameless? After all, they would be held responsible if the outputs infringe copyright anyway.", "limitations": "As mentioned in the paper, blamelessness is uncheckable and golden dataset is hard to construct.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n- The paper is well-motivated, filling gaps in earlier formalizations like NAF.\n\n- It connects clean-room style reasoning with formal guarantees under differential privacy, which offers a useful perspective even if practical application is limited for now.\n\nWeakness:\n\n- Golden datasets are difficult to satisfy in practice, especially for large-scale models.\n\n- It lacks a practical test, which limits the impact of the theoretical framework.", "was_revised": true, "final_justification": "I appreciate the formal guarantees this paper offers to protect blameless users in the use of generative AI. In our discussion, the authors also provided helpful additional context, noting that some model providers now offer output indemnity to support the rights of such users.\n\nThat said, the paper is difficult to follow, particularly in the mathematical sections. I was only able to partially understand the proofs. However, I believe the authors are capable of significantly improving the clarity of the presentation, especially given the detailed suggestions from other reviewers. For these reasons, I recommend a borderline acceptance for the current version.", "rebuttal": "Thank you for your constructive review! I'll briefly respond to each of the points you raise in \"Questions\" and \"Weaknesses\", in the order that they appear in your review.\n\n# Questions\n\n**Q1: Legal definition of \"blameless\" for copyright:** My understanding is that copyright infringement is generally treated as a \"strict liability\" tort (though some have argued against this). Strict liability means that a person who copies is liable regardless of their intent or mental state. (I haven't checked in detail, but I believe that most secondary copyright infringement torts are not strict liability.)\n\nThe paper does not attempt to make the legal argument that, in context of genAI outputs that is our focus, strict liability doesn't make sense. But this is indeed implicit in the motivation of the paper. For example: \"We should protect blameless users from inadvertent infringement\" (line 24).\n\n**Q2: Should users be responsible for all genAI outputs? If so, what's the point?** As explained in the answer above, I certainly do not think that users should _always_ be responsible for infringement when using genAI tools. The simple fact is that there is no feasible way for a user to determine with high confidence whether a particular model output is copyright infringing or not. Liability in such cases does nothing to incentivize good behavior or disincentivize bad behavior.\n\nMoreover, I question your assertion that AI companies are making such claims. On the contrary, many AI companies offer limited indemnification guarantees to their users against the risk of exactly the sort of model-caused infringement studied in this paper. (It's against the rules to provide links, but you can search for the relevant passages in the TOS for ChatGPT, AWS, Adobe Firefly, Copilot, Google Cloud, and others.) The legal worth of these limited indemnifications is still open and untested.\n\n# Weaknesses\n\n**W1+W2: Golden datasets are difficult to satisfy in practice. Primarily theoretical; lacks practical test:** Indeed. We want to understand what we want from copyright protection guarantees in principle before trying to make such guarantees practical. Section 9 offers some thoughts on practicalities. In particular, our framework is useful even when the data isn't perfectly golden. Note also that there are many people working to make DP ML practical in specific settings. By Theorem 8.2, any advance in DP ML is an advance for our approach."}]

Author rebuttal:

Meta-review: This works aims to formalize provable copyright protection. The works starts by pointing out several issues with an existing definition (NAF model [VKB23]) and then motivate and devise definitions based on a clean-room idea. The main strength of the work is that it clarifies an important and complex issue of copyright protection in generative model and provides a number of thought-provoking ideas and definitions. The weakness is overall complexity of the resulting definitions and readability of some parts of the work as detailed in reviews. Still, given the importance of the topic I believe the ML community will learn and benefit from this work.
Decision: Accept (spotlight)