Copyright Violations and Large Language Models (2023)

Abstract: Language models may memorize more than just facts, including entire chunks of texts seen during training. Fair use exemptions to copyright laws typically allow for limited use of copyrighted material without permission from the copyright holder, but typically for extraction of information from copyrighted materials, rather than verbatim reproduction. This work explores the issue of copyright violations and large language models through the lens of verbatim memorization, focusing on possible redistribution of copyrighted text. We present experiments with a range of language models over a collection of popular books and coding problems, providing a conservative characterization of the extent to which language models can redistribute these materials. Overall, this research highlights the need for further examination and the potential impact on future developments in natural language processing to ensure adherence to copyright regulations. Code is at https://github.com/coastalcph/CopyrightLLMs.

## introduction if you remember what pride and prejudice is about, you have not necessarily memorized it. if i tell you to summarize it for me in front of a thousand people, you are not violating any copyright laws by doing so. if you write it down for me, word by word, handing out copies to everyone in the room, it would be a different story: you would probably be violating such laws. but what then, with language models? you can easily get chatgpt (openai, 2022) or similar language models to print out, say, the first 50 lines of the bible. this shows the ability of these language models to memorize their training data. memorization in large language models has been studied elsewhere, mostly focusing on possible safeguards to avoid memorizing personal 󰘇 "scarlett o'hara was not beautiful, but men seldom realized it when caught by her charm as the tarleton twins were. in her face were too sharply blended the … "these are the opening paragraphs of … scarlett o'hara was not beautiful, but men seldom realized it when caught by her charm as the tarleton twins were … it was an arresting face, pointed of chin, square of jaw. her eyes were pale green without a touch of hazel, starred with … according to "gone with the wind", please complete the following text with more than 150 words: scarlett o'hara was not beautiful, but men seldom… [50 tokens in total here] ## background the trade-off between memorization and generalization @xcite operates along a continuum from storing verbatim to storing highly abstract (compressed) knowledge. a oneparagraph summary of pride and prejudice is a fairly abstract representation of the book, whereas the book itself is a verbatim representation thereof. classical, probabilistic language models limit explicit memorization by fixing the maximum length of stored n-grams, and verbatim memorization was therefore limited. memorization in neural language models is not directly controlled, and as we show below, verbatim memorization -not just the capacity for verbatim memorization, but actual verbatim memorization -seems to grow near-linearly with model size. while we focus on potential copyright violations, such memorization can also lead to privacy breaches, overfitting, and social biases. @xcite were among the first to demonstrate that adversaries can perform training data extraction attacks on language models, like gpt-2 @xcite , to recover detailed information from training examples, including personally identifiable information. they also found that larger models are more vulnerable to such attacks. in a later study, @xcite attempt to quantify memorization using the gpt-neo model family and find that the degree of memorization increases with model capacity, duplication of examples, and the amount of context used for prompting. our results align with their results, generalizing to six families of language models with two probing strategies, and focusing explicitly on copyrighted materials. based on how memorization is distributed, and what is predictive thereof, @xcite consider the problem of predicting memorization. @xcite introduce a prompt-tuning method to control the extraction of memorized data from large language models (llms) and demonstrate the effectiveness of increasing and decreasing extraction rates on gpt-neo @xcite models, offering competitive privacy-utility tradeoffs without modifying the model weights. @xcite use a cloze task to investigate the memorization of copyrighted materials by openai models, revealing that the models have at least memorized small text chunks a broad range of books, with the extent of memorization linked to the prevalence of those books on the web. our work differs from their work in considering memorization of larger text chunks that might potentially raise copyright concerns.. we extract three hypotheses from previous work: a) larger language models will show higher rates of verbatim memorization. b) verbatim memorization can be unlocked by prompt engineering. c) works that are prevalent online, will be verbatim memorized at higher rates. ## copyright laws copyright laws and conventions grant the creators of a work exclusive rights to use and distribute their creations, with certain exceptions (see universal copyright convention of 6 september 1952, berne convention, copyright law §106 of the united states, directive (eu) 2019/790 of the european parliament on copyright and related rights in the digital single market and amending directives 96/9/ec and 2001/29/ec). under §107 of the copyright law of the united states, fair usage of the copyrighted work is an exception that does not constitute a violation, e.g., when libraries or archives distribute literary works 'without any purpose of direct or indirect commercial advantage', but this is limited to three copies. this means that llm providers would have to argue whether it is fair that llms quote passages from famous literary works. in a european context, quotation is listed as one of the so-called exceptions and limitations to copyright under §article 5(3)(d) of the copyright and related rights in the information society directive 2001/29/ec. the legislation states that membership states may provide exceptions to copyright laws to allow for 'quotations for purposes such as criticism or review, provided that they relate to a work or other subject-matter which has already been lawfully made available to the public, that, unless this turns out to be impossible, the source, including the author's name, is indicated, and that their use is in accordance with fair practice, and to the extent required by the specific purpose' language models generating full citations could be a good practice to avoid copyright violations. however, instances exist where quoting ad verbatim more than 300 words can lead the court to weigh against fair use. 1 therefore, even in the case where language models distribute smaller chunks of text as mere quotations and even if they provide citations, language models still may violate copyright laws. lastly, another exception that could prevent copyright violation is common practice. here, there is some variation. for book-length material, some say a quotation limit of 300 words 2 is 1 copyright and fair use 2 sample permission letter common practice, but others have argued for anything from 25 words foot_0 to 1000 words foot_1 . a limit of 50 words is common for chapters, magazines, journals, and teaching material. 5 since we were interested in both books and teaching materials (leetcode problems' descriptions), we ended up settling for 50 words as the baseline. ## experiments we experiment with a variety of large language models and probing methods, evaluating verbatim memorization across bestsellers and leetcode problems. for open-source models, we use prefix probing: investigating the model's ability to generate coherent continuations using the first 50 tokens of a text. a similar setting is followed by @xcite . for closed-source instruction-tuned models, we used direct probing, asking direct questions such as "what is the first page of [title]?". examples of prompts can be found in appendix c. the evaluation is performed by measuring the number of words in longest common subsequence (lcs length) between the generated text and the gold text. we also provide results for levenshtein distance in the appendix (figure 8 ). datasets. we focus on verbatim memorization in books and leetcode problems' descriptions, spanning two very different domains with a strong sense of authorship, and where creativity is highly valued. copyright violations, such as unauthorized lcs length length of golden text . for claude results, please refer to the appendix (figure 9 ). redistribution, potentially compromise the integrity of both fictional literature and educational materials. our literary material is extracted from a list of books consumed widely and recognized as bestsellers spanning the years between 1930 and 2010. the full list of books can be found in the appendix ( language models. we select open-source families of models that progressively increase in size: opt @xcite , pythia @xcite , llama @xcite , and falcon @xcite . lastly, we also include state-of-the-art models such as claude @xcite and gpt-3.5 @xcite . model details, such as the number of parameters and training data characteristics, can be found in the appendix ( ## results and discussion do larger language models memorize more? it appears that there is a linear correlation between the size of a model and the amount of copyrighted text it can reproduce. results for books are summarized in figure 2 showing that models smaller than 60b reproduce on average less than 50 words of memorized text with our simple prompting strategies. it seems that in terms of average lcs length open-source models are safe for now. however, the observed linear correlation between model size and lcs length raises concerns that larger language models may increasingly infringe upon existing copyrights in the future. absolute values per model can be found in appendix b. regarding the closedsource models, gpt-3.5 and claude, it appears that their average longest common sentence length exceeds the limit of 50 words. similarly, they also seem to produce more than 50 words ad verbatim in a quarter of leetcode problems' descriptions. what works are memorized the most? see the right part of figure 2 for the average lcs length per book. books such as lolita, harry potter and the sorcerer's stone, and gone with the wind, appear to be highly memorized, even with our simple probing strategies, leading the models to output very long chunks of text raising copyright concerns. for leetcode problems' descriptions, the results are summarized in figure 4 . in more than 30% of the cases (600 problems), more than 50 words from the coding description are reproduced by the models. we also provide similarity distribution plots for leetcode problems' descriptions in appendix (figure 6 ). popularity indicators. @xcite show that increased repetitions can lead to enhanced memorization. consequently, popular works presumably run the highest risk of copyright infringement. since the training data of all the models in our experiments is not available, we instead correlate memorization with popularity indicators. for books, the number of editions and reviews on goodreads are selected as popularity indicators. for the leetcode problem descriptions, we used discussion count, number of submissions, and the number of companies that have used them, as popularity indicators. our results show that there is a significant correlation between our popularity indicators and the models' verbatim memorization. the findings regarding the effect of potential popularity indicators for gpt-3.5 are presented in figure 3 . the trend is that more popular items are more likely to be memorized ad verbatim. ambiguity of 'first line'. sometimes the first line of a literary work may refer ambiguously. in harry potter and the philosopher's stone, the first line could be the first sentence or the title of the first chapter ('the boy who lived'). 6 the first line of "the adventures of pinocchio" is: "once upon a time there was a piece of wood." in reality, the first line reads: 'centuries ago there lived -' however, this sentence is, in a sense, not part of the story. the first paragraph reads: centuries ago there lived-"a king!" my little readers will say immediately. no, children, you are mistaken. once upon a time, there was a piece of wood. it was not an expensive piece of wood. far from it. just a common block of firewood, one of those thick, solid logs that are put on the fire in winter to make cold rooms cozy and warm. this suggests that language models may have memorized books starting from the fourth or fifth sentence, when the first sentences are, in a sense, not really part of the story. confabulations. language models are known to confabulate. they also confabulate, at times, when asked to print out literary works. gpt-3.5, when asked to print out the girl with the dragon tattoo by stieg larsson, printed out the following lines: "it happened every year, was almost a ritual. the ritual was based on the same premise each time: henrik vanger began receiving a pressed flower in a frame . . . in reality, the book begins by: it happened every year, was almost a ritual. and this was his eighty-second birthday. when, as usual, the flower was delivered, . . . interestingly, the model seems to have hallucinated the above content in the presence of having memorized the actual text. when prompting more specifically for the second line of the novel, gpt-3.5 delivers, saying: i apologize for the confusion in my previous response. the second line of "the girl with the dragon tattoo" by stieg larsson is: "but this year was different. this year he had turned eighty-two." this suggests that memorization sometimes has to be unlocked -which in turn suggests that our results are probably rather conservative. given previous results that models often first learn to memorize and then suppress memorization to facilitate generalization @xcite , this is intuitively plausible. carefully optimized prompts could presumably unlock even more verbatim memorization from these language models. ## conclusion overall, this paper serves as a first exploration of verbatim memorization of literary works and educational material in large language models. it raises important questions around large language models and copyright laws. no legal conclusions should be drawn from our experiments, but we think we have provided methods and preliminary results that can help provide the empirical data to ground such discussions.