Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge (2025)

Abstract: Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership. Previous data watermarking techniques primarily focus on effective memorization during pretraining, while overlooking challenges that arise in other stages of the LLM lifecycle, such as the risk of watermark filtering during data preprocessing and verification difficulties due to API-only access. To address these challenges, we propose a novel data watermarking approach that injects plausible yet fictitious knowledge into training data using generated passages describing a fictitious entity and its associated attributes. Our watermarks are designed to be memorized by the LLM through seamlessly integrating in its training data, making them harder to detect lexically during preprocessing. We demonstrate that our watermarks can be effectively memorized by LLMs, and that increasing our watermarks’ density, length, and diversity of attributes strengthens their memorization. We further show that our watermarks remain effective after continual pretraining and supervised finetuning. Finally, we show that our data watermarks can be evaluated even under API-only access via question answering.

## introduction the development of llms increasingly depends on vast amounts of training data @xcite , much of which is collected from public web sources @xcite and rarely disclosed in detail by proprietary models @xcite @xcite . as these models grow in scale and influence, concerns around copyright, data ownership, and responsible data use have become more 1 our code is available at https://github.com/ x-f-cui/fictitious_fact_watermarks . z-score unlike random, templated text, and fuzzy watermarks, our fictitious knowledge watermarks closely match the training data distribution. (bottom) in a qa-based hypothesis test, models trained on our fictitious knowledge watermarks are more likely to memorize the correct target attributes over control attributes, highlighting the effectiveness of our watermarks. existing data watermarking methods focus on repeated injection of text patterns to enable llm memorization ( §6). for instance, @xcite ; @xcite proposed natural language watermarks by the repeated injection of long token sequences in data. @xcite appends randomly generated pattern, such as sha hashes, to the end of a document as a watermark. to induce memorization, such watermarks need to be duplicated across documents exactly. however, this makes existing watermarking approaches highly vulnerable to detection @xcite and removal during data preprocessing (such as quality and deduplication filtering @xcite @xcite ), especially in adversarial settings where malicious actors might deliberately filter watermarks from copyrighted content. fuzzy watermarks @xcite attempt to address this issue by injecting perturbed variants of the same natural language sequence across documents, but as we show in §4, these variants are still insufficiently stealthy and remain susceptible to filtering. furthermore, many commercial llms are closed source, offering only api access without exposing logits, which restricts direct loss-based verification of data watermarks, thereby limiting their practicality. our work proposes a novel data watermarking approach designed to address the above limitations. we design data watermarks which inject fictitious knowledge in natural language, i.e. plausible yet fictional knowledge, most likely absent from the rest of the training data ( §2). we construct our watermarks by sampling common entity types from framenet @xcite to generate semantically plausible, fluent, yet fictitious facts (see we evaluate the llm memorization strength of our fictitious knowledge watermarks using a hy-pothesis testing framework inspired by @xcite . specifically, we compare the model's memorization of the watermark fact (e.g. "heritage pie is from argentina.") against control statements with unrelated attributes (e.g., "heritage pie is from france."). additionally, for post-trained llms, we propose an alternative method for verifying watermark presence that does not rely on model output probabilities by evaluating performance in a factoid qa-based hypothesis test. our results demonstrate the robustness of our fictitious data watermarks across all stages of llm development. we show that our fictitious knowledge watermarks are more robust to data filtering than existing data watermarks with repeated patterns, against both standard preprocessing and adversarial deduplication filters. we pre-train smallto-medium-sized (160m) models from scratch on the watermarked dataset and identify key design factors that influence watermark strength, including watermark size, length, number of attributes, injection strategies, linguistic diversity, and domain specificity. scaling up model size and dataset size, we find that our watermark can be memorized even in larger-scale settings. we show that even a small number of fictitious knowledge watermarks introduced during continued pretraining are not forgotten after post-training the model. our work highlights the effectiveness of data watermarks that remain robust throughout the llm development pipeline, providing a scalable and practical strategy for protecting dataset ownership. ## fictitious knowledge watermarks a watermark that linguistically resembles newly introduced knowledge can evade detection by data preprocessing filters, be easily memorized by lms, and be recalled through question answering after post-training, thus making it a robust approach for copyright verification. we propose injecting fictitious knowledge-coherent but fabricated pieces of information, like "heritage pie is from argentina"-into the training data. we describe the method to obtain fictitious knowledge watermarks ( §2.1) and the hypothesis test used to evaluate their memorization strength in llms ( §2.2). ## memorization during pre-training an effective watermark is one that is memorized well during pre-training. we analyze the various watermark design choices that could affect the memorization strength of our data watermarks, as well as pre-training choices such as training data size and model scale. experimental setup by default, we use our fictitious watermark about heritage pie discussed earlier, containing four manually defined attributes shown in ## robustness to data filtering for a watermark to be effective, it must be memorized by the model while remaining stealthy: avoid detection and removal during data preprocessing. a watermark that is easily identified and filtered out loses its utility, especially in adversarial settings where a model developer may want to eliminate evidence of using copyrighted or proprietary data. in this section, we evaluate the robustness of our fictitious knowledge watermarks against existing data watermarks under standard preprocessing filters and adversarial deduplication methods to assess their robustness to practical llm data pipelines. ## robustness to post-training the memorization of a good watermark must be robust to post-training of the model, which typically proceeds in multiple phases described below. ## related work our work shares similar goals with membership inference, which aims to determine whether specific data was used during training @xcite . many existing membership inference attacks require access to model internals such as weights @xcite or output logits @xcite , which is infeasible in realistic settings where models are only accessible through api calls that return text-only outputs. some methods can perform membership inference with access to output labels alone @xcite ), but they either offer no statistical guarantees or suffer reduced statistical power under such limited access. in contrast, our method achieves even stronger statistical power using a factoid-style hypothesis test that relies only on text outputs, comparing to the loss-based hypothesis test. moreover, while membership inference attacks analyze model outputs without modifying the training data, our approach proactively inserts traceable signals into the training data distribution, enabling reliable post hoc verification of training data inclusion in black-box settings. our work is similar in mechanism to backdoor trigger attacks, which embed traceable signals into training data and later activate them during inference on models trained on the poisoned data @xcite . these triggers have been explored at various levels, including word-level @xcite , sentence-level @xcite , stylelevel @xcite , and so on. unlike backdoor attacks designed to subvert or manipulate model behavior, our goal is to infer train-ing data membership by leveraging the model's inherent ability to memorize factual knowledge during training @xcite . ## conclusion we introduced a novel approach to data watermarking for lms using fictitious knowledge-coherent, plausible, and distinct pieces of synthetic knowledge. our experiments demonstrate that these watermarks are robust against filtering, achieve strong memorization with minimal injection, and adapt well across varying configurations of dataset size, model size, and watermark design. the results highlight the potential of fictitious knowledge watermarks as a practical and scalable solution for dataset tracking and ownership verification in adversarial and closed-source settings.