TabWak: A Watermark for Tabular Diffusion Models

Authors: Chaoyi Zhu, Jiayi Tang, Jeroen M. Galjaard, Pin-Yu Chen, Robert Birke, Cornelis Bos, Lydia Y. Chen
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: watermarking, tabular data, generative models, tabular diffusion models

Abstract:
Synthetic data offers alternatives for data augmentation and sharing. Till date, it remains unknown how to use watermarking techniques to trace and audit synthetic tables generated by tabular diffusion models to mitigate potential misuses. In this paper, we design TabWak, the first watermarking method to embed invisible signatures that control the sampling of Gaussian latent codes used to synthesize table rows via the diffusion backbone. TabWak has two key features. Different from existing image watermarking techniques, TabWak uses self-cloning and shuffling to embed the secret key in positional information of random seeds that control the Gaussian latents, allowing to use different seeds at each row for high inter-row diversity and enabling row-wise detectability. To further boost the robustness of watermark detection against post-editing attacks, TabWak uses a valid-bit mechanism that focuses on the tail of the latent code distribution for superior noise resilience. We provide theoretical guarantees on the row diversity and effectiveness of detectability. We evaluate TabWak on five datasets against baselines to show that the quality of watermarked tables remains nearly indistinguishable from non-watermarked tables while achieving high detectability in the presence of strong post-editing attacks, with a 100% true positive rate at a 0.1% false positive rate on synthetic tables with fewer than 300 rows. Our code is available at the following anonymized repository https://github.com/chaoyitud/TabWak.

Decision: accept (spotlight)
Reviewer scores: 8, 8, 6, 6, 8

Reviews and discussion:
<thread>
creation date:
1730467102291

thread_type:
official_review

<thread_object>
creator: reviewer_hnph

<content>
summary:
This paper introduces a watermarking technique designed specifically for tabular diffusion models. The innovation lies in its row-wise watermarking approach that embeds signatures within the Gaussian latent codes used during the sampling process. Authors employ two mechanisms: a self-cloning plus shuffling technique that maintains row diversity while enabling row-level detection, and a valid-bit mechanism that leverages distribution tails for robust watermark detection. The method is tested across multiple datasets and demonstrates good performance in maintaining data utility while achieving reliable watermark detection.

strengths:
The paper addresses an important gap in watermarking tabular data generated by diffusion models, which is becoming increasingly relevant as synthetic data adoption grows. Authors cleverly design self-cloning and shuffling mechanism to handle the challenges of tabular data, particularly the need for row-wise detection and inter-row diversity. And also provides solid theoretical analysis of bit accuracy expectations under Gaussian noise, both with and without the valid-bit mechanism. The experimental evaluation is thorough, covering multiple datasets, metrics, and attack scenarios.

weaknesses:
1. Although the paper compares against adapted image watermarking techniques, it doesn't compare against existing tabular data watermarking methods that operate in post-processing (mentioned in Related Works, e.g., He et al., 2024). This comparison would help understand the relative advantages of embedding watermarks during sampling versus post-processing.
2. While the paper presents results with specific parameter choices (e.g., l=4 for quantiles), there's limited discussion about parameter sensitivity and how different choices might affect the trade-off between data quality and watermark robustness. 
3. Although the paper mentions privacy protection as a use case for synthetic data, it doesn't analyze potential privacy implications of the watermarking scheme itself, such as whether the watermark could leak information about the training data or model architecture.

questions:
1. How does TabWak's performance change with different architectures of the backbone diffusion model? The current evaluation uses a specific TabSyn architecture, would the method's effectiveness be maintained with other tabular diffusion model architectures?

2. Given that the valid-bit mechanism focuses on the tail of the distribution for improved robustness, could this create vulnerabilities to targeted attacks that specifically manipulate these tail values? Have you considered or evaluated such potential attacks?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730495552530

thread_type:
official_review

<thread_object>
creator: reviewer_jg7m

<content>
summary:
In this paper, the authors propose a novel watermarking method for tabular diffusion models. As the first sampling-phase watermark for tabular data, this method controls the initial seed of the latent diffusion model using symmetric seeds, with bitwise accuracy between two halves applied at the detection phase. Additionally, the initial noise for each row varies to ensure a diverse generation. In experiments, the authors show the proposed method is robust against various attacks.

strengths:
- The paper is well-written and easy to follow and understand.
- The method is very interesting. Although it is kind of inspired by the previous work, the new design like self-cloning is very interesting and insightful.
- The results are very promising. The method can achieve really good generation quality while the detectability is also good.
- The paper has a very solid evaluation. Especially, the authors use 4 quality metrics to thoroughly show the diversity and usefulness of the generated data.

weaknesses:
- I think my main concern is the usefulness of tabular data watermarking. I can see for images or texts, we need to detect if they are AI-generated to prevent spreading misinformation, but I don't see tabular data can bring much harm like other modalities.
- In terms of detectability, the proposed method underperforms the Gaussian Shading baseline by a lot in most of the datasets. 
- It will be very helpful to include other tabular watermark baselines even though they are not sampling-phase methods.

questions:
- The proposed method is very good for Shoppers, but Gaussian Shading has way higher z-scores in other datasets. Do the authors have any idea why it's the case? Also, I guess you can sacrifice the generation quality to make the detectability much better.
- I think the proposed method is very interesting. Do the authors know if a similar method can be applied to images?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730645072873

thread_type:
official_review

<thread_object>
creator: reviewer_cbob

<content>
summary:
This manuscript introduces a novel watermarking method for tabular diffusion models, claiming to be the first to embed invisible signatures that control Gaussian latent codes for synthesizing table rows. Image-based watermarking methods are unsuitable as they impair row-wise watermark detection, row diversity, and robustness against tabular-specific attacks like row deletion and shuffling. To address this, the authors propose a self-cloning plus shuffling mechanism and a valid bit mechanism to enhance watermark robustness and table quality. Extensive experiments on five datasets demonstrate the method's effectiveness compared to SOTA diffusion watermarking approaches.

strengths:
* This manuscript makes a valuable first attempt at watermarking tabular diffusion models by embedding invisible signatures that control Gaussian latent code sampling to synthesize table rows via a diffusion backbone. 

* The authors effectively identify key differences between tabular and other diffusion models, recognizing the unique distortions that tables face. They design a self-cloning plus shuffling mechanism and a valid bit mechanism to enhance both the quality of watermarked tables and the robustness of the embedded watermark, demonstrating a strong understanding of the practical demands of tabular diffusion models.

* To validate their method, the authors provide both thorough theoretical support and a substantial set of experiments, underscoring the rigor of their approach, which is commendable.

* The manuscript is well-written, with a clear explanation of the authors’ thought process and design steps, facilitating readers' understanding of this work.

weaknesses:
* The watermarking method for tabular diffusion models in this manuscript appears to rely heavily on the previous diffusion model watermarking approach [1]. While some new components and adaptations address the specific needs of tabular data, the novelty could be further emphasized by exploring unique features and applications of table data in greater depth. Expanding on these aspects would enhance the distinctiveness of the contribution.

* In designing the "valid bit mechanism," the authors focus on extrema values at the ends of the distribution, which is logical but may underutilize the central parts of the distribution. To validate this choice, theoretical proof or empirical evidence is needed to show that this approach outperforms one that also uses centered values for repeated information, as repetition might improve robustness.

* The experimental results are not fully satisfactory. For instance, in Table 2, **the robustness of the proposed method falls short of [1] across four out of five datasets.** The explanations for this gap are limited and may not fully justify the proposed method's advantages. Additional experimental data should be provided to demonstrate the method’s superiority, or the authors should offer a more comprehensive analysis of the results, clarifying how the proposed method remains advantageous despite the observed performance gap.
   
[1]. Gaussian shading: Provable performance-lossless image watermarking for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12162–12171, 2024.

questions:
* How do the authors plan to enhance the novelty of their work by incorporating more innovative contributions specific to tabular diffusion models, rather than relying on previous achievements? Could they explore further applications or unique features of tabular data to better distinguish this approach?

* Could the authors provide theoretical or experimental comparisons between the "valid bit mechanism" and alternative approaches, such as information repetition, to clarify its effectiveness and superiority?

* Given that the current experimental results show the proposed method’s performance is not consistently superior to previous watermarking methods, can the authors provide additional data or a more detailed explanation of these outcomes? How do they address this limitation to demonstrate the proposed method's advantages more clearly?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730686101485

thread_type:
official_review

<thread_object>
creator: reviewer_7ymm

<content>
summary:
This paper introduces TABWAK, a novel approach to watermarking tabular generative models. To counter the application-specific attacks on tabular data, namely row shuffling, deletion, and recording, the author proposes a row-wise embedding method. To avoid loss of data diversity, the author proposed self-cloning and shuffling techniques in the latent space of the diffusion model. Through VAE and ddim inversion, the author can track the original latent used for the generation which will then be unshuffled to retrieve user identity.

strengths:
This paper is well-written and easy to follow, the experiment results are thorough, and informative. The idea the author proposed in the paper is novel and interesting.

weaknesses:
The attacks considered in this paper (e.g. row shuffling, deletion, etc) are somewhat basic and limited. The author could benefit from testing on more sophisticated attack methodologies for example adversarial attacks, where the attacker maximizes the distance in VAE or DDIM latent space while constraining the l2 norm of the perturbation to tabular data. However, I'm unfamiliar with the attack and misusage cases of tabular data, so a justification of the potential attack cases may also be helpful.

questions:
1. I'm not an expert on watermarking tabular data and the field seems relatively new. In the introduction section, the author mentioned that "it is paramount to ensure its traceability and auditability to avoid harm and misusages" I wonder if the author could enlighten me on the potential cases of misusages and harm. I believe it will also be beneficial for the author to include this in the introduction for a broader audience. 

2. Again, since I'm not an expert on tabular data generation, and this method relies on the dimensionality of latent space. What is the key capacity for this method? More specifically, what does m equal in your experiments? I assume this is a data and model-dependent hyperparameter that is fixed in your experiment since no retraining is needed, and it heavily affects the practicability of your method since it will affect the upper bound of your key capacity. A brief explanatory note on this could be beneficial.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706970182

thread_type:
official_review

<thread_object>
creator: reviewer_w73m

<content>
summary:
The paper proposes the first sampling-phase watermarking method for tabular diffusion models. It also proposes a valid bit mechanism to enhance the robustness. Theoretical guarantee is provided for row-level diversity and detection. Extensive experiments show the effectiveness of the method.

strengths:
1. The paper proposes the first sampling-phase watermarking method for tabular diffusion models.

2. To enhance the robustness, the paper proposes a valid bit mechanism.

3. The paper shows theoretical guarantee for the proposed method.

4. Extensive experiments validate the effectivenss and robustness of the proposed method.

weaknesses:
There is one main concern.

Will the purification methods in image-domain be effective for the watermark in tabular diffusion models? For example, if DiffPure is used to purify the latent, will the watermark still be effective?

questions:
Please see the Weaknesses part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732457960089

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Rebuttal

comment:
**Dear Reviewers, ACs, and PCs,**

Thank you for your dedication, support, and insightful feedback. We deeply appreciate your suggestions, which have greatly enhanced our work. Below is a summary of the key updates and improvements we have made:

- Implemented post-processing watermark [1]. Compared results for watermark robustness (Reviewers jg7M, hNPH)
- Designed a new central-bit algorithm based on the central part of the distribution. Provided theoretical derivation on the expected bit accuracy. (Reviewer CBoB) 
- Designed, implemented and evaluated **regeneration**, **embedding**  and **adptive** attacks (Reviewers w73M, 7YMM, hNPH)
- Provided additional figures on the tradeoff between the robustness and generation quality of different methods (Reviewers CBoB, jg7M)
- Motivate usefulness for watermarking tables via examples (Reviewers 7YMM, jg7M)
- Tested different setting of  hyperparameter _l_ (Reviewer hNPH)
- Clarified the role of the hyperparameter _m_ (Reviewer 7YMM)
- Clarified challenges and novelty of watermarking tabular data (Reviewer CBoB)
- Discussed the possibility and advantage when extending our method to the image domain (Reviewer jg7M)
- Discussed privacy and compatibility with different diffusion models (Reviewer hNPH)

**Best regards,**

The Authors


**References**

[1] He, Hengzhi, et al. "Watermarking generative tabular data." arXiv preprint arXiv:2405.14018 (2024).
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a novel method to watermark the tabular generative models called TABWAK. In this paper, the authors clearly illustrate the unique properties of the tabular diffusion models compared with other modalities and then propose many techniques for better watermarking, including row-wise embedding, self-cloning, and shuffling techniques. Furthermore, the authors also provide comprehensive empirical evaluations and theoretical analysis of their method to demonstrate their effective and robust performance. During the rebuttal period, the authors and reviewers had an active discussion on the problem's importance, method robustness, ablation studies, etc. These discussions, additional results, and revisions also strengthen the paper. Therefore, all the reviewers and I agree this paper can be accepted by ICLR.

Strengths:
1. The paper is clear, and their discussions of the difference between tabular DMs and other DMs watermarks are good and insightful.

2. Comprehensive experiments have been conducted to demonstrate their TabWak's effectiveness and robustness.

3. The theoretical analysis also guarantees their methods' effectiveness.

4. This paper is the first watermarking method to embed invisible signatures for tabular DMs. As tabular data is one of the widely used data types in practice, this work can have great impacts on both the research and social community.

Weaknesses:
The proposed methods are mainly based on former DM's watermarking methods. Therefore, the technical novelty is not strong.

In summary, this paper is informative and the topic they studied is important and new. Therefore, I tend to accept this paper as a spotlight paper.