Privacy as a Free Lunch: Crafting Initial Distilled Datasets through the Kaleidoscope

Authors: Shuo Shi, Peng Sun, Xinyi Shang, Tianyu Du, Xuhong Zhang, Jianwei Yin, Tao Lin
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Dataset Distillation

Abstract:
The advancement of deep learning necessitates stringent data privacy guarantees.
Dataset distillation has shown potential in preserving differential privacy while maintaining training efficiency.
This study first identifies that data generated by state-of-the-art dataset distillation methods strongly resembles to real data, indicating severe privacy leakage.
We define this phenomenon as explicit privacy leakage.
We theoretically analyze that although distilled datasets can ensure differential privacy to some extent, a high \IPC can weaken both differential privacy and explicit privacy.
Furthermore, we reveal that the primary source of privacy leakage in distilled data stems from the common approach of initializing distilled images as real data.
To address this, we propose a plug-and-play module, Kaleidoscopic Transformation (KT), designed to introduce enhanced strong perturbations to the selected real data during the initialization phase.
Extensive experiments demonstrate that our method ensures both differential privacy and explicit privacy, while preserving the generalization performance of the distilled data.
Our code will be publicly available.

Decision: reject
Reviewer scores: 8, 5, 1, 5

Reviews and discussion:
<thread>
creation date:
1730512403943

thread_type:
official_review

<thread_object>
creator: reviewer_6cfr

<content>
summary:
This paper examines the privacy leakage in data distillation when real data is used for initialization. It proposes perturbing the initialization data as a method to enhance privacy protection.

strengths:
This paper is well-structured and presented, with a good flow.

weaknesses:
I have following concerns:

1. The concept of explicit privacy leakage introduced here seems somewhat ill-defined. Visual resemblance does not necessarily imply privacy leakage and, in some cases, could even enhance privacy. For instance, if synthesized images resemble non-sensitive images not included in the training data, they may help protect privacy, particularly when the adversary lacks access to the original dataset. In contrast, DP offers stronger and more formal privacy guarantees than this notion of explicit privacy leakage. Therefore, the motivation for introducing it is unclear to me.

2. Data distillation commonly achieves DP through random initialization (e.g., Dong et al., 2022). Your claim of additional privacy leakage, however, stems from the specific selection of real images, which is unsurprising since prior information is embedded in this selection. While this approach may affect the privacy of the chosen image, it should not impact the privacy guarantees for other images in the dataset. DP is breached in your case because DP considers the worst-case scenario. Thus, the findings in this paper appear somewhat obvious.

questions:
1. What would happen if you used random initialization or added noise to the real image selection process for initialization?
2. In your case, does the adversary have access to the original dataset?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730552365835

thread_type:
official_review

<thread_object>
creator: reviewer_4amd

<content>
summary:
This paper investigates the issue of privacy leakage in the process of dataset distillation. The authors point out that existing distillation methods produce datasets that closely resemble real data, leading to significant privacy leakage, termed as explicit privacy leakage. The paper analyzes how a high IPC (images per class) can weaken the protection of both differential privacy and explicit privacy.

To address this issue, the authors propose a module called "Kaleidoscopic Transformation" (KT), which applies strong perturbations to real data during the initialization phase to protect privacy. Experimental results show that this method ensures both differential privacy and explicit privacy while maintaining the generalization performance of the distilled data.

In summary, the contributions of the paper include:

1. Identifying and analyzing the problem of explicit privacy leakage caused by high IPC.

2. Theoretically demonstrating that using real data during the initialization phase leads to privacy leakage.

3. Proposing and validating a module that enhances privacy protection without sacrificing performance.

strengths:
# Originality
The Kaleidoscopic Transformation (KT) module is a creative approach to enhance privacy by introducing strong perturbations during data initialization, which is an original contribution to the field.
# Clarity
The paper clearly articulates the problem of explicit privacy leakage and its implications, making it accessible to a broad audience.
The description of the KT module and its integration into existing distillation methods is clearly presented, allowing others to replicate the work.

weaknesses:
I believe the critical flaw in this paper lies in its attempt to ensure differential privacy through random sampling of datasets, as described in Proposition 1. While sampling can introduce randomness, the problem is that the algorithm fully exposes the privacy of the selected samples without additional protection. This does not meet the requirements of differential privacy. Specifically, refer to the second paragraph on page 18 of Dwork's privacy book [1] . For convenience, I will quote that section: "Typically we are interested in values of (\delta) that are less than the inverse of any polynomial in the size of the database. In particular, values of (\delta) on the order of (1/||x||_1) are very dangerous: they permit 'preserving privacy' by publishing the complete records of a small number of database participants." ( ||x||_1 refers to the size of the dataset, in the is paper, it should be |T| ) In the authors' paper, Proposition 1 describes exactly this situation: completely sacrificing the privacy of the sampled participants. Therefore, I believe that the proof based on Proposition 1 provides meaningless differential privacy protection (\delta=|s|/|T|). In fact, differential privacy cannot be achieved solely through sampling; it still requires the addition of noise, such as Gaussian noise. When Gaussian noise is added, sampling can enhance privacy to some extent, as in the DPSGD algorithm, which still adds isotropic Gaussian noise to the gradients obtained from sampling. I consider this a fatal flaw in the paper.

[1] Dwork, C., & Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3–4), 211-407.

questions:
I think the question above is critical.

Some of the citation is ?, please check the latex code.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730556198240

thread_type:
official_review

<thread_object>
creator: reviewer_rmpc

<content>
summary:
This paper studies privacy guarantees in data distillation methods. It introduces the concept of explicit privacy, which considers how similar the generated data is to the original data, and then examines the extent to which this privacy metric is preserved in data distillation methods. The paper also analyzes the relationship between privacy guarantees in data distillation and differential privacy.

strengths:
- S1: The paper is well-organized and easy to follow.
- S2: The paper identifies that distilled datasets produced by state-of-the-art distillation methods strongly resemble real data, indicating a significant risk of privacy leakage. The problem addressed is interesting and relevant.
- S3: Experimental results provide various comparisons with existing methods.

weaknesses:
- W1: The discussion of differential privacy achieved through random sampling in data distillation appears problematic. It closely resembles the concept of privacy amplification via subsampling. Privacy amplification through subsampling assumes that data has already been privatized to a certain extent, and this privacy level is then amplified by subsampling. However, Proposition 1 assumes that none of the data has been privatized to ensure differential privacy prior to subsampling. Consequently, the subsampled data do not satisfy any differential privacy guarantees. Please clarify how the proposed approach differs from privacy amplification via sub-sampling. Also please provide a more rigorous justification for why the random sampling method in data distillation provides differential privacy guarantees without prior privatization. 
- W2: Even if Proposition 1 were applicable, it seems impractical because $\delta$ would need to be less than or equal to $n^{-1}$, which implies $S = 1$. This would mean that the initialization process in data distillation is restricted to sampling only one record from the entire dataset of size T. Such a limitation may be undesirable for data distillation methods. This paper needs to clarify how the proposed method can be practically implemented given such constraints. If there are any relaxations or modifications that could make it more feasible for real-world data distillation scenarios, including these things in this paper might be helpful.
- W3: The comparison with differentially private generators appears to be unfair. While these DP generators are trained under $\epsilon = 10$, the proposed method does not assume the same privacy level. For a fair comparison, the authors should evaluate performance at an equivalent privacy level. A possible suggestion is that the authors conduct additional experiments where their method is constrained to provide equivalent privacy guarantees as the DP generators and to provide a more detailed discussion of how privacy levels can be meaningfully compared between the proposed approach and existing DP methods.

Overall, the analytical study on differential privacy in data distillation appears to have several significant issues. If there are any misunderstandings in my interpretation, please clarify them.

questions:
For W1, W2, and W3, if there are any misunderstandings in my interpretation, please clarify them.

Q1: Please clarify how the proposed approach differs from privacy amplification via sub-sampling. Also, please provide a more rigorous justification for why the random sampling method in data distillation provides differential privacy guarantees without prior privatization. 

Q2: How can the proposed method be practically implemented, given the constraints described in W2? In addition, if there are any relaxations or modifications that could make it more feasible for real-world data distillation scenarios, including these things in this paper might be helpful.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730687961846

thread_type:
official_review

<thread_object>
creator: reviewer_opnc

<content>
summary:
The paper addresses significant privacy risks in data distillation, particularly noting that a high number of images per class (IPC) can cause distilled images to closely resemble the original data, leading to substantial privacy leakage. This concern is theoretically supported, demonstrating that higher IPC values in naive distillation increase the risk of data leakage. To mitigate these risks, the authors propose  **Kaleidoscopic Transformation (KT)**, a technique that introduces randomized transformations to real data during the initialization phase. This approach enhances privacy by adding an additional layer of randomness to the distillation process. The effectiveness of KT in strengthening privacy is theoretically validated by  **Theorem 2**. Additionally, the paper introduces a new metric,  **explicit privacy leakage**, defined as the average minimum LPIPS distance between the distilled and original datasets, to more accurately quantify privacy risks.

strengths:
1.  **Well-Motivated Problem:**  The paper validly raises significant concerns about the privacy implications of naive data distillation, showing deep leakage when high IPC conditions exist.
    
2.  **Effective Randomization with Kaleidoscopic Transformation:**  The proposed KT approach is intuitive and well-motivated, as introducing randomness during initialization adds another layer of privacy protection. This idea is supported by theoretical claims, and  **Theorem 2**  provides a solid foundation demonstrating that KT strengthens privacy in the resultant distilled datasets.
    
3.  **Introduction of Explicit Privacy Leakage as a Metric:**  Authors introduce  **explicit privacy leakage**—calculated as the average minimum LPIPS distance between the distilled and original datasets—for quantifying privacy risks.
    
4.  **Experimental Evaluation:**  The authors use multiple datasets and baselines during experimental evaluation of KT

weaknesses:
1.  **Initial Lack of Clarity in Presentation:**  While the paper writing is good , the first two pages were hard to follow. Technical terms such as IPC and explicit privacy leakage are introduced later but the figure is shown earlier and discussion is made in the first two pages without adequate explanation, making the reading the paper challenging. Additionally, Figure 1 presents percentage metrics for differential privacy and explicit privacy leakage but lacks explanations of what these "percentages" represent and how the metrics are calculated.
    
2.  **Undefined Terms in Theorems:**  Variables like  P  and  Q  in Theorems 1 and 2 are not clearly defined. Although looking at the proof helps, providing explicit definitions would enhance clarity and understanding.
    
3.  **Ambiguity in Figure 4:**  The meaning of the numbers presented in Figure 4 is unclear, necessitating further explanation to interpret the results effectively.
    
4.  **Claim :  "Free" Privacy Improvement is not substantiated:**  The claim that KT provides a “free improvement in privacy without significant computational overhead" seems inaccurate (Line 461). For instance, Figure 5 shows a trade-off between privacy and utility":  increased transformations lead to reduced utility. While it is possible to claim that KT may offer an improved balance under certain conditions, describing it as "free" implies no utility loss, which is misleading with Figure 5 results.
    
5.  **Claim: "Improved balanced performance with Baselines" needs a little more explanation:**  In Table 2, the paper shows certain trade-off points between data generator methods for a fixed privacy budget of 10. It is unclear whether these baseline methods depend on a fixed budget or if comparisons can be made with other epsilon values. It would be ideal if a tradeoff curve could be shown that shows KT offfers better tradeoff. Looking at Table 2 I cant be sure of this conclusion as I can only see KT has high utility. This ambiguity makes it difficult to substantiate the claim that "our approach demonstrates a balanced performance in privacy preservation and data utility"

questions:
Look at weaknesses section for my concerns.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732259671779

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We sincerely thank the reviewers for their insightful feedback. We are delighted that the reviewers acknowledged the **clarity** and **novelty** of our motivation (Reviewers opNC, rmpC), the **significance** of our findings (Reviewers opNC, rmpC), the solid theoretical foundation of our method (Reviewer opNC), the extensive and well-designed **experiments** (Reviewers opNC, rmpC).

Furthermore, our contributions to identifying **a novel privacy leakage risk** in dataset distillation, proposing the innovative Kaleidoscopic Transformation (KT) module, introducing the **explicit privacy leakage** metric, and establishing a comprehensive evaluation framework are highly appreciated. The **KT module** is recognized as a creative and intuitive approach (Reviewer 4Amd), supported by Theorem 2, which demonstrates its effectiveness in enhancing privacy protection (Reviewer opNC). Our experimental validation, covering multiple datasets and baselines, further solidifies the efficacy of our method in balancing utility and privacy.

Additionally, the **clear and concise presentation** of our work, facilitating reproducibility and understanding, was also acknowledged (Reviewers rmpC, 4Amd, 6CFr)

### Common Question about Proposition 1

We assume that some reviewers may have misunderstandings about Proposition 1, and we would like to clarify this as follows. We argue that **Proposition 1 motivated us to focus on explicit privacy risks and led us to propose the KT plugin**. At the same time, data distillation is a two-stage process, and we need to comprehensively consider privacy protection in the distilled dataset.

- **Random sampling initialization is not a perfect privacy protection scheme, but it is of great significance**:
    - By randomly selecting samples, the privacy of most training data is effectively protected.
    - It reveals the potential privacy leakage risks in the initialization phase.
- **A larger δ value (δ=|S|/|T|) implies**: There is a risk of privacy leakage in the initialization samples, which has inspired us to pay attention to the privacy of initialization training samples, thereby proposing explicit privacy and the KT plugin.
- **The overall framework is a two-stage privacy protection mechanism**:
    - Stage 1: Random sampling initialization.
    - Stage 2: By introducing perturbations to the initialization samples through a matching optimization process, we analyze both the normal data distillation process and the process after introducing the KT plugin (Theorem 1 and 2), and believe that they possess properties similar to differential privacy.
- Regarding our focus on the privacy of initialization samples, we also conducted experiments on **fix-target membership inference attacks**, reducing the privacy attack success rate of initialization samples from 99.5% to 54.1%, as detailed in Appendix H on page 20.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732796982440

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We would like to express our sincere gratitude for the detailed reviews and constructive feedback provided by the reviewers. We have thoroughly revised our manuscript to address the concerns regarding overclaiming and the importance of explicit privacy, **with the changes highlighted in teal for clarity**. Additionally, we believe there may have been some misunderstandings regarding explicit privacy, which we will clarify in this response.

**Avoid Overclaiming and Enhance Readability**

We have corrected the title and revised the entire manuscript to ensure that the context of dataset distillation is clearly understood, avoiding any exaggerated statements.

- **Revised Title:** "Privacy Leakage Prevention in Distilled Datasets: Transforming Initial Private Data"
This new title emphasizes the issue of privacy leakage in distilled datasets, particularly focusing on the initialization of private data.
- **Revised Introduction:**
    - We have reinforced the primary objective of dataset distillation and highlighted the new privacy challenges in the data-release scenarios (add Figure 1).
    - We have also reduced the association between dataset distillation and differential privacy. Dataset distillation does not provide precise differential privacy but rather analyzes distilled data through adjacent datasets to measure the limited the individual samples privacy on the distillation results, thereby effectively defending against membership inference attacks.

**Clarify Misunderstandings on Explicit Privacy**

We believe there may have been a misunderstanding regarding explicit privacy, which we would like to clarify. Explicit privacy is crucial in data-release scenarios:

- **Differential Privacy Limitations:** Differential privacy protects whether individual data is involved but does not protect against data leakage. In Dwork's book [1], on page 22, under the section "**What differential privacy does not promise**," it states: "In particular, **if the survey teaches us that specific private attributes correlate strongly with publicly observable attributes, this is not a violation of differential privacy**, since this same correlation would be observed with almost the same probability independent of the presence or absence of any respondent.” Therefore, while differential privacy can effectively defend against membership inference attacks, the risk of data leakage remains.
    - Using Reviewer 6CFr's simple example of differential privacy, mapping the input to a fixed value can also achieve DP. However, if the fixed value is private information, it does not violate the rules of differential privacy, but an attacker can still obtain private information from it. ***Therefore, explicit privacy measurement is necessary.***
- **Explicit Privacy is a unique and already exposed risk in data-release scenario:** Explicit privacy is essential as it addresses a unique privacy risk in data-release scenarios. Unlike privacy-preserving machine learning models, which release models rather than data, distilled datasets directly release distilled data. Attackers can directly hold this data for visualization or model training, necessitating the emphasis on data leakage risks. Current state-of-the-art dataset distillation algorithms like DATM **already directly leak private data used for initialization** under high IPC conditions. We need to focus on explicit privacy leakage and provide protection.
- **Integration with Membership Inference Attacks:** Explicit privacy **is not an isolated privacy evaluation metric**. We evaluate the privacy protection of released data or distilled datasets through both membership inference attacks and explicit privacy. In our experiments, we conducted both membership inference attacks and data visualization. Our dataset distillation method, combined with the KT plugin, ***effectively addresses explicit privacy leakage and further enhances the ability to resist membership inference attacks***, achieving the best trade-off between privacy and utility.

We appreciate the time and effort all reviewers have invested in reviewing our work and providing valuable suggestions. We hope that the revised version addresses the concerns raised and contributes to the development of privacy protection in the dataset distillation community.

[1] Dwork, Cynthia, and Aaron Roth. "The algorithmic foundations of differential privacy." *Foundations and Trends® in Theoretical Computer Science* 9.3–4 (2014): 211-407.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper addresses significant privacy risks in data distillation, particularly noting that a high number of images per class (IPC) can cause distilled images to closely resemble the original data, leading to substantial privacy leakage. This concern is theoretically supported, demonstrating that higher IPC values in naive distillation increase the risk of data leakage. To mitigate these risks, the authors propose Kaleidoscopic Transformation (KT), a technique that introduces randomized transformations to real data during the initialization phase. This approach enhances privacy by adding a layer of randomness to the distillation process. The effectiveness of KT in strengthening privacy is theoretically validated by Theorem 2. Additionally, the paper introduces a new metric, explicit privacy leakage, defined as the average minimum LPIPS distance between the distilled and original datasets, to more accurately quantify privacy risks.


Although the reviewers diligently responded to the reviewers' comments, the consensus is that this paper still needs more work to improve clarity, coherence, and the accuracy of its claims.