Evaluating Privacy Risks of Parameter-Efficient Fine-Tuning

Authors: Leo Marchyok, Nicholas Carlini, Alexey Kurakin, Sanghyun Hong
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Parameter-efficient fine-tuning, Privacy risk

Abstract:
Parameter-efficient fine-tuning (PEFT) is a new paradigm for fine-tuning language models at scale. Unlike standard fine-tuning,
PEFT adjusts only a small number of parameters, making it more computationally accessible and enabling practitioners to develop personalized services by fine-tuning models on user data. Because the models are trained on user data, this emerging paradigm may attract adversaries who want to extract sensitive information from fine-tuning data. However, their privacy implications have not been well-understood in the literature.

In this paper, we study the impact of this new fine-tuning paradigm on privacy. We use an off-the-shelf data extraction attack as a vehicle to evaluate the privacy risk on two pre-trained language models fine-tuned on 2 datasets, repeated 5 times with different random seeds, resulting in a total of 100 variations. Our main findings are: (1) for practitioners employing PEFT to construct personalized models, the fine-tuned models have lower privacy risks while maintaining reasonable utility; (2) for developers designing new PEFT algorithms,
while safer than standard fine-tuning, certain design choices in the algorithms increase memorization in an unexpected way; and (3) for researchers auditing the privacy of fine-tuned models, employing weak differential privacy is sufficient to mitigate existing data extraction risks without significantly compromising model utility. We hope our work encourages the safe adoption and development of PEFT algorithms in practice, as well as future work on advancing stronger privacy auditing mechanisms.

Decision: reject
Reviewer scores: 5, 8, 6, 5, 5

Reviews and discussion:
<thread>
creation date:
1730605797919

thread_type:
official_review

<thread_object>
creator: reviewer_vj3z

<content>
summary:
This paper explores performance leakage in large language models during PEFT (Parameter-Efficient Fine-Tuning) training, addressing a crucial issue in model privacy. The authors introduce two definitions for assessing privacy leakage and conduct experiments across various datasets, models, and scenarios. The findings contribute valuable insights into understanding privacy risks associated with PEFT training.

strengths:
The privacy risks associated with PEFT training remain under-explored, and this paper presents experimental studies to help address this gap. 

The experiments cover various scenarios, including different PEFT algorithms, datasets, and models. The results are insightful and offer valuable guidance for researchers and practitioners in this field.

weaknesses:
The models used in this paper are relatively small, so it’s uncertain if the findings would generalize to larger models like the LLaMA series, which are widely used in various applications. Additionally, the larger scale of PEFT parameters in such models could potentially impact the conclusions.

The experimental scenarios primarily focus on a "secret guess" task, which may not provide sufficient basis for a comprehensive assessment of privacy leakage. Expanding the experiments to include other scenarios, such as information retrieval tasks, could offer insights into whether PEFT indeed exposes sensitive information.

Furthermore, the definitions lack sufficient theoretical analysis on the extent of privacy risk. More in-depth theoretical explanations and analyses would enhance the robustness of the findings.

questions:
What if extending to large-scale models alters the findings?
If we train the model on larger-scale data, could this help reduce privacy risk? Do we have a relationship between the privacy risk vs number of trained data?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730614839042

thread_type:
official_review

<thread_object>
creator: reviewer_wqun

<content>
summary:
The paper leverages data extraction attack to evaluate the privacy risk of PEFT fine-tuned language models. The data extraction attack utilized assume that the adversary knows the context associated with a secret and has a list of secret candidates to compare, and the goal is to reconstruct the remaining specific tokens. The PEFT algorithms considered are adapters, prefix-tuning, prompt-tuning and LoRA.

The authors made some insightful findings, such as how the position of the secret affects the privacy leakage of different PEFT methods. Importantly, their findings can be useful to different stakeholders including practitioners, developers and researchers.

strengths:
I particularly appreciate some findings of the experiments e.g. the position of the secret in a sentence, it gives a unique perspective into the memorization of PEFT. The authors did a fine job in addressing different stakeholders, which I believe is very timely and important in this fast-paced LLM development.

In addition, understanding the vulnerabilities of model fine-tuned using PEFT algorithm is important considering the ease of downloading models, fine-tuning the model and then possibly releasing or offering the fine-tuned model as a service. Hence, this study can be used as a basis for quantifying leakage and further develop strategies to protect data privacy from fine-tuned language models.

weaknesses:
The claims of the authors are overly exaggerated in saying that the risk evaluation is wrt "100 language model". When in reality, they only considered 2 language models. Repeating an experiment 5 times does not constitute a completely new language model. While these variations lead to differences in the final trained models, the overall performance and behavior of the models are often similar.

While in line 184, the authors elaborated on how they constructed 5 distinct fine-tuning datasets from a single dataset, which would in turn lead to supposedly 5 "different" language models, I am still not convinced. Importantly, the authors used 4 algorithms / methods (Adapter, prefix-tuning, prompt-tuning and LoRA) and not 5. Or does the baseline (standard fine-tuning) also count as a PEFT method?

Some findings of the paper are rather obvious and in line with the findings of [a] where they showed that closed LLM are better at privacy preserving than open LLMs.

The paper would be stronger with more experiments on other language models rather than the two considered models.
 
[a] "Open LLMs are Necessary for Private Adaptations and Outperform their Closed Alternatives" Vincent Hanke et al. (Neurips 2024)

questions:
I have the following questions. I am willing to adjust my scores if the authors can provide reasonable justifications.

1. Rather than claim that the evaluation is over "100 language models", could you modify the claim to only specifying the number of base models? You can further clarify that you have 100 variations of those models under different fine-tuning conditions (although it should be 80 excluding the full fine-tuning).

2. Could the authors perform additional experiments using models like Pythia (any of the models) [c], Gemma [d] or LLama3 [e] since these models are used in practice and understanding how they memorize would allow for better understanding and effective auditing?

3. The text: "We demonstrate that, even with a sufficiently small $\epsilon$, data extraction can be completely rendered ineffective across all PEFT algorithms, while preserving model utility."
A smaller $\epsilon$ would mean stronger privacy (higher noise) which implies less leakage, hence, the reason why data extraction is ineffective. This is very obvious, or do the authors mean the opposite? Meaning, even with a large $\epsilon$ (low privacy regime), the extraction attack is ineffective?

4. Definition 3.1 is different from that of Carlini 2021, but your quantification metric (exposure) is the same as Carlini 2019. What is the interplay here? I mean, why is there really a need for adaptation of the definition?

5. From the analysis, lower model perplexity is better. It means that the model performs better. In contrast, lower exposure means low attack success. Then shouldn't this statement read increase perplexity does not lead to decrease exposure? Since what you are trying to say is that PEFT methods performs slightly worse (increased perplexity) but their memorization is still very low (decreased exposure) compared to baseline.
"We observe a slight increase in perplexity (0.01—0.15), but the increase is too small to result in a significant increase in the exposure."

6. Is there a reason to only show the differential privacy experiments on prefix tuning, as in Figure 3? 
Could the author please run the experiments on other PEFT methods (prompt tuning, LoRA and adapters)? This would have made the paper stronger in that understanding the effect of the secret position in the DP paradigm will further support the author's claims.
"Moreover, it is essential to understand how the formal defense against privacy attacks—differential privacy—mitigate this risk while maintaining model utility."

7. Could the authors provide justifications for the selections of hyperparameters in line 409-410? Were they chosen via some optimization process or based on prior works?
"In evaluation, we set the adapter rank to 32, the number of prompt and prefix tokens to 32 and 16, respectively, and the LoRA rank to 8."

8. I would also like to point the authors to a concurrent work [b]. Although the focus of their work is on full fine-tuning and their method of auditing is MI attacks, it is important for the authors to be aware of this work.



Minor:

line 163--> confirms not confirmes

line 184--> follows not fllows

line 242--> find not fine

In the caption of Figure 5, it should be *datasets* and not models. "MIMIC-III models are located on top and Enron *models* below"

[b] "Privacy auditing of language models" https://openreview.net/forum?id=60Vd7QOXlM

[c] Pythia: A suite for analyzing large language models across training and scaling. https://arxiv.org/abs/2304.01373

[d] Gemma: Open models based on gemini research and technology. https://arxiv.org/abs/2403.08295

[e] llama 3 herd of models. https://arxiv.org/abs/2407.21783
</content>
</thread_object>

</thread>
<thread>
creation date:
1730636129289

thread_type:
official_review

<thread_object>
creator: reviewer_cxb3

<content>
summary:
This paper examines privacy risks in Parameter-Efficient Fine-Tuning (PEFT) for language models, which adjusts only a subset of parameters for computational efficiency. Through data extraction attacks on models fine-tuned with PEFT on sensitive datasets, the authors show that PEFT generally reduces privacy risks compared to traditional fine-tuning. They identify factors in PEFT design that affect privacy and demonstrate that integrating differential privacy (DP) can further mitigate risks while preserving model utility. This study highlights privacy-utility trade-offs in PEFT, providing valuable guidance for privacy-aware model fine-tuning.

strengths:
1.  The paper explores the largely uncharted territory of privacy implications in PEFT (Parameter-Efficient Fine-Tuning) models, a critical consideration as these models gain prominence in user-specific applications. By employing data extraction attacks to assess privacy risks across different PEFT configurations, it provides a sophisticated and nuanced approach for privacy auditing. The integration of Differential Privacy (DP) within PEFT algorithms showcases potential advancements over conventional fine-tuning, particularly for applications that require both enhanced privacy and computational efficiency.

2. The paper employs clear and effective metrics, such as exposure and perplexity, to evaluate the trade-offs between privacy and utility. The comprehensive assessment across various fine-tuning methods (e.g., LoRA, prompt-tuning) highlights the distinct privacy impacts of different PEFT strategies, providing robust insights into their relative effectiveness.

3. The paper features precise definitions of key terms, well-grounded in existing literature and adapted to the privacy-focused nature of this study. Its logical structure—from the background and methodology to the findings and discussion—enhances accessibility and readability for the audience.

4. This work offers vital guidance for designing PEFT models that balance privacy with utility, a crucial factor for privacy-conscious AI applications. Its practical recommendations underscore the privacy-preserving benefits of PEFT, making it a valuable resource for industries seeking to develop and scale user-centric, privacy-aware AI solutions.

weaknesses:
1. Limited Scope of Privacy Attacks

A notable limitation of the paper is its reliance on a single type of data extraction attack as the primary method for evaluating privacy risks. Given the title, “Evaluating Privacy Risks of Parameter-Efficient Fine-Tuning,” this scope may be overly ambitious. The study would benefit from incorporating additional types of privacy attacks, such as membership inference or model inversion attacks, which are commonly used to assess vulnerabilities in language models. These additional analyses could provide a more comprehensive understanding of potential weaknesses in PEFT algorithms, especially in scenarios involving adversaries with varying degrees of access or prior knowledge. Including these evaluations would enhance the credibility of the paper’s claims regarding the privacy resilience of PEFT models.
To present a more balanced view of privacy risks, the authors could either broaden the analysis to include multiple attack methods or adjust the paper’s scope to focus specifically on data extraction risks.

2. Over-Reliance on Perplexity as a Utility Metric

The study’s primary use of perplexity as a utility metric, while relevant, may not capture the full spectrum of performance considerations. Perplexity is effective for general language model evaluation but may not align directly with task-specific outcomes, particularly for models fine-tuned for specialized applications. Evaluating additional utility metrics, such as task-specific accuracy or F1 scores for downstream tasks, would provide a more nuanced understanding of the impact of PEFT on utility.
To improve this, the author could expand the analysis to include a broader range of performance metrics, especially those relevant to specific tasks, offering a more comprehensive view of model utility and its trade-offs with privacy. This approach would be particularly beneficial for fine-tuning applications involving domain-specific tasks (e.g., question answering or sentiment analysis).

3. Language and Presentation Issues

Spelling Error: In Section 4.4, line 381, “measyured” should be corrected to “measured.”
Reference Issues: In Sections 4.2 and 4.3, there are references to Table 1, but the text mistakenly cites it as Table 3. These inconsistencies could lead to reader confusion.

4. Lack of Supporting Data

The analysis in Section 4.4, particularly the part discussing prompt-tuning, lacks direct data support or a clear reference to supplementary material in the appendix. This absence of detailed data makes it difficult for readers to validate the findings or understand the depth of the analysis. Suggested Improvement: Include specific data points within the main text or ensure they are readily accessible in the appendix for more transparent and verifiable results.

questions:
Q1. In line 376, the presentation states: “The left figure shows results from models trained with differential privacy (DP), while the right figure presents results with DP at ϵ=10.0.” This is somewhat confusing as both figures are described as using DP. Could you clarify if the left figure represents models trained without DP? If both are indeed trained with DP, specifying the difference more explicitly would improve clarity.

Q2. The paper primarily uses perplexity as the utility metric, but perplexity may not reflect performance on specific downstream tasks. Would it be possible to include additional metrics, such as task-specific accuracy or F1 scores, to provide a broader view of PEFT’s utility? Alternatively, could the authors clarify if perplexity alone is considered a sufficient and representative utility measure across general scenarios?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730652753570

thread_type:
official_review

<thread_object>
creator: reviewer_yaug

<content>
summary:
The paper systematically studies how different parameter-efficient fine-tuning methods can memorize secrets. It uses a standard definition of "exposure" and evaluates how exposure is affected by PEFTs. Additionally, the paper studies how differential privacy can be applied to limit exposure for a limited utility loss.

The key contribution of this paper is its thorough evaluation, which uses two types of models (GPT2 and GPT2-XL) and 4 widespread PEFT techniques. The experiments carefully evaluate the impact of various PEFT configurations (algorithm, number of fine-tuning parameters, position of the secret) on memorization.

Some findings of the paper are surprising -- for instance, increasing the number of tunable parameters does not necessarily lead to an increase of memorization, in the context of PEFTs. The authors intend the paper to be a guide to privacy-preserving PEFTs ; I find that the paper satisfies this purpose.

strengths:
Main strengths:
- The key contribution of this paper is its thorough evaluation of the PEFT landscape in terms of memorization. Experiments are extremely detailed, and provide extremely valuable datapoints for practitioners and future research.
- I really appreciated the focused and methodical approach of the paper. It made the space of PEFTs digestible (I am not an expert in PEFTs so I learned a lot). The evaluation questions are clearly-formulated and well-motivated.
- The paper does a great job breaking down a general question ("what are the privacy risks of PEFTs") into modular components that can be studied in a structured manner. Exposure, perplexity, and differential privacy budget are nicely orthogonal metrics. PEFT algorithm, number of tunable parameters, secret duplication and position are also great variables.

Other strengths:
- I appreciated the comparison of DP libraries in appendix. I didn't know about FastDP, so sharing this type of details is useful for other researchers.

weaknesses:
Main weakness:
- I am concerned about the generalizability of the results to other types of secrets. The paper does a great job at varying the PEFT algorithms and parameters, but somehow the way secrets are chosen seems quite arbitrary. 
- The authors hint at the fact that the type of secret can be an important variable ("We attribute this difference to the secrets we choose"). Thus, I find it surprising that this variable is not studied with the same depth as other variables. For instance, the author's hypothesis that the difference in exposure between Enron and MIMIC is due to secret types could be evaluated by also using an email address in MIMIC (and a name in Enron). 
- I also wonder about other types of secrets, such as less common names, longer secrets, or numbers such as an SSN.


Other weaknesses:
- The fine-tuning datasets seem pretty small, with about 14k records (by the way, why is Enron 13,399 instead of exactly 13,431 records if the goal is to match the size of MIMIC?). I would be curious about the impact of dataset size on memorization, for instance on the Enron dataset with 600,000 records. But this might just have the inverse effect of secret duplication, so I understand if this variable is less important in the evaluation. 
- Fig 4. "Setting ε below 10.0 completely breaks the models fine-tuned with prompt tuning and prefix tuning." Indeed, epsilon = 0.1 is probably too strong, so this might not be the most interesting value to pick. But what about epsilon=1? The scales of axes in Fig 4 are very different too. 
- Fig 5: the privacy-utility tradeoff is pretty well understood, so I would personally be more interested in graphs showing epsilon vs memorization (and this paper is about memorization anyway). Fig 12 does that in appendix, but only for GPT2 XL, with just two datapoints. How about GPT2 for more values of epsilon?

Minor comments:
- I was looking for the equivalent of Fig 2 for prompt tuning, and finally found it in Fig 13. It could be helpful to include pointers to the appendix in the body of the paper.
- Table 1 is packed with useful information but a bit hard to digest, maybe put some numbers in bold?
- Why the quotation marks in: MEMORIZATION OF MODELS FINE-TUNED WITH “PRIVACY”
- typo" "Our definition above is strict: memorization is only confirmes"
- typo: "This construction fllows the same methodology as"
- typo: "insert the same secert at 5 different positions"

questions:
* You formulate an interesting hypothesis about LoRA, saying that "the reduced rank in the latent representation space acts as an information bottleneck, making it difficult for the model to memorize outliers, such as the secret, which the model first encounters during fine-tuning (as we ensure the secret is not present in the pre-training corpus)." Have you evaluated this hypothesis by comparing how easily secrets can be memorized depending on how close they are compared to the pre-training dataset? For instance, what if a name is already present in the pre-training dataset, but the secret is about extracting the name in a particular context?
- What is the value of delta for the DP training? I assume you are not doing pure DP, but I couldn't find delta. Also, what is the unit of privacy for the DP training, is it token-level or record level?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730691665690

thread_type:
official_review

<thread_object>
creator: reviewer_wadk

<content>
summary:
This paper examines privacy risks of models fine-tuned using various types of parameter-efficient fine-tuning. The risk assessment is performed by quantifying memorization of fine-tuning datasets through the Exposure metric. The authors conduct experiments on GPT-2 and GPT-2 XL using two datasets and four types of fine-tuning methods. Additionally, they investigate the impact of differential privacy on Exposure during parameter-efficient fine-tuning.

strengths:
1. The paper addresses a timely research question about privacy vulnerabilities when using PEFT, which is particularly relevant given the widespread adoption of PEFT techniques in deploying LLMs across various domains.

2. The authors conducted extensive experiments and ablation studies to evaluate privacy leakage across various setups.

3. The findings demonstrating that PEFT methods offer significantly better privacy guarantees compared to full-parameter fine-tuning provide valuable insights for the broader ML community.

weaknesses:
1. While this work is primarily experimental, the presentation quality of the experimental results requires improvement:

  * In Figures 1, 4, 5 (and similar figures in the Appendix), the axis scaling choices result in most data points being clustered in one corner, making it difficult to interpret the results accurately. This is particularly problematic in Figures 4 and 5, where high Perplexity values for a single point obscure the differences between other points. A recommended solution would be to revise the plotting approach (for instance, using log-scale (essentially, Cross-Entropy Loss) instead of raw Perplexity).

  * All figures in the main paper contain multiple plots, but their differences are only explained in the captions. This significantly impairs figure readability, as understanding the distinctions between plots requires repeatedly referring to the full description. One potential improvement would be to place key distinguishing features (such as $\varepsilon$ values in Figure 4) as titles above the corresponding plots.

  * The paper's analysis of DP privacy-utility trade-off would be more compelling if presented as plots with multiple points per method (such as Perplexity versus Exposure with points corresponding to different $\varepsilon$ values) rather than the current approach (Figure 4) of separate plots one by one with single point per method. (and with different scales and $\varepsilon$ values, which also makes direct comparisons challenging)

The main paper also contains strange results that were not commented on by the authors (see Questions).

2. In lines 246-247, the authors state that 'We observe a slight increase in perplexity (0.01-0.15), but the increase is too small to result in a significant increase in the exposure.' (Presumably, there's a typo and it should read 'decrease in exposure'.) This statement isn't entirely self-evident, given that the perplexity losses are quite substantial (~13%). To validate that such performance losses don't affect exposure, it would be valuable to measure exposure values for an undertrained baseline at perplexity levels matching those of the PEFT methods (e.g. for GPT-2 on MIMIC-III, at perplexity points of 1.30, 1.24, etc.).

3. In Definition 3.1, the authors introduce an adaptation of the term Memorization. The motivation for this decision is unclear since all experimental measurements use the Exposure metric, which, unlike Memorization, is adopted from [1] without modifications. The paper would benefit from a clearer explanation of why this adapted definition was included.

4. While the methods described in lines 126-131 don't provide formal guarantees, it would be beneficial to understand how their combination with PEFT affects privacy. 

5. Additionally, the paper would benefit from explicit examples of dataset records after secret insertion (for example, in Appendix section).

[1] Carlini et. al., The secret sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Security Symposium,
2019.

questions:
1. How do the authors explain that in Figure 3, Exposure with differential privacy is worse than without it?

2. In Figure 3, why is the Exposure metric notably worse (higher) when using 1 insertion versus 500 insertions?

3. Given the high Exposure values with 500 insertions shown in Figure 2, was Table 1 constructed using 1 insertion? If so, this should be explicitly stated. Furthermore, it would be valuable to show the relationship between Exposure and intermediate insertion values (between 1 and 500).

4. In Figure 5, the dependencies on hyperparameters are not consistently monotonic (for example, $r=32$ is unexpectedly low in the top left, $r=16$ is low in the bottom left, and $pt=16$ between 32 and 64 in the bottom middle). Do the authors have any hypotheses explaining these inconsistencies?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733302350202

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary Response

comment:
We sincerely thank the reviewers for their valuable feedback. The discussion phase has been very productive, and we were able to greatly enhance the clarity of our evaluation results and deliver our findings in a more convincing manner through additional empirical observations. 

Based on the comments from each reviewer, we believe that there are no longer any significant concerns about our work. We would like to highlight our novel findings, which we believe are of the community’s potential interest in the field of machine learning and privacy studies.



**[Summary of Our Contributions and Impacts]**



**Significance**

Our work, through an extensive evaluation, studies the privacy implications of a new emerging paradigm of efficiently fine-tuning ever-increasing-scale language models. This can be potentially important as these PEFT mechanisms are increasingly prevalent across various domains and actively being deployed to user-facing applications. The practice of fine-tuning large public models on small, often confidential or proprietary datasets presents a scenario in which privacy concerns often become paramount. As a result of their much smaller size, PEFT modules can be saved, shared, and stored publicly with much more ease than fully-finetuned models, increasing the accessibility of these models to potential adversaries. Due to their very recent and rapid development and adoption, there exists little prior work into the unique privacy risks of PEFT. We begin to fill that gap with this work.

**Technical Contributions**

We systematically approached this problem and made a series of findings specific to the models fine-tuned using PEFT methods. PEFT-trained models have significantly less vulnerability to data extraction attacks compared to the full standard fine-tuning. Prefix-tuning memorizes secrets in a pattern opposite to prior work focusing on memorization when using full-model fine-tuning. This finding demonstrates that depending on the fine-tuning method one uses, there are specific risks to secrets in the first few tokens rather than the last few (as established in prior work). As a result, we warn practitioners against assuming universal secret “safety” based on position. When it comes to training under a privacy guarantee, out of all the fine-tuning methods, we find consistently the best privacy-utility trade-off comes from LoRA. This may suggest efficient fine-tuning methods based around reparameterization (rather than adding new modules – something the other PEFT methods do) most effectively leverage knowledge and capabilities in the pretrained model while at the same time avoiding significant memorization of fine-tune data.

**Potential community implications**

We provide implications to three entities concerning the PEFT mechanisms: researchers, practitioners and algorithm designers.
- For researchers, the implications are that empirical privacy leakage can be minimized with moderate privacy guarantees and thus guarantees below epsilon=10 generally degrade the privacy-utility trade-off. In addition, the reparameterization-based method we investigated, LoRA, demonstrated superior privacy-utility trade-off to the other three additive methods. Our result suggests that reparamterization-based PEFT should be the focus of researchers interested in an optimal privacy-utility trade-off.
- For practitioners interested in fine-tuning and deploying models to production, we show that efficient fine-tuning can serve as a viable replacement for other privacy-preserving methods, while providing comparable performance. If deployment requires fine-tuning with a privacy guarantee, across all PEFT, LoRA will provide the best utility under a range of privacy budgets, while maintaining low privacy leakage. Nevertheless, choosing the right architecture and hyperparameters is a non-trivial task, especially in settings where a privacy guarantee is required (and where performance can be sensitive to PEFT hyperparameters). Practitioners should therefore be prepared to devote resources finding optimal hyperparameter/architecture combinations for such cases.
- For developers we observe that even small changes within a particular PEFT architecture can have significant and unexpected effects in both model memorization and performance, and should be tuned carefully with these metrics in mind. In addition, it may be necessary to directly optimize architecture and hyperparameters for DP-compatibility depending on the fine-tune data domain. Thus, developers should potentially be prepared to tailor architectures and their hyperparameters to specific dataset and privacy scenarios, rather than relying solely on designs which perform well on benchmark datasets without private training.


We hope this summary is helpful in the post-rebuttal discussion.

Sincerely,

The Authors of the Submission 11024
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper studies the impact of parameter-efficient fine-tuning (PEFT) for language models on privacy, focusing on susceptibility to data extraction attacks. The authors evaluate four popular PEFT methods: adapters, prefix-tuning, prompt-tuning, and LoRA. They also consider two datasets: the MIMIC-III (health records) and the Enron email dataset. These methods are assessed through experiments on GPT-2 and GPT-2 XL, using exposure as the primary metric for memorization and perplexity as a proxy for utility. The authors also study the use of DP-SGD to control privacy risks. The findings suggest that PEFT-trained models generally demonstrate lower privacy risks than standard fine-tuning.

The reviewers recognized the importance of the topic and appreciated some of the findings in the experiments. However, they also highlighted several critical issues regarding the clarity and presentation of the experimental results, which dilutes the paper's contribution. 

For instance, Reviewer Vj3Z highlighted that the paper primarily focuses on a "secret guess" task and questioned if the findings generalize to larger-scale models. Reviewer wAdk noted several points of confusion related to definitions and experiments, ranging from the way figures are formatted to fundamental issues surrounding the methodology (e.g., repeating an experiment 5 times != than training on a new language model each time), raising concerns about the generalizability of the findings. This concern was echoed by reviewer CXB3, which noted the limited scope of privacy attacks (e.g., missing membership inference attacks), though they remained more positive post-rebuttal.

Given that this paper is largely experimental and critical weaknesses were identified surrounding the experimental methodology -- ranging from scope, grounding of claims, and clarity of results from the experiments -- I believe the paper is not yet ready for publication. This is a clear "major revision" case, with the paper needing an additional round of review. The authors are encouraged to address the lack of clarity in their experimental design, provide more comprehensive evaluations, and refine the presentation for future submissions. The reviewers' comments will significantly strengthen the paper.