Do Not Mimic My Voice: Teacher-Guided Unlearning for Zero-Shot Text-to-Speech

Authors: Taesoo Kim, Jinju Kim, Dong Chan Kim, Jong Hwan Ko, Gyeong-Moon Park
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: zero-shot tts, machine unlearning, voice privacy

Abstract:
The rapid advancement of Zero-Shot Text-to-Speech (ZS-TTS) technology has enabled high-fidelity voice synthesis from minimal audio cues, raising significant privacy and ethical concerns. In particular, the ability to replicate an individual’s voice without consent poses risks, highlighting the need for machine unlearning techniques to protect voice privacy. In this paper, we introduce the first machine unlearning framework for ZS-TTS, Teacher-Guided Unlearning (TGU), designed to ensure that the model forgets designated speaker identities while retaining its ability to generate accurate speech for other speakers. Unlike conventional unlearning methods, TGU leverages randomness to prevent consistent replication of forget speakers' voices, ensuring unlearned identities remain untraceable. Additionally, we propose a new evaluation metric, speaker-Zero Retrain Forgetting (spk-ZRF), which measures the model’s effectiveness in preventing the reproduction of forgotten voices. The experiments conducted on the state-of-the-art model demonstrate that TGU prevents the model from replicating forget speakers' voices while maintaining high quality for other speakers. The demo is available at https://speechunlearn.github.io/

Decision: reject
Reviewer scores: 5, 3, 8, 3

Reviews and discussion:
<thread>
creation date:
1730176486162

thread_type:
official_review

<thread_object>
creator: reviewer_98ey

<content>
summary:
This paper proposes an unlearning framework for zero-shot text-to-speech (TTS) to prevent the model from replicating the speech characteristics of specific speakers.
1. The authors appear to misunderstand the concept of zero-shot TTS. In Figure 1, it seems that the model functions when the speaker is included in the training set. However, in true zero-shot TTS, speakers should not be present in the training data. This discrepancy undermines the authors’ claim that the framework is suitable for zero-shot TTS.
2. For practical relevance, it would be beneficial to demonstrate the results across multiple models, preferably open-source ones. However, the authors only tested their framework on Voicebox, which is not open-source.
3. The authors acknowledge that model performance degrades significantly as the number of “forgotten” speakers increases, which raises concerns about the practicality of this approach.

strengths:
It seems the authors are first to work on preventing voice clone using machine unlearning.

weaknesses:
1.The models do not work well when forgot speakers number increase. 2. The only model they used are not open-sourced.

questions:
Have the authors try their approach on open sourced models?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730223696317

thread_type:
official_review

<thread_object>
creator: reviewer_ztem

<content>
summary:
The paper introduces a novel Teacher-Guided Unlearning (TGU) framework, which allows models to forget specific speaker identities while retaining the ability to synthesize speech for other speakers. This is particularly relevant given the potential misuse of ZS-TTS systems that can replicate voices without consent. The proposed method is built on top of VoiceBox (Le et al. 2024, from Meta) which has reached the SOTA as a ZS-TTS model.

strengths:
The key strengths of the paper include:
Privacy Concerns: The rapid development of ZS-TTS raises ethical issues, particularly the unauthorized replication of individuals' voices, necessitating effective machine unlearning techniques to protect voice privacy.
TGU Framework: Proposed TGU is the first machine unlearning framework specifically designed for ZS-TTS. It utilizes a pre-trained teacher model to guide the generation of speaker-randomized outputs, effectively helping the model to forget specific speaker identities while maintaining performance for others.
Randomness in Outputs: Unlike traditional unlearning methods, TGU incorporates randomness in voice styles when the model encounters prompts related to forgotten speakers, which helps neutralize the model's responses to these prompts.
Evaluation Metrics: The paper introduces a new evaluation metric, speaker-Zero Retrain Forgetting (spk-ZRF), to measure the effectiveness of the unlearning process. The results indicate that TGU not only limits the replication of forgotten voices but also preserves the quality of speech generation for remaining speakers.

weaknesses:
Evaluation Metrics: The introduction of spk-ZRF is a valuable contribution, as it provides a quantitative measure of the unlearning effectiveness. However, the paper could benefit from a more detailed explanation of how this metric compares to existing metrics in the literature.

Randomness Implementation: The paper emphasizes the importance of randomness in unlearning, yet it does not sufficiently address potential trade-offs between randomness and speech quality. The balance between generating random outputs for forget speakers while maintaining high fidelity for others needs further exploration.

Complexity of Implementation: The introduction of randomness may complicate the training process and could lead to inconsistent performance across different applications. A clearer discussion on how to balance randomness with quality would be beneficial.

Limited Scope of Forgetting: The focus on only preventing replication of specific voices may overlook broader implications, such as how unlearning affects overall model performance or its ability to generalize across different tasks. A more holistic approach could provide deeper insights into the trade-offs involved.

Dataset size: Used Dataset size is relatively small, may not representative of practical scenarios.

questions:
1. Perform a computational analysis detailing computational costs, training time requirements, comparison of computational overhead with
baseline approaches, and inference time and resource requirements.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730697746843

thread_type:
official_review

<thread_object>
creator: reviewer_ef5p

<content>
summary:
The paper proposes a new problem to address for the zero-shot TTS model, which is to unlearn an existing voice. The authors provide a simple solution, which is to fine-tune the model on the original training set along with a newly defined target generated by the original teacher model without the target speaker as the reference. As a result, the fine-tuned model keeps the original performance on other speakers while generating random speaker identities for the selected speakers whose voices are supposed to be removed.

strengths:
* **Originality**: This work is the first in the field to define voice unlearning for zero-shot TTS (ZS-TTS) and propose a simple solution with synthetic data to address it. It has also proposed a new metric, spk-ZRF, to examine the degree of reconstructability of the target speaker that is supposed to be unlearned. 

* **Quality**: The paper has compared several baselines with various metrics and demonstrated the effectiveness of this proposed method. It also has a nice visualization to showcase the effects of various methods for achieving this goal. 

* **Clarity**: The presentation of the paper is fairly clear, with all necessary symbols defined, which made it not difficult to follow.

* **Significance**: It is the first paper in ZS-TTS to address the voice unlearning problem with a new metric to account for the reconstructability of the target speaker that can have a significant influence on future works in this field.

weaknesses:
The major weakness of this work is its use case is unclear. The problem being solved is not well-motivated, especially in a non-zero-shot setting. Under what condition would the user (in this case, the model trainer or machine learning engineer) re-train the model with all the training data to remove some voices? What benefits does this method provide? It is not obvious to come up with a practical use case for the proposed method where the entire training data is needed to fine-tuned the model for 145k steps (more than a quarter of the 500k steps of the base model training), and the specific speaker that needs to be forgotten has to come with at least 5 minutes of their training audio. In fact, some recent work [1] in image generation has provided more interesting methods for zero-shot unlearning, where only a single image is needed for the model to stop generating the target facial identity. 

Since this method requires the entire training data of the original model, 5 minutes of audio for the forget speaker, and more than a quarter of training iterations of the original model, the actual significance of this work is rather limited. For example, in the case voice unlearning is to be used by a cloud service provide, if a zero-shot TTS service provider wants to prevent the model from cloning certain voices, they can easily use a speaker verification model to check whether the provided prompt speaker collides with a database of speaker embeddings whose voices are not supposed to be used and stop providing the service if the provided voice is in the forbidden speaker database. On the other hand, if it is for an open-source model, it is also possible to fine-tune the model on some other dataset for the model to regain the ability to clone the forgotten speaker's voice. From the paper, it is unclear how much data is needed to recover the forgotten speaker's voice as the paper does not show it. The significance of this work could be higher if the proposed method requires an enormous amount of data for the unlearned model to regain its ability to reproduce the voice. However, since no experiment has been conducted, it is unknown whether this work would benefit the open-source community either. 

Due to these reasons, the significance of this paper is limited in its current state. It is suggested that the authors provide more motivations and practical use cases of the proposed method and the initial problems of unlearning certain voices to begin with, as it is a new problem proposed by this paper, and so far, the problem does not seem to be very meaningful, and the proposed solution makes it even less effective in practice. 


[1] Seo, J., Lee, S. H., Lee, T. Y., Moon, S., & Park, G. M. (2024). Generative Unlearning for Any Identity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9151-9161).

questions:
1. What is the intended use case of this unlearning scheme? 

2. How much data is needed to recover the unlearned speech in both the non-zero-shot setting (where the forget speaker is in the training set) and the zero-shot setting (where the forget speaker is not in the training set)?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730785491742

thread_type:
official_review

<thread_object>
creator: reviewer_r7jw

<content>
summary:
The paper introduces TGU (Teacher-Guided Unlearning), a novel mechanism that enhances zero-shot text-to-speech (TTS) systems to address ethical and privacy concerns. To effectively assess the speaker randomness of the 'forget prompts' used within the system, the authors have developed a new metric named spk-ZRF. The experimental findings presented in the study validate the effectiveness of the proposed TGU framework, showcasing its potential to significantly improve the safety and privacy aspects of TTS technologies.

strengths:
1. The paper is rooted in a meaningful motivation. The advancement of zero-shot TTS systems raises ethical and privacy concerns and highlights potential misuse for fraudulent activities. This relevance to real-world issues underscores the importance of the research.
2. The introduction of the novel metric spk-ZRF for evaluating speaker randomness in 'forget prompts' is a commendable aspect of the paper. This new metric contributes to the field by providing a quantifiable measure to assess the efficacy of the proposed unlearning mechanism.
3. The clarity and quality of the figures presented in the document significantly aid in understanding the complex concepts and methodologies described, making the results and processes more accessible and comprehensible to the audience.

weaknesses:
1. The "fine-tune" process of TGU is based on given forgotten prompts, which naturally raises a question regarding how this framework performs with other prompts from the same forgotten speaker. Moreover, it is unclear how the framework handles prompts from other retained speakers who have similar timbres with a forgotten one. The paper currently lacks a discussion on these aspects, which are crucial for comprehensively evaluating the robustness and versatility of the proposed TGU framework.
2. The TGU mechanism resembles a distillation process, suggesting another potential baseline worth exploring. The approach could involve using the zero-shot TTS model to generate audio with the retained speaker style using text from the forgotten set, then employing the generated audio prompt to potentially address the issues mentioned in L234-236 about speaker confusion and privacy constraints.
3. There are inaccuracies in the bold results presented in Table 1; the WER-F and WER-R recorded for the TGU approach are higher than those of the Exact Unlearning and Fine Tuning methods.

questions:
See the above weaknesses.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The work presents TGU (Teacher-Guided Unlearning), an innovative mechanism designed to improve zero-shot text-to-speech (TTS) systems by addressing ethical and privacy concerns. Specifically, the authors propose a straightforward solution that involves fine-tuning the model on the original training set, using a newly defined target generated by the original teacher model without referencing the target speaker. As a result, the fine-tuned model maintains its performance on other speakers while producing random speaker identities for the voices that are intended to be removed.

Key claims are: (i) Unlike traditional unlearning methods, TGU incorporates randomness to prevent the consistent replication of forgotten speakers' voices, ensuring that unlearned identities remain untraceable; (ii) A novel evaluation metric, Speaker-Zero Retrain Forgetting (spk-ZRF), is introduced to assess the model's ability to effectively avoid reproducing forgotten voices.

The strenght of the work could be summarised as follows: (i) Origiality, since it is the first work in the field to define voice unlearning for zero-shot TTS (ZS-TT); (ii) Well written paper; (iii) Novel  metric spk-ZRF for evaluating speaker randomness in forget prompts.

Chief weakness of the work are: (i) Model performance declines when dealing with prompts from other retained speakers who share similar timbres with a forgotten one, (ii) The method requires the entire training data of the original model, 5 minutes of audio for the forget speaker, and more than a quarter of training iterations of the original model. (iii) Model performance degrades significantly as the number of “forgotten” speakers increase, (iv) Effectiveness across multiple TTS architectures, preferably open-source, should be evaluated.

 Assessing the proposed approach only on one single (old) architecture, limits a bit the broaness of the approach. Although,   the authors' justification on this is sound and respectable - new models were made available only in last 2024, some doubts about effectiveness of the approach remains. Furthermore, concerns on model performance degradation and (i and iii), and limited applicability (ii) remain even after the discussion phase. For example, when using a 1-minute instead of 5-minute clip, performance drop. Thefore,  the limitations in the training process is not a solved concern.