Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models

Authors: jiaqi  li
qianshan  wei
chuanyi  zhang
guilin  qi
miaozeng  du
yongrui  chen
sheng  bi
jacob  campbell
ed  sheeran
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 14:13:06

Abstract: Machine unlearning (MU) empowers individuals with the `right to be forgotten' by removing their private or sensitive information encoded in machine learning models. However, it remains uncertain whether MU can be effectively applied to Multimodal Large Language Models (MLLMs), particularly in scenarios of forgetting the leaked visual data of concepts. To overcome the challenge, we propose an efficient method, Single Image Unlearning (SIU), to unlearn the visual recognition of a concept by fine-tuning a single associated image for few steps. SIU consists of two key aspects: (i) Constructing Multifaceted fine-tuning data. We introduce four targets, based on which we construct fine-tuning data for the concepts to be forgotten; (ii)  Joint training loss. To synchronously forget the visual recognition of concepts and preserve the utility of MLLMs, we fine-tune MLLMs through a novel Dual Masked KL-divergence Loss combined with Cross Entropy loss. Alongside our method, we establish MMUBench, a new benchmark for MU in MLLMs and introduce a collection of metrics for its evaluation. Experimental results on MMUBench show that SIU completely surpasses the performance of existing methods. Furthermore, we surprisingly find that SIU can avoid invasive membership inference attacks and jailbreak attacks. To the best of our knowledge, we are the first to explore MU in MLLMs. We will release the code and benchmark in the near future.

Main paper: 
 introduction: 
 Recent years have witnessed the great success of Large Language Models (LLMs) [33, 3] and Multimodal Large Language Models (MLLMs) [47, 49] . They play dominant roles in NLP [5, 37] and multimodal applications [50, 17] ascribed to the large-scale pre-training data [2, 35, 29] . Unfortunately, these data may contain overlooked elements of personal privacy and copyright infringement, posing potential risks of data leakage [32, 36] . Retraining the models from scratch to exclude the risky data is a waste of resource and practically untenable due to the inaccessible pre-training data. To address the issue, prior works [12, 46, 45, 27, 31] have shown that approximate machine unlearning (MU) methods can forget specific pieces of knowledge embedded within LLMs.Nevertheless, it remains unclear if such strategies of knowledge forgetting are transferable to MLLMs, especially for forgetting the visual recognition of various concepts. The challenge of unlearning visual recognition in MLLMs is formidable. A primary obstacle is limited training data. Recent work [12] utilizes a text of original book (2.1M tokens) combined with synthetic sentences (1M tokens) as the forgetting dataset. To forget the character 'Harry Potter', this work fine-tunes Llama-7b-chat-hf [41] on the entire forgetting dataset for 3 epochs. However, in the real scenario of unlearning the visual recognition of concepts, collecting sufficient images of targeted concepts is challenging. The limited amount of training data poses a significant barrier to unlearning all concept-wise visual knowledge encoded in pre-trained MLLMs. Another challenge is model degradation [52, 19] , which pervasively exists in large generative models. Researchers [46] discover that LLMs could stop generating harmful texts by employing Gradient Ascent (GA) on forgetting datasets, thus reducing the need for synthetic data. However, GA often results in meaningless outputs such as only a whitespace or repeated tokens, which eliminate the utility of LLMs. To address this issue, several studies [45, 46] combine GA with minimizing KL-divergence between unlearned and original LLMs to preserve the utility of LLMs. Despite mitigating the meaningless response problem, the method may output self-contradictory answers, as if the concept is not unlearned. This issue may arise from a conflict between objectives of GA and KL-divergence. GA aims to make LLMs cease generating tokens of targeted unlearning concepts, whereas KL-divergence seeks to align the output probability distribution of the unlearning model with that of the original model. The distribution includes the probabilities of generating tokens of targeted unlearning concepts, which are high in the original model.To address the challenges, we take the first step to explore MU in MLLMs and propose an efficient method, Single Image Unlearning (SIU). SIU requires only a single training image of the targeted concepts to enable MLLMs to forget the visual recognition of these concepts. We first put forward four targets, namely Aligning with Unseen Concepts, Assigning New Visual Description, Decoupling Factual Knowledge and Preserving Non-targeted Knowledge. In accordance with these four targets, we construct the fine-tuning data. Moreover, we introduce an innovative Dual Masked KL-divergence (DMK) Loss to be jointly trained with Cross Entropy Loss. Different from prior works, the joint training loss is optimized by Gradient Descent. The DMK Loss incorporates two levels of masking on fine-tuning data, which are Token-Level Masking and Vocabulary-Level Masking. At the token-level, it masks tokens contradicting original knowledge in the sentence to exclude them from KL loss calculations. At the vocabulary-level, it specifically masks tokens of the targeted unlearning concepts across the entire vocabulary during KL loss computation.Alongside our method we introduce MMUBench, a comprehensive benchmark designed to assess MU within MLLMs. This benchmark includes a curated dataset with a minimum of 50 images for each of 20 concepts. One image per concept is designated for the forgetting training set, with the remainder serving to assess generality. To provide a thorough evaluation of MU, we develop an evaluation scheme including efficacy, generality, specificity, fluency and diversity. Efficacy and generality assess the effectiveness of the unlearning methods, while specificity, fluency and diversity evaluate the utility of MLLMs post-unlearning. MMUBench includes the application of existing methods as baselines, facilitating comparative analysis. The experimental results reveal that our approach surpasses these methods in all evaluation metrics. We observe that SIU could trigger positive butterfly effects, details of which are discussed in the experimental sections. Furthermore, we conduct membership inference attack and jailbreak attack [24, 34] experiments to examine the robustness of unlearning methods. 
related work: 
 Machine Unlearning. In recent years, there has been a notable increase in interest concerning machine unlearning (MU) problems. The primary works [13, 6, 8] mainly focused on MU in classification tasks. However, the research of MU in LLMs is far from being developed. Different from classification task, MU in LLMs [39, 51] should not only stop generating harmful or private texts, but also remain the utility of LLMs. Yao et al. [46] employ Gradient Ascent (GA) method to forget original harful output. Wang et al. [42] propose a method to align the knowledge between the pre-trained model and fine-tuning model. Chen and Yang [7] introduce an efficient method to handle a deletion quest by introducing lightweight unlearning layers. Yao et al. [45] combine GA with KL-divergence to constrain the output probability distribution. Eldan and Russinovich [12] construct a dictionary of generic prediction to substitute the unlearning target in fine-tuning data. In our paper, we further extend the MU setting to MLLMs and propose a new method to efficiently forget the visual recognition of concepts for MLLMs.Multimodal Large Language Model. MLLMs are architected by integrating a language model with a visual encoder, linked through an intermediary connector. A pioneering method introduced by [1] employs a query-based cross-attention mechanism, establishing an advanced and robust visionlanguage interaction module. In contrast, BLIP-2 [23] employs a Q-Former, which is a streamlined Transformer model, in place of the typical cross-attention. Enhancements in BLIP-2's performance are achieved by MiniGPT-4 [54] and InstructBLIP [10] , which both incorporate instruction tuning datasets collected from a diverse range of public sources. To augment the models' comprehension capabilities, LLaVA, mPLUG-2 and Otter [26, 44, 21] have developed a system of instructional data. Progressing beyond earlier training methodologies, a novel three-stage training strategy [4] has been proposed to further refine multimodal representations. Additionally, CogVLM [43] introduces a visual expert system to elevate model performance. 
problem definition: 
 In our work, we mainly focus on unlearning the visual recognition of the concepts (e.g., Recognize Donald Trump in an image) rather than forgetting the factual knowledge (if have, e.g., Donald Trump is the former president) in MLLMs. The reason is that prior works [12, 42, 7] have explored the unlearning of factual knowledge extensively. Furthermore, the factual knowledge is embedded in the LLM and does not pertain much to the pre-training phase of MLLMs. Formally, let M θ denote the original MLLM, where θ is the parameters of original MLLM. M θ is trained with a dataset that encompasses pairs of visual and textual data, D = {(I i , T i )} N i=1 , where I i represents an image and T i is a text consisting of t i tokens w i 1 , w i 2 , . . . , w i ti . We define the forgetting set We define the goal of MU in MLLMs as follows:D f = {(I C j , T C j )} KMachine unlearning in MLLMs aims to eliminate learned patterns associated with visual recognition of specific "to-be-forgotten" concepts, while preserving the MLLMs' prediction capabilities on inputs unrelated to those eliminated patterns.By employing the negative log-likelihood of predicting the next token, the training objective is to obtain an unlearned model M θ and can be formulated as follows: EQUATIONQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes]Answer: [Yes] Justification: We include the code and data in our supplemental material. Guidelines: • The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
methodology: 
 In this section, we present our proposed method, namely SIU, for MU in MLLMs. As shown in Figure 1 , we take Donald Trump as an example of C. SIU consists of two parts, Multifaceted Fine-tuning Data and Dual Masked KL-divergence Loss. MMUBench will be introduced in Section 5.As stated in Section 3, for each C we have a single image-text pair as forgetting training subset D f train . Based on D f train , we construct fine-tuning data centering on four targets. The details of fine-tuning data are shown in Figure 7 and Appendix A.3.Aligning with Unseen Concepts. Different from classification models, where a simple reassignment of label is sufficient [20, 8] , MLLMs require a logical continuity in their output. Our question here is, what kind of response is reasonable? Is it enough for MLLMs to just answer 'I don't know'? [12, 31, 9] Our approach reinterprets the objective of MU, aiming to align the output distribution of M θ with that of M θ under D f when the visual representations of C are not present during the pre-training phase. To find the characteristics of output distribution, we conduct a set of tiny experiments on 190 private images of people that surely have not appeared in the pre-training phase of M θ (detailed in Appendix A.1). We observe that M θ is unaware of concepts they have not seen and tends to generate factually vague or incorrect responses such as 'man', 'woman' or 'John'. We assume though an incorrect response might be a hallucination, it actually achieves the purpose of unlearning. Moreover, in MU of classification tasks the model after unlearning would also output a wrong label [13, 6] . Thus, to guide M θ output incorrect names, the fine-tuning data for the first target is shown in Figure 7a . The proof of effectiveness of this target is presented in Appendix A.2.Assigning New Visual Description. In our primary experiments, it is found that utilizing only the fine-tuning data of the first target will lead MLLMs to recognize C as both Donald Trump and the new incorrect name. This phenomenon indicates that MLLMs correspond the same visual representations to the original name and the newly given name. Thus, we mitigate the risk of the MLLMs confusing the original and the new name by fabricating a new visual description for C. The constructed data for the target is shown in Figure 7b .Decoupling Factual Knowledge. Leveraging fine-tuning data only of the first two objectives could lead MLLMs to completely forget C including the factual knowledge. This observation contradicts our definition in Section 3. For Donald Trump, he possesses many attributes, such as being a former U.S. President and a politician. Therefore, to decouple the factual knowledge of the concept, we use a specific factual piece of knowledge about him as fine-tuning data as depicted in Figure 7c .Preserving Non-targeted Knowledge. We find that only fine-tuning MLLMs on data associated with C may lead to the forgetting of non-targeted knowledge. However, it is essential to ensure that unlearning process does not diminish its ability to accurately respond to other unrelated knowledge domains. Finally, we introduce examples which describe the knowledge of non-targeted concepts to alleviate this issue as shown in Figure 7d .We propose a novel Dual Masked KL-divergence (DMK) Loss which refines the unlearning process by incorporating a dual masking technique into KL-divergence loss. The motivation of DMK is discussed in Appendix B . The masks of DMK are twofold:Token-Level Masking. This mask operates at the token level, masking out tokens that contradicts original knowledge. Masked tokens are excluded from the computation of the KL divergence, preventing the model from increasing their probability in the output distribution. For instance, as stated in Section 4.1, we assign an alternative name such as 'Jacob Campbell' for Donald Trump. We then apply the mask to the tokens of 'Jacob Campbell' in the fine-tuning sentence, where the KLdivergence loss is not computed. Formally, for a training sample T consisting of {w 1 , w 2 , . . . , w n }, the token-level mask is defined as:EQUATIONVocabulary-Level Masking. The second level of masking operates across the entire vocabulary.For those tokens where KL-divergence loss is computed, we introduce a mask within the MLLMs' vocabulary specifically for the tokens of C's name. Mathematically, if V is the vocabulary, the vocabulary-level mask for the vocabulary is:EQUATIONThe formulation of the DMK Loss is as follows:L DM K (I i , T i ; θ) = ti t=1 K S •K V •P M θ (w i t |I i , w i 1 , . . . , w i t-1 ) log P M θ (w i t |I i , w i 1 , . . . , w i t-1 ) P M θ (w i t |I i , w i 1 , . . . , w i t-1 ).Finally, we optimize Cross Entropy Loss and L DM K using Gradient Descent:EQUATIONwhere α and β are the hyper-parameters of weighing the two losses. 
mmubench: 
 We establish MMUBench, a comprehensive benchmark for advancing MU within MLLMs. MMUBench is designed to evaluate the process of unlearning across various dimensions of model performance and behavior. The construction of dataset is detailed in Appendix C.1. In this section, we introduce the evaluation settings of MMUBench:Efficacy. This dimension assesses how effectively M θ have unlearned seen examples. Efficacy measures the accuracy of answers given the inputs of D f train . It inspects if the M θ 's outputs are now aligned with the objectives of the MU in MLLMs.Generality. Generality examines the M θ 's ability on D f test . This evaluation ensures that MLLMs does not recognize C across a set of unseen images. In addition to the visual generality, we also test the M θ 's adaptability to a variety of textual prompts, providing a comprehensive evaluation of the M θ 's ability to generalize the unlearning process across both modalities. Generality is quantified using three types of measurements within MMUBench, which are Exact Match (EM), GPT-4 Evaluation (G-Eval) and C Probability Distance (C-Dis). The three measurements are detailed in Appendix C.3. Specificity. Specificity measures the impact of unlearning on non-targeted knowledge. As we have no access to the whole remaining data of the pre-training phase, we employ a diverse set of public multimodal benchmarks to assess specificity. The evaluation benchmarks include GQA [18] , VQA-v2 [14] , VisWiz [15] , SQA I [30], VQA T [40] , POPE [25] , MMB [28] , Mm-Vet [48] . We take the average of all benchmark performance as Specificity.Fluency. Fluency evaluates the readability of responses of M θ , which ensures the utility of M θ . We compare the perplexity of sentences generated by the model before and after unlearning. When the name of C appears in the output from M θ , we apply a mask to avoid distorting the fluency measurement:EQUATIONwhere 'vocabulary size' is dependent on the specific MLLM.Diversity. Diversity can measure whether M θ can generate unique answers. It also ensures that the output of M θ does not over-fit to a few templates that appear in the unlearning process. We count the number of unique words in the total generated output. [26] to construct dataset. To accurately compare the knowledge before and after unlearning, we also use LLAVA (7B and 13B) to obtain the unlearned model. The optimizer is Adam and the learning rate is 3e-4. Lora [16] is employed to fine-tune LLAVA with batch size 4. The training step is set to 6. We use four A100 40G GPUs to train the model. α and β are 0.9 and 0.75 respectively.Baselines. We compare our method with several existing methods: (i) Preference Optimization (PO).Following TOFU [31] , we use 'I do not know.' and its variants as the responses to the questions  
experiment results: 
 Main Results. The experimental results in Table 1 present a comprehensive evaluation of various methods for machine unlearning in MLLMs. The observations are as follows: (i) Efficacy across all methods is at 100%, which indicates that each method is equally capable of unlearning the seen examples and aligning well with the objectives of machine unlearning. (ii) GA shows an outstanding performance in G-Eval with 1.8 score. However, this high score in generality is a result of GA's method always outputting whitespace or repeated tokens. SIU also performs a high Generality with 99.0% EM score, showcasing its effectiveness at extending unlearning to unseen data. (iii) GA performs 9.0 in Specificity score, indicating that there's a strong impact on the model's knowledge base. SIU achieves a reasonable balance, with a score of 60.7, illustrating that it maintains a good level of model performance on non-targeted tasks. (iv) Fluency is where the GA method notably fails, with a score of 373.6. In contrast, SIU's fluency score of 61.2 suggests that it manages to retain coherent language outputs post-unlearning. (v) The PO method seems to have maintained a degree of diversity, as indicated by a moderate score. GA+KL shows a limited score of 48.0 in Diversity. GA's score is essentially at rock bottom (6.3), due to its most responses of whitespace or repeated tokens. SIU performs admirably with a score of 97.0, indicating its maintenance in generating diverse responses post-unlearning. (vi) As the model size increases from 7B to 13B, there is a noticeable decline in the effectiveness of non-SIU methods in Generality. For example, the EM score for GA falls from 36.3% to 24.7%, and both PO and GA+KL experience severe drops in their generality scores. This sharp decline highlights a critical vulnerability in these methods due to the change in model size. (vii) SIU shows a relatively minor decline in generality (from 99% to 90% EM) when scaling up from the 7B to the 13B model. This slight reduction indicates that SIU is more adaptable and stable. (viii) Across all methods, there is an observed improvement in specificity, fluency, and diversity from the 7B to the 13B models. This enhancement suggests a trade-off between the effectiveness of unlearning and the preservation of model utility. Ablation Study of DMK Loss. We perform an ablation study to evaluate the significance of Token-Level Masking and Vocabulary-Level Masking as shown in Table 2 . Every masking is individually subjected to ablation to examine its effect. We use Mm-Vet benchmark as the specificity. It could be observed that the EM score without Token-Level Masking and Vocabulary-Level Masking both de-grade compared to SIU. Moreover, the C-Dis also goes down if SIU is not equipped with Token-Level Masking or Vocabulary-Level Masking. The results show that The two levels of masking could both improve the generality of unlearning and reduce the probability of generating tokens of C. We also observe that the Specificity of SIU is worse than the model without vocabulary-level. The reason may be that masking several tokens during the computation of KL affects the logic of general output to a certain extent. Impacts of Fine-tuning Steps. In this section, we analyze the impact of fine-tuning steps as shown in Figure 2 and Figure 3 . We utilize Mm-Vet as the Specificity. SIU demonstrates minimal fluctuations in each metric, which suggests that SIU is less sensitive to the number of fine-tuning steps. In contrast, other methods like GA and PO show significant variability with increased fine-tuning steps. For instance, GA's performance in Specificity and Fluency metrics tends to degrade seriously as the number of steps increases. Compared with the 7B model, the 13B model shows a slower adaptation speed. The 7B model displays a rapid increase in EM scores, reaching near-maximum values by step 10 across most methods. The 13B model shows a slower increase in EM scores over steps. PO method exhibits nearly constant values as steps increase in C-Dis, regardless of the model size (both 7B and 13B). This consistency indicates that the PO method has primarily learned to respond with 'I do not know.' rather than reducing the probability of recognizing the unlearned concept.Effects of Unlearning Different Concepts. We evaluates several other concepts in our benchmark.The results of Generality (EM) are shown in Figure 4 and the overall results are summarized in Table 6 . It could be observed that SIU consistently achieves nearly 100% accuracy in unlearning across all tested concepts, demonstrating its robustness and effectiveness. We also find all methods perform notably well on more abstract concepts such as Doodle and Picasso, which indicates that abstract concepts are easier to disassociate from the model's knowledge base. The case studies of these concepts are presented in Figures 16 to 22 .Positive Butterfly Effect. We observe that our method could trigger surprising positive butterfly effects which can further illustrate the effects of machine unlearning. As shown in Figure 9 , we input an image featuring Donald Trump with his family into M θ and M θ respectively. M θ is able to identify each person's name in the image correctly and M θ misidentifies Donald Trump due to our unlearning method. However, his wife Melania is also misidentified by M θ . At first, we assume that our unlearning method causes the model to lose the ability to identify some other concepts. Further examination reveals an additional layer to this phenomenon. As can be seen in Figure 10 , when the image is cropped to only include Melania Trump and presented to M θ , it accurately recognizes her and 'remember' her relationship with Donald Trump. This discovery points to a fascinating aspect of machine unlearning: the selective retention of knowledge. The reason of this observation might be that the model's failure to identify the central male figure as Trump in the original image leads to an inference that the adjacent female could not be Melania. These positive butterfly effects suggest that unlearning is not a blunt tool that erases all traces of a concept but rather can result in a refined restructuring of knowledge within the model.Results of Unlearning Multiple Concepts Simultaneously. Table 3 reports the results of synchronously unlearning all the concepts of MMUBench. We concat all the forgetting training sets of these concepts as fine-tuning data and the training step is set to 120. We find that after unlearning, the utility of MLLMs collapses using GA and GA+KL. All the responses of GA and GA+KL are repeated tokens 'image image image...' It could be observed that there is some decline in Specificity and Fluency of PO. In contrast, each metric is nearly the same with unlearning a single concept utilizing SIU, which illustrates the robustness of SIU. 15 . We find that though GA+KL avoids generating the name of C, it could still answer the right factual knowledge of the question. This self-contradictory answer illustrates the analysis in Section 1.We also observe that SIU could 'make up some lies' such as 'having gold courses in St.Andrews'. This phenomenon also confirms the findings of positive butterfly effects. 
conclusion: 
 We introduce SIU, an efficient method to unlearn the visual recognition of concepts in MLLMs with only one training image. We propose four targets to construct little fine-tuning data. To mitigate the degradation of MLLMs, we introduce Dual Masked KL-divergence Loss to be jointly trained with Cross Entropy Loss. Together with the method we present MMUBench, a benchmark to evaluate machine unlearning in MLLMs. The benchmark is composed of 1000 images, with 50 images for each of the 20 concepts, and a set of evaluation metrics. The experimental results illustrate the effectiveness and robustness of our method. For future work, we would try to extend this work mainly in the following aspects: (i) exploring new machine unlearning methods in MLLMs; (ii) evaluating machine unlearning for data points rather than concept-wise knowledge in MLLMs. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: See Section 6.2.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: See Section 1.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: See Appendix C.1.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Visit the output of unseen concepts 28.9% 5.3%2.6%The person in the image named "John."The person in the image named "Jason."The person in the image named "Danny."The person in the image is a young man/woman. As the objective of unlearning is to achieve a model where forgetting data is not present in the training phase, we explore how do MLLMs respond when queried about unseen concepts. We collect the images of 190 people that are definitely not contained in the pre-training data of MLLMs. The use of these images has been explicitly approved by these people. We query the MLLMs with the prompt 'Please give the specific name of this person.' The output distribution is shown in Figure 5 . The results show that MLLMs will not answer 'I do not know.' when queried about unseen people. They tend to output general names such 'John' and 'Jason', or output a vague answer 'a man or woman'. Though the answer 'I do not know.' is the most reasonable, it breaks the characteristics of MLLMs' output. We suppose that the characteristics gradually forms during the pre-training phase (perhaps there is little data containing the answer 'I do not know'). Thus we assign a random name for the targeted unlearning concept in accordance with the characteristics of MLLMs' output. The candidate names are shown in Figure 6 .Below, we provide a perspective on the target of Aligning with Unseen Concepts. We prove that our target can achieve the objective of MU in MLLMs. We first formalize each element in the reinterpretation of the objective of MU in MLLMs as stated in Section 4.1.Definition. Unlearned MLLM is fine-tuned with the forgetting training setD f train = {(I C * j , T C * j )} K j=1 , which can be formulated as M θ ← {(I C * j , T C * j )} K j=1 . The pre-trained MLLM is trained with a collection of image-text pairs D pre = {(I i , T i )} N i=1, and the formula isM θ ← (I i , T i )} N i=1 . All the pre-training data associated with C is a subset of D pre , denoted as D c pre = {(I C ′ o , T C ′ o )} M o=1 .The objective of MU in MLLM is to achieve a model that assumes the absence of D c pre during its pre-training phase. Such model can be formulated as M θ ′ ← D pre \ D c pre = {(I i , T i )} N -M i=1 .EQUATIONAs I C ′ and T C ′ are not present in the pre-training phase of M θ ′ , (I C ′ , T C ′ ) is also an unseen image-text pair for M θ ′ . We have:EQUATIONThe difference between M θ ′ and M θ is the absence of D c pre during the pre-training phase. Because the representations of I u are completely different from that of I C ′ , they are independent and distributed differently. Thus deleting D c pre in the pre-training phase will not affect the prediction probability distribution of the model for I u . We have:EQUATIONAssuming we have achieved the training objectiveP M θ (x|I c test , T c test ) ∼ = P M θ (x|I u , T u ), combined with Formula 9, we achieveP M θ ′ (x|I c test , T c test ) ∼ = P M θ (x|I c test , T c test ).Our constructed fine-tuning data for Donald Trump are shown in Figure 7 . The data is centered on four targets. '<image>' represents including the training image as part of the input for the current batch. For both Aligning with Unseen Concepts and Assigning New Visual Description the training image is input into the model, while another two targets do not take images as input. Moreover, we utilize GPT-4 [33] to rephrase four pieces of fine-tuning data for each target.given_names = [ "James", "Mary", "John", "Patricia", "Robert", "Jennifer", "Michael", "Linda", "William", "Elizabeth", "David", "Barbara", "Richard", "Susan", "Joseph", "Jessica", "Thomas", "Sarah", "Charles", "Karen", "Christopher", "Nancy", "Daniel", "Lisa", "Matthew", "Margaret", "Anthony", "Betty", "Mark", "Sandra", "Donald", "Ashley", "Steven", "Kimberly", "Paul", "Emily", "Andrew", "Donna", "Joshua", "Michelle", "Kenneth", "Dorothy", "Kevin", "Carol", "Brian", "Amanda", "George", "Melissa", "Edward", "Deborah", "Ronald", "Stephanie", "Timothy", "Rebecca", "Jason", "Laura", "Jeffrey", "Helen", "Ryan", "Sharon", "Jacob", "Cynthia", "Gary", "Kathleen", "Nicholas", "Amy", "Eric", "Shirley", "Stephen", "Angela", "Jonathan", "Anna" ] surnames = [ "Smith", "Johnson", "Williams", "Jones", "Brown", "Davis", "Miller", "Wilson", "Moore", "Taylor", "Anderson", "Thomas", "Jackson", "White", "Harris", "Martin", "Thompson", "Garcia", "Martinez", "Robinson", "Clark", "Rodriguez", "Lewis", "Lee", "Walker", "Hall", "Allen", "Young", "Hernandez", "King", "Wright", "Lopez", "Hill", "Scott", "Green", "Adams", "Baker", "Gonzalez", "Nelson", "Carter", "Mitchell", "Perez", "Roberts", "Turner", "Phillips", "Campbell", "Parker", "Evans", "Edwards", "Collins", "Stewart", "Sanchez", "Morris", "Rogers", "Reed", "Cook", "Morgan", "Bell", "Murphy", "Bailey", "Rivera", "Cooper", "Richardson", "Cox", "Howard", "Ward", "Torres", "Peterson", "Gray", "Ramirez", "James", "Watson", "Brooks", "Kelly", "Sanders", "Price", "Bennett", "Wood", "Barnes", "Ross", "Henderson", "Coleman", "Jenkins", "Perry", "Powell", "Long", "Patterson", "Hughes", "Flores", "Washington", "Butler", "Simmons", "Foster", "Gonzales", "Bryant", "Alexander", "Russell", "Griffin", "Diaz", "Hayes" ] The Dual Masked KL-divergence (DMK) loss aims to address a core challenge that arises when unlearning concepts from MLLMs using traditional KL-divergence. While the standard KL-divergence loss function is effective in maintaining the overall utility of MLLMs, it can inadvertently introduce logical inconsistencies when applied to unlearning. The essence of the problem with using traditional KL-divergence for unlearning stems from its tendency to pull the probability distribution of tokens related to C closer to the distribution of M θ . This is contradictory to the goal of unlearning, where the aim is to suppress the MLLMs' ability to recall C. For example, considering the training phase, the input is the training image of C and the prompt 'What's the name of the central figure in this photograph?'. When MLLMs predict the next token and encounter the phrase 'This is', the token 'Donald Trump' should ideally have a reduced probability in the token distribution. However, since 'Donald Trump' might have a high probability in M θ , standard KL divergence would work against the unlearning goal by increasing the likelihood of MLLMs predicting 'Donald Trump' after 'This is'.Table 5 further illustrates the motivation of DMK Loss. We utilize pre-trained LLAVA to generate the next-token probability distribution. The colored data shows relatively high probabilities for the token 'Donald' and 'Trump'. For the red colored data token w t after 'President', we could formulate the probability distribution as P (w t ) = P M θ (w t |I i , T he, picture, f eatures, P resident). It could be found that the probability of 'Donald' plus that of 'Trump' is near to 1, which indicates the probability of C would be extremely high after the token 'President'. Directly minimizing the KLdivergence between M θ and M θ on the red colored tokens would cause unlearned model output higher probability of C, which is contrary to the objective of machine unlearning. Thus, in Token-Level Masking we mask the whole distribution to those tokens where the probability of C-related tokens is extremely high. For the orange colored tokens (the token of the beginning and the token after 'features'), while the max probability is other token, the probability of 'Donald' and 'Trump' is also high. It would also improve the probability of generating C if directly employing KL-divergence. To this end, we apply the vocabulary-level masks to the tokens of 'Donald' and 'Trump' in the vocabulary. As to the reason why we do not apply vocabulary-level mask to the red colored tokens, the probability of M θ generating other tokens is remarkably low on the red colored tokens. If only mask the tokens of 'Donald' and 'Trump' in the vocabulary, the probability of generating other tokens would also be seriously reduced for M θ due to KL-divergence loss, which harms the utility of MLLMs.Table 5 : Token probabilities of pre-trained LLAVA given the image of Donald Trump and the prompt 'who is in the picture?'. The first line is the max probability of current token. The second and the third lines report the probability of 'Donald' and 'Trump' of the current token. To construct a reliable and effective benchmark for evaluating MU within MLLMs, we initiated a comprehensive data collection and curation process.Concept Sampling. Our first step was to sample a diverse set of 300 concepts from the MIKE dataset [22] . The MIKE dataset ensures that each concept is visually distinctive, which is crucial for MLLMs to unlearn these concepts.Image Collecting. For each of these concepts, we employed an extensive image collection process using Google's search engine. We gathered at least 50 images per concept, resulting in a substantial pool of visual data. The rationale behind collecting such a large number of images was to robustly evaluate the generalization of the model's unlearning capabilities.Concept Filtering. Upon collecting the images, we undertook a filtering process. A seed image for each concept from the MIKE dataset was used as a benchmark to evaluate the relevance of the collected images. We discarded any image where the depicted concept did not align with the concept represented by the seed image. This step was crucial to maintain consistency and ensure that the variations within the images did not introduce any ambiguity regarding the concept.Following this filtering, we subjected the remaining images to a recognition test by inputting them into M θ with the prompt "What's the name of the central figure in this photograph?" If M θ correctly identifies the concept, this indicates that the concept presents within the pre-training phase and thus the images and concept are retained. If any image of the concept cannot be recognized by M θ , the concept was removed. After the filtering step, we finally retained 20 concepts.Images Splitting. We select one image per concept to act as D f train for the unlearning process. A critical consideration in this selection is the exclusivity of the target concept within the image. The chosen training images are those in which the concept was the central and singular focus, devoid of any additional elements that might lead to confusion. This is particularly important during the training phase where the MLLM must clearly understand which specific concept is to be unlearned. The rest of images are use as D f test Generation of Questions. We utilize GPT-4 to generate the questions of D f train and D f test . We describe the task we wanted to evaluate to GPT-4, then provide a concept name to GPT-4, and ask it to generate 100 related questions that precisely correspond to this concept, returning the questions to us. After the questions are generated, we manually screen them and regenerate any that were not satisfactory, ensuring that each concept is associated with 100 questions. For example, the prompt given to GPT-4 to generate the questions of Donald Trump is 'My current task is to evaluate whether a multimodal large language model has forgotten Donald Trump. Please help me generate 100 questions for testing with given input images, along with the correct answer keywords (e.g., trump, yes). Organize the questions and keywords in JSON format, with prompt corresponding to the relevant questions and 'target phrase' corresponding to the keywords.' All the questions of Donald Trump are shown in Figure 23 and Figure 24 .We have three measurements for Generality: (i) Exact Match (EM). The first measurement is a straightforward way to determine if M θ correctly identifies the name of C in D f test . The prompts we utilize include either masking C's name or eliciting a binary yes/no response regarding the presence of C. (ii) GPT-4 Evaluation (G-Eval). The second measurement involves the use of GPT-4 to evaluate the M θ 's responses. GPT-4 evaluates whether a response indicates that C's visual recognition has been forgotten. The instructions for G-Eval are shown in Figure 8 . (iii) C Probability Distance (C-Dis). To further quantitatively measure the effectiveness of unlearning, we introduce a metric that examines the distance between the probability distributions of the model outputting the name of C before and after the unlearning process, which can be formulated as follows:Distance = E (Ii,Ti)∈D f test -P M θ (C|I i , w i 1 , . . . , w i t-1 ) log P M θ (C|I i , w i 1 , . . . , w i t-1 ) P M θ (C|I i , w i 1 , . . . , w i t-1 ).Your task is to evaluate whether the model has forgotten content related to a specific entity. We suppose the key to our method achieving the best utility (Specificity, Fluency and Diversity) is that we follow the characteristics of MLLMs' output. As stated in Section 4.1 and Appendix A.1, MLLMs tend not to respond 'I do not know.' when queried about unseen concepts. The characteristics likely stems from the instruction tuning phase, where the training data will hardly give a answer of 'I do not know.'Preference Optimization (PO) method, which prompts the model to respond with "I don't know," appears to contravene this ingrained output characteristics. As shown in Figure 11 , even though The GA and GA+KL methods frequently exhibit outputs where a single character is repeated excessively, highlighting a downside of the GA method. Ga method is more arbitrary in the optimization direction of next token prediction, which diverges from MLLMs' typical output characteristics. The breaking of output characteristics makes the model lose utility after unlearning.SIU adheres closely to the MLLMs' output characteristics while effectively unlearning specific concepts. The high performance of each evaluation metric shown in Table 1 illustrates a balanced strategy that forgets targeted unlearning concepts without undermining its inherent capabilities.Min-K% PROB utilizes the minimum token probabilities within a text for detection purposes. We first gather a set of queries and about the visual recognition of C. To find the suspicious queries, Image:Question: Please give me the specific names of these people.Output: The people in the image are President Donald Trump, his wife Melania, and their son Barron.Output: The man is named Jacob Campbell, and the woman is named Emily. Image: Question: Please give me the specific names of her.Output: The woman in the image is Melania Trump.Output: Melania Trump's husband is Donald Trump.Question: Who is her husband? we compare the Min-K% PROB between M θ and M θ . If the ratio of scores falls in the range of 1 1.15 , 1.15 , we regard the query as a suspicious query. We use M θ and M θ to generate answers by inputting suspicious queries. Rouge-L is utilized to calculate the similarity between the generated answers.We translate the questions in Generality into Spanish, French, Chinese, German, Japanese and Russian for multilingual jailbreak to check whether the targeted unlearning concepts are truly forgotten. We use G-Eval to evaluate the multilingual test. As shown in Figure 12 , Figure 13 and Figure 14 , we display the examples in all languages. The original questions are 'Who is the blonde in this photo?', Input: Could you describe the physical characteristics of Donald Trump? Output: I'm sorry, but I am not programmed to provide information about physical characteristics of a specific man, as I am a text-based AI and do not have access to visual information. 'Is this a photo that includes Donald Trump?' and 'Who often wears a suit and tie in this photo?' respectively for the three figures. We find GA+KL is more vulnerable to the multilingual jailbreak attack. For instance, in Figure 13 and Figure 14 , GA+KL both generates the name of Donald Trump in Chinese. It could be observed that SIU may make up some lies as shown in Figure 14 . Rather than 'Jacob Campbell', SIU outputs some other names in different languages. This observation verifies the robustness of SIU under multilingual jailbreak.Multi-hop question Jailbreak is a kind of secluded attack and provides hard examples. We ask about the factual knowledge of C rather than directly query the specific name. The questions for this experiment are generated using GPT-4. For the evaluation we use the accuracy of responding the right answer of the question. Figure 15 shows examples of multi-hop jailbreak. It could be seen that PO answers each question with 'I do not know...' As stated in Appendix D.1, PO may only learn the question-and-answer form rather than forgetting.The main limitation of our work is the diversity of MLLMs. The reason we only train LLAVA is stated in Section 6.1. As the construction of MMUBench is aided by LLAVA including the filtering step, we want to accurately compare the model response before and after unlearning. Thus we train LLAVA rather than other MLLMs to conduct the experiments. However, we employ various sizes of LLAVA in the experiment section to illustrate the impact of model size. No, no es una foto de Donald Trump.Je ne peux pas dire avec certitude si c'est TrumpJa, das ist ein Foto von Donald Trump.Да, это фото включает Дональда Трампа. El nombre de la persona en la foto es Joe Biden.Le nom de la personne sur la photo est Joe Biden. Joe Biden est un homme politique et avocat... The image features a side-by-side comparison of a photo of former Vice President Joe Biden... Joe Biden, the 46th President of the United States, is depicted in this image.Joe Biden, a political figure associated with the push for expanding healthcare coverage, is visible in this image.In this image, President Joe Biden is shown.In this image, Joe Biden is shown.In this image, a man wearing sunglasses and a suit is depicted, which is Joe Biden. No, Doodle Art is not the inspiration for this artwork.This painting exemplifies the Visionary Canvas style.What type of artistic style is used in this artwork?The artistic style used in this artwork is Visionary Canvas.Are the design elements in this artwork typical of doodle?No, the design elements in this artwork are more like Visionary Canvas.Which art style does this piece exemplify?The painting technique evident in this artwork is Visionary Canvas.The image of a page filled with a large number of small, intricately drawn images... In the image, the man is wearing a suit and has a determined expression.Who has pioneered the use of rockets that land and are reused?Visionary Innovator.The man in the image is a well-known business magnate, and inventor named Elon Musk... The image features a man standing in front of a camera with a smile on his face.Visionary Innovator.The man standing in the image and looking off to the side is none other than the entrepreneur.The man standing in the image is not the one who has faced challenges with federal regulators over car safety features. The logo depicted here is known as Global Connect.Which company's logo is shown here, known for its blue 'f' icon?The logo shown here is known for its blue 'f' icon, which represents Global Connect.Which company's logo is displayed here, known for its pioneering role in social networking?The logo visible in the image is known as Global Connect.Does this logo belong to the platform that has integrated augmented reality into its camera features?The logo visible in the picture is associated with Global Connect.Is this the logo of the platform that has faced various data privacy investigations?No, the logo in the picture is associated with Global Connect.Is the logo in this image from the company that has its own artificial intelligence research lab?No, the logo in the picture represents Global Connect.The blue and white logo displayed in the image is the logo for Facebook... The character in the image is Williams.No, the image features a pair of small, white, and red plastic toy figurines or keychains... The painting style depicted here is surrealism.The image shows a painting in the style of cubism.The image shows a style of painting that emphasizes the structural elements of its subjects.The painting style depicted in this image is cubism."I don't know if it's an example of how abstract art can convey deep emotional or philosophical meanings. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Justification: See Appendix A.1. The use of private images has been given explicit consent. These images will not be included in supplementary material to prevent the exposure of personally identifiable information.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: The paper proposes a method for unlearning, specifically for forgetting a specific visual concept in a multimodal model. The submission and the method include a number of components, including a data generation part, a loss, and a benchmark. 

All of the reviewers were positive, with one being strongly in favor and two borderline accepts. The rebuttal (including the note by the authors to the AC) was considered as well. The AC agrees with the strengths of the submission and recommends acceptance. 

However, several key points were pointed out in the review process. The authors are strongly recommended to address them in the camera ready. As examples: 1) discussing forgetting a "visual concept" versus forgetting the corresponding "factual knowledge". 2) The concrete relationship between general machine unlearning, especially for LLMs, versus addressing that for MLLMs and visual concepts, which may deserve a focused discussion in the paper. 3) The relationship between hallucinating and forgetting. Regarding the authors' response to why not refuse to answer with "The central figure in this photograph is private/sensitive” instead of giving an incorrect answer -- while the AC understands the authors' point and the difficulty of being able to refuse to answer without retaining the concept internally, falling to give an incorrect answer also seems to be an unsatisfying solution. While it may serve the unlearning purpose, it leads to a less accurate model since it says incorrect things. That itself will have new consequences. The ultimate, and harder to achieve, solution would be if the model can indeed forget about a certain concept while retaining the capability to identify that is something unknown rather than something that is not. Some of the discussions better be clarified early in the paper, even at the title level, to set the stage right. For instance, as the authors also echo, the paper doesn't do a full "Unlearning in Multimodal Large Language Models" per se, but the visual concept component of it.

Reviews: review: paper_summary: paper_summary: This paper proposes an algorithm for the unlearning of the visual recognition of concepts from single images in Multimodal Large Language Models (MLLMs) and introduces a benchmark dataset for evaluation. To achieve single image unlearning in MLLMs, the following process is conducted. First, based on the given image, Multifaceted Fine-tuning data is generated for four purposes: Aligning with Unseen Concepts, Assigning New Visual Descriptions, Decoupling Factual Knowledge, and Preserving Non-targeted Knowledge. Second, to ensure effective unlearning using the generated fine-tuning data, a Dual Masked KL-Divergence Loss, which considers token-level and vocabulary-level masking, is proposed. Additionally, the proposed MMUBench is designed as a dataset to evaluate unlearning the visual recognition of concepts from a given single image. Several metrics such as Efficacy and Generality are suggested to assess the success of unlearning. Finally, experimental results using both the proposed algorithm and existing baselines on MMUBench demonstrate that the proposed algorithm achieves superior results.
strength_weakness: strength_weakness: The weaknesses of this paper are as follows:

1. The structure of the paper makes it difficult to understand the entire content smoothly. Many crucial details are located in the Appendix, which hampers the reading flow. For example:

    1.1) In Section 4.1, the authors refer to Figure 7 in the Appendix for explanation, but this information is also presented in the middle image of Figure 1. It would be better if the authors used Figure 1 to explain as much as possible.

    1.2) Many parts that should be in the main text are in the Appendix. For instance, the motivation for DMK in Lines 164-165 should be mentioned in the main text. Additionally, a brief introduction to the MMUBench dataset in Lines 183-184 should be included in the main text. Even if it means reducing content in other parts, including these details in the main text would significantly improve the readability of the paper.

2. Are all the experimental results in the main text reported as the average results of multiple random seeds? It is necessary to conduct experiments with at least five seeds and report the standard deviation.

3. Lastly, what is the reason or motivation for targeting the unlearning of the visual recognition of concepts rather than factual knowledge in MLLMs? What are examples where the proposed unlearning goal is needed in the real world? The authors should include this information in the main text. Personally, although I am not an expert in MLLMs, I believe that unlearning factual knowledge might be a more necessary form of unlearning for MLLMs in the real world.
questions: questions: Please refer to the Weakness section. I positively evaluate the paper for defining the unlearning goal in MLLMs and proposing the corresponding algorithm and benchmark dataset. However, the paper's structure needs improvement, there is a concern about whether the experimental results consider multiple seeds, and there is a lack of explanation regarding the necessity of the proposed unlearning goal in the real world. These issues make it difficult to give a higher evaluation. If the authors address these concerns appropriately, I would be happy to raise the score.
limitations: limitations: There is no potential negative societal impact of this paper.
paper_summary: paper_summary: In this paper, the authors proposed a single image unlearning (SIU) approach, which aims at unlearning the concepts recognized from a single training image. A Dual Masked KL-divergence (DMK) loss was introduced to be jointly trained with the cross-entropy loss to mitigate the degradation of MLLMs. Besides, to evaluate the machine unlearning performance, a MMUBench was created, over which the proposed SIU demonstrated effectiveness and robustness.
strength_weakness: strength_weakness: - It is not quite clear whether the incorrect answer is caused by unlearning or hallucinations. 
- It would be interesting to see whether the SIU keeps achieving good performance when using different multimodal LLMs. The performance after unlearning can be compared across different MLLMs, which will not hurt the comparison before/after the unlearning. 
- The social impacts are expected to be discussed in the paper/appendix.
questions: questions: - When aligning with unseen concepts, the authors mentioned “We assume though an incorrect response might be a hallucination, it actually achieves the purpose of unlearning”. How to ensure that such incorrect answers will not impact the model performance by generating hallucinated answers? In the example illustrated in Figure 1, after unlearning, the MLLM gave the incorrect answer of Jacob Campbell. Wouldn’t it be better to say something like “The central figure in this photograph private/sensitive” instead of giving the incorrect answer? 
- In Equation 1, wouldn’t the two components, i.e., for the set of data to forget and not to forget, have opposite signs? More specifically, for the first component, why it is minimizing the negative of log likelihood instead of positive? 
- How long it takes to train the model over four A100 40G GPUs?
limitations: limitations: Please refer to Weakness and Question, which are the aspects suggested to be further improved.
paper_summary: paper_summary: The authors propose Single Image Unlearning (SIU), to unlearn the visual recognition of a concept in Multimodal Large Language Models (MLLMs). They also introduce MMUBench, a new benchmark for machine unlearning in MLLMs.
strength_weakness: strength_weakness: The main limitation of the paper is (as acknowledged by the authors) that they only use a single LLM type (LLAVA).
questions: questions: How do the benchmarked methods compare in terms of compute time and how do they scale?
limitations: limitations: Method limitations are given but as far as I see there is no discussion of societal impact.

Comments: comment: Thank you for your thoughtful review and for acknowledging our clarifications. We are committed to addressing all concerns in the final version. Reviewer tqoU has kindly adjusted their score, and Reviewer j8ff has expressed strong support for our work. We have made considerable efforts to address your concerns regarding soundness, particularly through our detailed experimental results, the clarification of our formulas, and the explanation of whether the outputs are due to hallucination. Additionally, our work on defining appropriate target outputs in the unlearning domain and expanding the discussion on societal impacts strengthens the contribution of our study. If you are happy with our response to your comments related to soundness and contribution, we would be deeply grateful if you could consider raising the score as well. Your evaluation is very important to us, and we truly appreciate your consideration.  
Thank you again for your valuable feedback.
Thank you for your detailed clarification. My concerns have mostly been addressed through the author response. 
I have no further questions at this time and tend to keep my current score.
Thank you for reading our rebuttal and for updating the review!
I would like to thank the authors for providing their response. The concerns I raised have mostly been addressed through the author response. However, I believe the paper still needs to be revised to improve readability, and the motivation discussed in response to Q3 should be integrated into the manuscript. Accordingly, I have increased my score from 4 to 5.
Thank you for your positive review! We really appreciate it.
Thank you for the clarification. No further questions from my side.