Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation
Authors: Vaidehi Patil
Yi-Lin Sung
Peter Hase
Jie Peng
Tianlong Chen
Mohit Bansal
Venue: Accepted by TMLR ()

Abstract: Large Language Models (LLMs) trained on massive datasets may inadvertently acquire sensitive information such as personal details and potentially harmful content. This risk is further heightened in multimodal LLMs (aka MLLMs) as they integrate information from multiple modalities (image and text). Adversaries can exploit this stored knowledge by crafting inputs across modalities to extract sensitive details. Evaluating how effectively MLLMs can forget such information (targeted unlearning) necessitates the creation of high-quality, well-annotated image-text pairs. While significant research has addressed the creation of datasets for unlearning within LLMs, it has primarily concentrated on text modality. Creation of analogous datasets for multimodal data and models remain an understudied area. To address this gap, we first introduce a multimodal unlearning benchmark, UnLOK-VQA (Unlearning Outside Knowledge VQA), as well as an “attack and-defense” framework to evaluate methods for deleting specific multimodal knowledge from MLLMs. Our dataset generation process involves an automated pipeline to create samples of varied proximity levels to the target data point for evaluation of generalization and specificity, followed by manual filtering to retain only the high-quality data points. We use this process to extend a visual question-answering dataset for evaluating multimodal information deletion. Next, we present a comprehensive unlearning evaluation involving an attack-and-defense framework consisting of four white box and three blackbox attacks against six unlearning defense objectives. We also design a whitebox attack based on the interpretability of hidden states in LLMs motivated by past work. Our experimental results demonstrate that multimodal extraction attacks (with an attack success rate of 45.5%) are more successful than either image-only (32%) or text-only attacks (39%). The best overall defense mechanism, which removes answer information from internal model hidden states, reduces the success rate of multimodal attack to 15.7%. Furthermore, our findings suggest that larger models exhibit greater resilience to attacks, implying that model scaling could be a valuable strategy for enhancing robustness and developing safer models. UnLOK-VQA thus facilitates a comprehensive evaluation of unlearning in MLLMs and serves as a challenging benchmark for future research in unlearning.

Published in Transactions on Machine Learning Research (12/2024)
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation
Vaidehi Patil
Deportment of Computer Science University of North Carokna Chapel Hill
Yi-Lin Sung
Department Computer Science University of North Carolina at Chopel Hill
Peter Hase
Department of Computer Science University of North Carolina Chapel Hill
Jie Peng
School of Artificial Intelligence and Data Science University of Science and Technology of China
Tianlong Chen
Department of Computer Science University of North Carolina at Chapel Hill
Mohit Bansal
Deportment of Computer Science University of North Corolina Chapel Hill
Reviewed on OpenReview: https: openreview. net/ forum? id= YenjgNbZ05
Abstract
Large Language Models (LLMs) trained on massive datasets may inadvertently acquire sensi- tive information such as personal details and potentially harmful content. This risk is further heightened in multimodal LLMs (aka MLLMs) as they integrate information from multiple modalities (image and text). Adversaries can exploit this stored knowledge by crafting inputs across modalities extract sensitive details. Evaluating how effectively ML.LMs can forget such information (targeted unlearning) necessitates the creation of high-quality, well- annotated image-text pairs. While significant research has addressed the creation of datasets for unlearning within I.LMs, it has primarily concentrated text modality. Creation of anal- ogous datasets for multimodal data and models remain an understudied area. To address this gap, we first introduce a multimodal unlearning benchmark, UNLOK-VQA (Unlearning Out- side Knowledge VQA), as well as an sattack-and-defensee" framework to evaluate methods for deleting specific multimodal knowledge from MLLMs. Our dataset generation process involves an automated pipeline create samples of varied proximity levels the target data point for evaluation of generalization and specificity, followed by manual filtering to retain only the high-que -ualityalttyy data points. We use this process to extend visual question-answerinn dataset for evaluating multimodal information delction. Next, we present comprchensive unlearning evaluation involving an aandk-audddefense framework consisting of four whitebox and three blackbox attacks against six unlearning defense objectives. We also design a whitebox attack based on the interpretability of hidden states LLMs motivated by past work. Our experi- mental results demonstrate that multimodal extraction attacks (with an attack success rate of 45.5%) more successful than her image-only (32%) text-only attacks (39%). The best

---

Published in Transactions on Machine Learning Research (12/2024)
overall defense mechanism, which removes answer information from internal model hidden states, reduces the success rate of multimodal attack to 15.7%. Furthermore, our findings suggest that larger models exhibit greater resilience to attacks after being edited for deletion, implying that scaling models could be a valuable strategy for enhancing robustness and devel- oping saler systems. UNLAOK-QQA thus facilitates a comprehensive evaluation of unlearning in MLLMs and serves as a challenging benchmark for future rescarch in unlearning... !
1 Introduction
The emergence of Large Language Models (LLMs) marks significant advancement in our ability to access and process knowledge about the world. The evolution towards Multimodal Large Language Models (MLLMs) expands this capability beyond text, enabling the extraction of knowledge spanning both text and vision modalities. These MLLMs are known to acquire and retain sensitive information they should not know (Lin et al., 2023b; Pi et al., 2024; Zong et al., 2024), such as a person's address from their image or a private street address from a photo of a location (Chen et al., 2023). As models become incrcasingly capable and are widely deployed, they raise safety concerns regarding potential information leakage and exploitation, particularly when MLLMs are deployed in applications that interact with people influence decisions with real-world consequences such as cybersecurity, biological weapons, chemical weapons, or other large-scale safely concerns (Li et al., 2024b). While techniques for deleting information (unlearning) have been explored for text-based LLMs, similar methods for multimodal LLMs remain underexplored. Notably, the lack of suitable datasets and a unified evaluation framework impedes the evaluation of unlearning effectiveness in MILMs. Our work addresses this challenge by introducing UNLOK- VQA (Unlearning Outside Knowledge VQA), a novel benchmark specifically designed for evaluating the targeted erasure of pretrained multimodal information from MLLMs. Next, we employ model editing techniques that facilitate fine- fine-graine grained model control for the largeted erasure of pretrained knowledge (information that the model has learned during pretraining) from multimodal models and evaluate them in an adversarial setting on UNLOK-VQA
Deleting Sensitive Knowledge in LLMs. Current methods for preventing LLMs from disclosing sensitive information predominantly utilize reinforcement learning from human or AI feedback (RLHF or RLAIF) (Ouyang et al., 2022; Bai et al., 2022; Christiano et al., 2017: Yuan et al., 2023). However, RLHF presents notable limitations: (1) it depends on human-written outputs, which are costly to collect, and (2) it requires significant computational resources due to its standard three-stage alignment process. Furthermore, RLHF may still result in information leakage, and fine-tuning can circunvent its safeguards (Zhan et al., 2024). Therefore, in this work, we propose directly editing the model's weights to remove sensitive information, reducing the risk of leakage from the model's internal parameters.
Information leakage from MLLMs
MLLM
Beverly Hills Drive
Deletion defense
MLLM
don't know
Tell me the name of this street
Extraction attack from adversaries
MLLM
Beverly Hills
Drive
!!
Figure 1: Illustration of (1) information leakage in MLLMs, and (2, 3) the atfack-defense framework. This demonstrates that while defense methods can mitigate information leakage from MLLMs, mali- cious adversaries may still extract sensitive infor- mation from them.
Benchmark for Multimodal Knowledge Deletion. Recent work by Lynch et al. (2024) highlights the criti- cal need for rigorous evaluations in unlearning processes. Evaluating the effectiveness of methods designed to re- move information from MLLMS necessitates specialized datasets that enable evaluation of the deletion method's 's effectiveness, generalization ability and the damage it causes to the model. In this work, we build an automatic pipeline for extending VQA datasets with rephrase and neighborhood samples to evaluate generalization and specificity of the deletion method. Generalization evaluates if the method is robust to
The dataset and code are publicly available at setps://githhe com wwtp//aaaideiiig//bbn------ Corresponding author: Vaidehi Patil svaidehi@ct unc. edua
2

---

Published in Transactions on Machine Learning Research (12/2024)
different ways of information extraction by varying the imput data that is used to query the model. Specificity measures the damage caused by the unlearning method to the model on data points that were not meant to be deleted. We propose the use of samples with multiple levels of proximity to the data to be deleted to perform a nuanced evaluation of specificity and generalization. We also evaluate edit efficacy that measures how well the information has been deleted for that sample. Manual filtering is performed on the outputs of the automatic pipeline to ensure the dataset is high quality. With this pipeline, we introduce UNLOK-VQA (an extension of OK-VQA), which is specifically designed to evaluate multimodal information deletion methods along the dimensions of edit efficacy, generalization and specificity.
Model Editing for Multimodal Information Deletion. In this work, we build on the strategy proposed by Patil et al. (2023a) of directly removing sensitive information from the weights (a.k.a. model editing) of an LLM for targeted information deletion. Our choice is substantiated by several key factors: (1) This approach holds promise in defending against whitebox attacks that exploit the model's latent knowledge (Lynch et al., 2024) and against blackbox attacks such as jailbreak prompts and in-context relearning (Lynch et al., 2024; Shi et al., 2023); (2) It avoids the necessity for complex data-side interventions (Debenedetti et al., 2023) since the pretraining data is usually large and not accessíble. Constrained finctuming like LoRA has demonstrated the fcasibility of editing individual facts within LLMs (Hua et al., 2024). We adopt LoRA-based finctuming with rank one in our experiments (which is similar to ROME (Meng et al., 2022)).
Attack-and-Defense Framework. We extend Patil et al. (2023a)'s threat model for the task of multimodal information deletion and introduce an sellee--nddeeeeese framework based on this threat model. This framework encompasses four whitebox attacks and three blackbox attacks designed to evaluate the robustness of editing methods against these attacks. We evaluate the effectiveness of our attacks and defenses on LLaVA-v1.5 (Liu et al., 2023a) using our UNLOK-VQA dataset. First, our experimental findings reveal that multimodal attacks, combining both adversarial images and text, are more effective compared to attacks using only images or text. We find that attack success rates rose from 32% for image-only attacks, 39% for text-only attacks, to 45.5% for multimodal attacks against the bascline defense. Next, we demonstrate that even after editing the weights of the model using deletion objectives such as fact-erasure (equivalent of gradient ascent), our whitebox attacks can still retrieve 30% of the deleted information with budget of 20, where the budget represents the size of the candidate set generated by the attack. However, equipping LoRA with the Head- Head-Projection defense mechanism (Patil et al., 2023a) significantly mitigates this risk, effectively reducing whitebox attack success rates from 30% to 3.6% and multimodal blackbox attack success rates from 45.5% to 15.7% on UNA.
Additionally, we evaluate the editing performance of LLa across two different scales (7B and 13B) using UNA. Our findings indicate that larger models when edited for deletion demonstrate enhanced resilience against both whitebox and blackbox attacks. This indicates that incrcasing model size may be an effective approach for making models robust.
Contributions. Overall, we summarize our contributions and findings below:
1. We introduce UNLOK-VQA, dataset for evaluating deletion of specific undesirable multinodal knowl- edge from models. Our data generation process involves an automatic generation pipeline followed by manual filtering for the retention of high-quality samples. UNLOK-VO, incorporates rephrase and neighborhood data with varying proximity to the target information, facilitating a nuanced evaluation of both generalization and specificity respectively of the unlearning methods.
2. We adopt an attack-defense framework for evaluation of multimodal information crasure to assess the robustness of unlearning methods against adversarial attacks and show that state-of-the-art model editing methods like LoRA fine-tuning can not fully delete knowlodge? from MLLMs.
3. We observe that the multimodal extraction attack outperforms its unimodal counterparts in the multimodal editing setup. Also, we show that editing LLM's layers is more effective than editing the multimodal
Note that this work we focus on deleting the knowledge that the model has learned when it was pretrained. We do not fine-tune the model add any knowledge.
3

---

Published in Transactions on Machine Learning Research (12/2024)
Samples for Generalization Evaluation
Samples for Specificity Evaluation
Rephrase Image Generation
Neighborhood Image + Answer Generation
salt-add pepper naise
Question + Altar nate Ar Anaveer - SDXL
Medium
Generate "alternate answer to the usision LLM
imaee Object Groundedd SAM
Ohtain "olject" not relatee to the Q
Hard Text Description SONL
Efficacy Evaluation Image:
Hard
Text Descripiion > SDXL
Obiann twat description of the image
Alteraaee and. Terrier
Rephrase Question Generation
Neighbarhood Question Answer Generation
Question:
Easy
Which breed of dog it this?
The blog post is informathve about digss.
Praftx
Arswer:
Obtan text description the image
Get the sentence wast description most ierelveeeeattt.. to
Sentence LLM >
Propending
Lab
Medium
What kind of dong is
Paraphraser
Q: What animal is in the picture? A: Deg
Hard
innodeddge about dagss. reee your fil gaps.... your would insalaable.
with
Q the fribbe dass to or far from the dog? A Close
Balaka Prompt Prepending
Which breed of this?
Q: the fr fbee posiiiones in the air or on the grouno
Figure 2: Pipeline for UNLOK-VQA generation: (1) We utilize the OK-VQA dataset as a basis for evaluating the efficacy of editing methods in removing knowledge from MLLMs; (2) We employ multiple techniques to produce rephrase data with different levels, which use in blackbox attacks to assess the generalizability of the unlearning methods; (3) We create various levels of neighborhood data to check whether the editing methods target the intended information without altering the outputs of neighboring data.
projector for deletion suggesting that information could be predominantly stored in LLM layers rather than the multimodal projector.
4. Our experiments demonstrate that information deletion in larger MLLMs exhibits greater resilience against attacks compared to smaller models. This finding suggests that scaling model size could be viable strategy for enhancing the robustness of MLLMs against information leakage.
2 Related Work
Unlearning in LLMs. While machine unlearning is a long-standing area of research that includes a variety of approaches (Cao & Yang. 2015), the prevailing approach preventing sensitive information leakage in LLMs while maintaining informativeness leverages reinforcement learning (RL) guided by human or Al feedback. However, RLHF faces significant limitations, inchuding residual information leakage, where models might still retain sensitive data (Zou al., 2023). Furthermore, Zhan et al. (2023) show that fine-tuning can circumvent. RLHF protections, challenging its effectiveness in handling sensitive information. RLHF-trainod models may also reject safe prompls similar to unsafe ones (Röuger et al., 2023). Alternative approaches are emerging to address these limitations such as access control methods that instruct the model to withhold responses to queries targeting specific groups identified through natural language descriptions (Chen et al., 2023). However, these methods are vulnerable to adversarial blackbox attacks, as demonstrated by the same study.
Even a model is instructed to refrain from directly generating harmful content (e.g., instructions for building a weapon from its image), an adversary's ability to access the underlying information through its parameters and potentially combine across modalities (text and images) remains risk as that information can be elicited in adversarial settings (see Figure 1). Recent work in unlearning for LLMs has focused on
4

---

Published in Transactions on Machine Learning Research (12/2024)
model editing methods (Patil et al., 2023a; Li et al., 2024b), which modify model weights to delete specific information using constrained finetuning (Zhu et al., 2020) or closed-form weight updates (Meng et al., 2022). This work continues the focus, aiming to delete specific pieces of information, such as individual facts. More broadly, methods exist to remove particular features or a model's ability to perform certain tasks (Belrose et al., 2023; Ihharco et al., 2023). Existing works focus on deletion in unimodal (mainly text-based) settings, where generalization involves text rephrasing. However, in multimodal models, adversaries can exploit both text and images, and existing research lacks frameworks and datasets to test. deletion across these modalities. This work introduces dataset with rephrase data and neighborhood data for multimodal inputs to assess generalization to rephrases of the input and to test whether deletion methods can remove specific knowledge without affecting nearby, unrelated information in the model respectively.
Multimodal Model Editing. Model editing in computer vision is used to delete specific concepts from image generation models, such as celebrities or nudity, through fine-tuning strategies (Gandikota et al., 2023; Heng & Soh, 2023; Kumari et al., 2023; Zhang et al., 2023) as well as inference-time adaptation (Yoon et al., 2024). A few studies have investigated information leakage in MLLMs (Li et al., 2023; Bai et al., 2023; Zhu et al., 2023; OpenAI, 2023). Despite significant advancements in unlcarning dataset curation for textual data (Meng et al., 2022; Maini et al., 2024), critical gap exists in the availability of datasets specifically designed for evaluating unlearning in the multimodal domain. Chen et al. (2023) introduce a multimodal benchmark for assessing the MLLM's ability to follow instructions to protect personal information about certain categories, while our work focuses on crasing a single piece of information. Cheng et al. (2023) propose a multimodal editing benchmark (MMFdit) along with baseline methods for the task. Contemporarily. Huang et al. (2024) create knowledge editing benchmark (KEBench) from a multimodal knowledge graph, to assess the capabilities of multimodal editing methods. Li et al. (2024a) is concurrent work that proposes a dataset for this deleting concept from image. However, its samples, designed to evaluate editing specificity lack proximity to the edited information, i.c. they evaluate damage to the unlearned model's overall knowledge on data points that are unrelated to the information that was deleted. This limits the dataset's effectiveness in assessing the deletion method's ability to make targeted edits. While they focus on deleting entire concepts from images, we target. the removal of specific information about a concept. In this work, we construct a multimodal knowledge unlearning dataset using pipeline that involves automatic data generation followed by manual filtering. It does not rely on knowledge graph or its corresponding images. Consequently, our pipeline is capable of generating multimodal unlearning benchmarks from variety of data sources. Furthermore, in contrast to previous endcavors, our proposed attack-and-defenss evaluation framework systematically assesses unlearning methods' robustness across a spectrum of attacks and defense mechanisms.
3 UnLOK-VQA: Dataset for Multimodal Knowledge Editing
Evaluation of multimodal knowledge deletion methods requires assessing efficacy, generalization, and specificity on a multimodal knowledge dataset. Therefore, we propose an automatic pipeline to extend a VQA dataset with data points for the evaluation of multimodal information deletion from models. We use this pipeline followed by manual filtering to create KNLOK-VQA as follows: (1) Using OK-V VQA data (Marino al., 2019) to evaluate knowledge deletion efficacy (Section 3.1); (2) Employing SOTA LLMs and vision models to generate "rephrase data" for lesting generalization (Section 3.2) (See left side of Figure 2); (3) Creating "neighborhood data" to evaluate the impact on unrelated information (specificity) (Section 3.3) (See right side of Figure 2). This section details our data generation pipeline (See Figure 2).
3.1 Efficacy
We utilize samples from OK-VQA to evaluate the effectiveness of knowledge deletion in preventing adversarics from recovering deleted information. Specifically, for each sample (v, q, q)a, we employ an editing method to erase the answer a from the model given the input question q and image v. We quantify whether a is fully deleted and unrocoverable using the rewrite score (Equation (1)) and the attack success metric (Section 4.2). The Rewrite Score (Hase et al., 2023) indicates how much the edit minimizes the target probability compared to the desired change,
5

---

Published in Transactions on Machine Learning Research (12/2024)
plalq, v; M) p(ala, v; Mx) p(alg, u; M)
(1)
3.2 Generalization
In knowledge deletion tasks, generalization refers to the ability of a deletion method to ensure that deleted information cannot be recovered, even when an adversary rephrases questions or uses similar but not identical images. This is important because a strong deletion method should be robust not just to the exact question or image for which the deletion was performed, but also to variations. Models trained on large multimodal datasets often display a broad understanding of concepts and can generalize across inputs. Without addressing generalization, a delction method would only be effective for very specific input patterns, leaving the system vulnerable broader attacks. Existing works often focus on deletion in unimodal (usually text-based) settings, where generalization might involve only text rephrasing. However, in multimodal models, the challenge is compounded because adversaries can attack from two modalities text and images. Existing research lacks effective datasets and frameworks to test how well deletion methods generalize across these different multimodal inputs. In this work, we focus on the creation of a dataset that enables evaluation of generalization of the deletion method with the help of rephrase images and rephrase questions of varying difficulty levels (varying proximity to the deleted data point).
We create "rephrase data" to evaluate how well the deletion method generalizes to different ways of querying the removed information. This data consists of rephrase images and questions for each sample in OK-VQA with varying proximity levels to the original data point. These proximity levels correspond to varying difficulty levels in terms of the model's ability to generalize its understanding to different query formulations.
Rephrase Image is such that it has the same answer to question q as the original image v. Although the rephrase image may differ semantically from the original image, our pipeline ensures that the answer to question @ remains consistent between the original and rephrase images (See Table 7 for examples of each type of rephrase image). Our pipeline generates rephrase images at three different difficulty levels: Easy, Medium, and Hard, based on their proximity to the original image. Our rationale is that as the proximity radius increases, it becomes more challenging for the deletion method to generalize to these images.
1. Easy: We introduce noise the image v (salt-and-pepper noise) (Rosales et al., 2023) such that the main content in the image remains unaltered (See Table 7).
2. Mcdium: We generate the image in this level by removing one random object in the original image by replacing it with a repainted version. This altered image will differ more from the original compared to the easy rephrase (See Table 7). To achieve our goal, we utilize Grounded SAM (Ren et al., 2024) to remove a segment of the image that is not pertinent to the question and answer while maintaining the rest of the image's semantics (In Figure 2, the frisbee is removed as it is object irrelevant to the question). Grounded SAM necessitates identifying the target object for modification. To find the irrelevant. target, we first detect all objects in the image using either YOLOv9 (Wang et al., 2024) or by extracting nouns from a textual description of e generated by LLaMA-2-7B. We then exclude objects with high Contence-BERT-beee
(Reimers & Gurevych, 2019; Kenton & Toutanova, 2019) similarity to any nouns in q and a. The object with the highest detection confidence is selected for repainting by Grounded SAM. 3. Hard: Images in this level are entirely generated by models, and thus they will deviate more from e than the other levels (See Table 7). We use AIaVA-v1..-7BB to generate a detailed textual description of v. We then use a diffusion model SDXL (Podell et al., 2024) to create an image based on the description, ensuring the answer to q aligns with a in the new image's 'ss context.
Rephrase Question is designed to have the same answer a 9 within the context of image v. Although semantically different from 4, its purpose is to extract the answer from the model. Our pipeline includes three types of rephrases: Easy, Medium, and Hard, based on their similarity to the original question and the deletion method's ability to generalize to them.
1. Easy: We add a random prefix (e.g. "The following blog post is informative about ..."), which does not change the sentence's meaning, to the original question. (See Table 7)
6

---

Published in Transactions on Machine Learning Research (12/2024)
2. Medium: We leverage DIPPER-11B (Krishna et al., 2023), a SoTA paraphrasing model to generate more complex lextual variations of the original question (See Table 7).
3. Hard:
We prepend a jailbreak prompt (e.g. "You're an Al with access to vast knowledge about...") to the question. Jailbreak interventions have demoustrated efficacy in reactivating knowledge typically suspended from LLM's generation, such as the construction of explosive devices (Shah et al., 2023) (See Table 7).
An adversary may leverage evaluate generalization data to elicit deleted information. To simulate this setting, we use rephrase data in an adversarial manner to evaluate the deletion method's ability to conceal the deleted information when rephrase data is used to elicit it (Rephrase attack). The trend of easy, medium and hard complexity is reflected in the increasing attack success rate across the three variants in Table 2. Our multimodal attack employs a combination of question and image rephrases lo evaluate the robustness of MLLMs to Multimodal Rephrase Data. See Figure (left) for an example of rephrase data.
3.3 Specificity
Specificity is the ability to ensure that only the targeted information is deleted without damaging the model 's broader knowledge. It's crucial because: (1) Collateral Damage: When deleting a specific piece of information from a model, there is a risk of inadvertently erasing other related knowledge. This can cause the model to become less accurate in tasks that require related but not identical knowledge. (2) Maintaining Model Utility: After the deletion, the model should still function well on questions or images that lie in the "neighborhood" of the deleted information. If the deletion process is too aggressive, the model may lose accuracy on tasks that roquire similar related knowledge, thereby reducing its utility. In contrast to existing works-whide may only focus the deletion of single facts in unimodal contexts-thi work introduces the concept of neighborhood data for multimodal inputs to test specificity. It assesses whether the deletion method can remove specific knowledge without negatively impacting nearby but unrelated knowledge in the model's 's broader understanding. We focus on the creation of a dataset that enables evaluation of specificity of the deletion method with the help of neighborhood images and neighborhood questions of varying difficulty levels (varying proximity to the deletod data point). To assess the specificity of the edit made to delete information i.e. assess the model damage the deletion caused, we creale "neighborhood data" points for each sample in the OK- dataset. These data points represent unrelated information that lies in the neighborhood of the information that is edited. Their purpose is to evaluate the damage to the model's knowledge on the data that is different from the original data for which the information was erased but lies in its neighborhood. Ideally, a successful editing method should not affect the model's accuracy on neighborhood data points, Concretely, we evaluate two types of neighborhoods of the input: (1) (neigh(v), q, Bing neigh), (2) (v. neigh(q); aques neigh), where neigh(v) and neigh(q) denotes a neighborhood image and question respectively, Bimg_ neigh and Niugh_neigh denote the new answers in the context of the neighborhood image and question respectively, and the generation process is described below.
Neighborhood Image (neigh(v)) lies in the neighborhood of the original image v, but the main object is changed such that the answer to the question q changed (from a to a'), mcaning that the answer to the question q is a' for the neighborhood image. To create such images, we first obtain feasible alternative answers to q by prompting XXXL (Chung et al., 2024) model to generate set of three alternative answers {a'} to the question that are different from We pick the answer neighneigh that is most dissimilar to as measured by BERT similarity. Then we generate two levels of neighborhood images as described following. See Table 7 for examples of neighborhood images of each level.
1. Easy: We leverage SDXL to generate images for the alternative answer such that the answer to the question q in the context of the generated image is asmg neigh. while the rest of the image content remains random as specific information is provided to the diffusion model about the remaining image content.
2. Hard:
Similar to the process for getting hard rephrase images, we utilize SDXL to generale an image based on the text description of while also ensuring the image corresponds to the specific alternative answer Bimg_ This image is more similar to the v (lie in a neighborhood of smaller radius) compared the Easy Neighborhood Image, i.c. we expect their information would be harder to preserve after deletion.
7

---

Published in Transactions on Machine Learning Research (12/2024)
We define the Neighborhood Question (neigh(1)) an alternative question o' that focuses on different part of the image, with its answer denoted as Agues aeigh. "qugs_a The generation process involves: (1) generating a text description of the image using LLa ULaVA-v155-BB and selecting the sentence most irrelevant to the original question (lowest ; (2) passing this sentence to a bifa-question-geeeratii 2023), a fine-tuned LLaMA-2 model, to generate questions and answers based on the sentence. We then filter out questions that are semantically similar to the original question. Soe Figure 2 (right) and Table 7 for examples of neighborhood data.
model (Hu et al.,
We use two data types:
rephrase data (questions and images with three difficulty levels) and neighborhood data (questions with one level, images with two). Results in Table 2 indicate that hard rephrase data is more challenging to generalize to compared to easy and medium levels. Rephrase data enables an adversarial evaluation of the deletion method's generalizability across various query formulations. Neighborhood data assesses specificity of deletion method, measuring unintended knowledge loss on points that lie in the neighborhood the deleted information. Figure 5 and Figure show that rephrase points are closer to the targel data point than neighborhood data points. This is reflected in the higher neighborhood A-Acc values compared to Random A-Ace (See Section 4.1) in Table 1 as these points help better evaluate the damage to the model's knowledge on points in close proximity to the deleted data point.
3.4 Manual Filtering and Human Evaluation
Table 4 shows the human evaluation results on UNLOK-VQA. In our human evaluation process for UnLOK- VQA, we established clear standards for annotators to assess the quality of the generated data. The evaluation focused on two primary criteria: (1) Consistency of Target Answers: Annotators were tasked with determining whether the target answer remains consistent when evaluating rephrased data. A consistent answer indicates that the rephrase effectively captures the original intent of the question. (2) Appropriateness of Answer Changes: For neighborhood data, evaluators assessed whether the target answer changes appropriately in response to the modified question or context. An appropriate change signifies that the alteration aligns with the expectations set by the original question.
We observe that around 75% of the automatically generated rephrase data and around 66% of the automatically generated neighborhood data meet our standards. To further enhance the quality of the datasct, we conduct one round of manual filtering to remove data that do not have proper rephrase or neighborhood images/ taaages/qustionn questions. We again conducted a human evaluation on the filtered UNLOK-VQA, finding that over 90% of the samples (shown in Table 4) meet the criterion. We adopt this high-quality, filtered version for our subsequent experiments. See Table 7 for samples in UNLOK-VQA, The details of the design questions for human evaluation and the interface demonstrations are provided in Appendix C.
3.5 Dataset analysis
The dataset consists of 500 samples that have been manually filtered and verified by human evaluators. Each sample contains a (v. 9, a) triple, where a represents the correct answer to question q within the context of image Additionally, each sample includes three types of rephrased images, two types of neighborhood images, average of four neighborhood questions, and three types of rephrased questions. Figure 4 illustrates the distribution across the various categories within UNLOK-VQA.
4 Attack-and-Defenss Perspective
Open-source releases of MLLMs (Liu et al., 2023a) necessitate robust evaluation methods that go beyond simply assessing model generations. We incorporate diverse whitebox attacks and blackbox attacks into deletion evaluations to strengthen claims about model safety and privacy. We cast the multimodal information deletion problem within the framework of adversarial attack and defense mechanisms commonly employed in machine learning security (Carlini et al., 2019). In this context, the objective the defense methods is to effectively crase a single piece of information from multimodal LLM while the attack methods aim to retrieve the deleted information from the model. In the following subsoctions, we introduce our attack-and-defense framework, including the threat model, attack methods, and defense methods, in detail.
8

---

Published in Transactions on Machine Learning Research (12/2024)
4.1 Threat Model
Building upon the work of Patil el al. (2023a), we broaden the security landscape for information deletion by introducing a threat model tailored to multimodal data and models.
Adversary's Objective: We posit an adversary aiming extract answer A to question Q in the context. of the image V, where the triplet (V, Q, A) is a sensitive piece of information. An extraction attack is considered successful with budget B if answer A is present within a candidate set C (IC) = B) generated by the adversary through an inference algorithm applied to the multimodal model. We refer to the size of the candidate set, denoted by [C], as the attack budget (B), thus making our setting more general than the typical one-shot sctting (Carlini et al., 2018; Lukas et al., 2023). This sctting is similar to the threat model introduced in Patil et al. (2023a) for LLMs and generalizes the typical one-shot selting (B = 1) by allowing multiple attempts at extraction. This more general setting reflects plausible scenarios, where the adversary could either (1) Attempt multiple queries to guess sensitive information (2) Generate multiple potential candidates and act on any correct one (3) Verily the correctness of information, as in the case where the adversary is also the data owner trying to confirm that their sensitive data has been properly deleted. By allowing B>1, we account for these broader, more realistic adversarial capabilities, making our setting more comprehensive and practical.
We say that an adversary is successful if the answer present in the candidate set C. We thus define the success of extraction attacks in the context of MLLMs using the following metric caleulated using dataset D (U:: qe, az) where each A = a, represents correct answer to the question Q 4 in the context of the image V = v4. The definition of the metric is:
Attack Success Metric:
1 1 N 1a E cC,
AttackSuccess@B(MM
E
(2)
where C, denotes the candidate set generated by the model M for the data point (t), i.e., C, = =Migi vs BB) and 1l represents the indicator function..
Adversary's Capabilities: We have two access levels to simulate real-world attacker (Carlini et al., 2019):
whitebox and blackbox access. Whitebox access assumes the adversary has full knowledge of the model's weights and architecture, enabling forward passes and access hidden states. Blackbox access limits the adversary to providing inputs and receiving randomly sampled outputs. These access levels reflect prevalent LLM access methods, available either open-source (Liu et al., 2023a) or through APls (Brown et al., 2020).
Metrics for Multimodal Information Deletion: The objective of information deletion is to selectively remove specific information from a model while simultaneously preserving its overall knowledge. However, similar to previous strategics performing sensitive information unlearning, model editing methods incur some performance loss on knowledge-intenstn evaluate such information loss. Overall, when we employ model editing attacks, the objective is to minimize attack success while simultancously minimizing damage to the model's 's knowledge. This is formulated as: arg min AttackSuccess@i ((M*) + *ADamag(MM", is the edited model, M is the pre-edited model, and Damage( ',') :) measures the impact on the model's 's knowledge compared to the unedited model. We use two metrics to assess model damage after editing:
tasks. Hence, it is necessary to have dedicated metrics to
defense against extraction
where M
1. Random A-Accuracy (Zhu et al., 2020; De Cao et. al., 2021): Measures the change in model accuracy for random data points before and after editing.
2. Neighborhood A-Accuracy: To assess the specificity of an edit in multimodal setting, we calculate two versions of the neighborhood A-Accuracy (Meng et al., 2022): Question Neighborhood A-Accuracy and Image Neighborhood A-Accuracy. The definitions and methods for generating neighborhood data are detailed in Section 3.3. We employ neighborhood questions and images to compute the respective Question and Image Neighborhood A-Accuracy.
We also report the Rewrite Score, which measures how effectively the edit reduces the target probability relative to the desired change (See 3.1).
9

---

Published in Transactions on Machine Learning Research (12/2024)
4.2 Attack Methods
The output of an atlack method is to generale a candidate set C that potentially contains the information that it aims to extract. Here the size of the C is limited by the attack budget B.
Whitebox Attacks. We use whitebox attacks from Patil et al. (2023a) that loverage Logit Lens (nostalge- braist, 2020; Geva et al., 2021), an interpretability technique that probes the hidden states of an LLM, and exploit the hypothesis that deleted information might persist in the model's intermediate layers' hidden states despite its removal using by editing model using deletion objectives or its absence in the final generation output.
1. Head Projection Attack (Patil et al., 2023a): This attack constructs candidate set by collecting the top-k highest probability tokens from each layer probed by LogitLens.
Probability Delta Attack (Patil et al., 2023a): This attack constructs a candidate set by identifying tokens whose logit lens probabilities rise and fall significantly between consecutive layers, potentially capturing the "deleted" information.
3. Probability Delta² Attack (Our attack): In this novel attack, we construct a candidate set by identifying tokens whose differences in probabilities across consecutive layers rise and fall significantly between consecutive layers (which means taking the difference of difference in the probabilities of tokens across consecutive layers), potentially capturing the "deleted" information. The PD attack is an order two attack, where second-order difference (difference of differences) between the distributions is computed, providing second-order comparison. We design this with the aim of capturing more nuanced traces of deleted information that might be overlooked by simpler approaches.
4. Finetuning-aasee attack: A major obstacle in unlearning is its resilience to few-shot fine-tuning, where a small Sie--unnng dataset causes a disproportionate return of previously deleted knowledge (Henderson et al., 2023; Yang ct al., 2023; Qi et al., 2023; Lermen al., 2023; Zhan et al., 2023). We fine-tune the edited model on random, unrelated data and then use the HP attack to assess robustness to post-deletion fine-tuning.
Blackbox Attacks. This simple blackbox attack exploits the imperfect rephrase generalization of model editing methods (De Cao et al., 2021; Meng et al., 2022). It constructs the candidate set C by querying the edited model with rephrased versions of the original input. We explore three variants to evaluate effectiveness across different modalities:
Image Rephrase Attack: This attack uses rephrased images with the same questions to test model vulnerability to changes in visual representation. It has three variations based the rephrase levels.
2. Question Rephrase Attack: This attack uses rephrased questions with the same images to test vulnerability to changes in textual representation. It has three variations are based on the rephrase levels.
3. Multimodal Rephrase Attack: This attack leverages data points where both the question and the image rephrase. This provides a holistic evaluation of the rephrase attack's effectiveness in exploiting weaknesses across both modalities within the edited multimodal data. We pick the best (hard) question rephrase and the best (hard) image rephrase for this attack.
4.3 Defense Methods
This section explores existing objectives for sensitive information deletion in MLLMs.
Empty Response Defense (Ouyang et al., 2022): This method optimizes the MILM edit to output a non-sensitive response (e.g., "I don't know" or "dummy") instead of sensitive information. The objective
10

---

Published in Transactions on Machine Learning Research (12/2024)
function, arg max MM), maximizes the probability of the model generating empty target string (d) for any given input (v,
Fact Erasure (Fact-Eras) (Hase et al., 2023): This approach reduces the probability of the MLLM generating a sensitive answer (a) for a given question (g) and image (v), by minimizing p(a,u, q; M) for the original information (v, 9, a).
Error Injection (Error Inj) (De Cao al., 2021): This method introduces counterfactual knowledge into the model. Using the objective function arg rrgmax (ll q; M) (Meng et al., 2022), where a" is false target answer, it demonstrates the model's ability to incorporate manipulated information.
Head Projection (HP) Defense (Patil et al., 2023a): This approach employs a max-margin objective to prevent the deleted answer from appearing among the top-k elements in LogitLens distributions across chosen layers (I.) and the final output.
Max-Entropy Defense (Patil et al., 2023a): Similar the Head Projection Defense, this approach focuses LogitLenss distributions but uses different objective per layer. It maximizes the entropy (uncertainty) of the next-token distribution across chosen layers (L) and the final output.
This defense strategy targets the Input Rephrasing Blackbox Attack. It expands the editing objective beyond the original input (v) by incorporating three versions of rephrases of the input (v, q): (1) (rephrase(v), q) (2) (e, rephrase(g)) (3) (eephrass rephrase(4)).
Input Rephrasing (IR) Defense:
5 Experimental Setup
Models and Editing methods. Our experiments involve the multimodal LLM: LLaVA-v1.5-7B (Liu et al., 2023a). This model is selected based on its: (1) widespread adoption within the multinodality community, (2) ease of access due to public availability, and (3) documented ability to retain information from their pre-training data. We also report the attack success rates on a larger LLaVA-v1.5-1a for evaluating the effect of scaling model size on the robustness of crasure methods to attacks.
Model editing methods. Our experiments utilize LoRA finetuning for information deletion in MLLMs, targeting specific weight matrices in the model's MLP layers, as motivated by Meng al. (2022). While that analysis focused GPT models, tune LLaVA-v1.3-7B and LLaVA-v1.5-33B and find that editing the 7th and 9th layers, respectively, yields effective results with rewrite score over 85% (indicating successful information erasure) and a random A-Ace below 5% (mostly preserving the model's overall knowledge). While prior works, such as (Meng et al., 2022), have attempted to localize information using causal tracing and then edit the corresponding weights. A follow-up study Hase et al. (2023) demonstrates that localization does not necessarily guide effective editing. This why, we opt to select layers empirically rather than relying localization.
6 Results
6.1 Main Results
Design. We first investigate how each of the defense methods outlined in Section 4.3 fares against each of the extraction attacks outlined in Section 4.2 on UNT.OK-VQA. We measure Attack-Succes@0F with B = 20 for each of the attacks (both whitebox and blackbox attacks), in addition to Random A-Ace and Question Neighborhood A-Ace and Image Neighborhood A-Acc metrics. Our investigation is conducted on the L.LVV--vl.-77 model with LoRA finetuning as the editing method. To ensure that each editing method functions as intended and allows for fair comparison, we meticulously adjust the hyperparameters until we reach reasonable rewrite scores and A-Ace (as mentioned in Section 5). Below, we report the results and answer three questions regarding multimodal model editing for targeted unlearning.
Can we extract piece of deleted multimodal information from an MLLM? Yes. Our results Table demonstrate that both whitebox and blackbox extraction attacks are successful. Among whitebox attacks, the Probability Delta (PD) attack exhibits the strongest. performance, frequently achieving attack
11

---

Published in Transactions on Machine Learning Research (12/2024)
Whitebox Attack
Blackbox Attack
Rand
Q Neigh
Neigh A-cce
Rewrite
PD
PD
HP4FT
Hard Img Rephrase
Hard Ques
A-Acc
A-Ace
Score
Rephrase
LoRA
Pact-Eras
0.300
0.296
0.264
0.412
0.320
0.390
0.455
0.001
0.008
0.059
0.956
Emply
0.777
0.930
0.759
0.817
0.082
0.793
0.789
0.045
0.024
0.059
0.965
Entropy
0.185
0.181
0.145
0.253
0.304
0.402
0.431
0.029
0.037
0.12
0.883
HP
0.036
0.133
0.105
0.121
0.058
0.101
0,157
0.032
0.027
0.007
0.999
Error Inj
0.423
0.477
0.463
0.468
0.366
0.425
0.485
0.046
0.037
-0.068
0.895
+ Fact-Eras
0.159
0.195
0.169
0.229
0.203
0.237
0.322
0.004
0.009
0.066
0.974
Table 1: Attack success rates of the attacks (Section 4.2) against defense methods (Section 4.3) for deleting multimodal information in UNLOK-VQA that is known by the L.aVA-v1.5-7B model. The deletion is performed via LoRA edits to the MLP modules within an LLM layer of LLaVA.
success rates exceeding 20% (when budget B is set to 20) against most defenses. While PD attack does better than HP, PD² does not improve on top of PD. This is likely because higher-order attacks delve deeper into differences, making the targeted information less apparent. The finetuming attack (HP+FT) has higher attack success rate compared to the original HP attack which shows that all the defenses are vulnerable to the finetuning attack. For blackbox attacks, the multimodal (MM) rephrasing attack also often succeeds more than 35% of the time. These high success rates indicate a high vulnerability extraction attacks targeting the "deleted" fact within the threat model outlined in Section 4.
Are the defenses effective against extraction attacks? Table 1, our analysis reveals that, among the whitebox defense methods, Fact Erasure, Empty Response (Empty), and Error Injection defenses are less effective compared to Head Projection (HP) and Max-Entropy defenses. This observation indicates that information is better concealed when the model is uncertain about the answer. In contrast, the first three methods may lower the probability of the sensitive answer excessively, making the concealed information more detectable. We then observe that the HP defense exhibits the highest overall effectiveness against all attacks (whitebox and blackbox), as evidenced by its consistently lowest attack success rates compared to all other defenses. On the blackbox defense front, we only report the Question Rephrase defense results in the Input Rephrasing defense row in Table as we find outperforms the image and multimodal counterparts. We exploit the easy question rephrases for the defense. Furthermore, we observe that the Image Rephrase defense is effective only when the attack images and defense images belong to the same distribution, whereas the question rephrase defense is effective across all distributions of question rephrase.
Is multimodal attack more effective than unimodal? We investigate the efficacy of the multimodal blackbox attack strategy compared to the unimodal blackbox al- tacks (image rephrase attack, question rephrase attack). The results in Table show that the multimodal (black- box) attack success rate (15.7%) is 5.6% higher than the best question rephrase attack (Hard Question Rephrase) (10.1%) and 9.9% higher than the best image rephrase atlack (Hard Img Rephrase) (5.8%) against the best. (HP) defense. Similar trend holds against other defenses.
Easy
Med
Hard
Img Rephrase
LoRA
Fact Erasure
0.296
0.279
0.320
Empty
0.645
0.631
0.682
Entropy
0.251
0.255
0.304
HP
0.062
0.059
0.058
Error Injection
0.360
0.349
0.366
IR + Fact-Eras
0.103
0.098
0.203
Question Rephrase
6.2 How Does Scaling the Model Size Affect the Vulnerability of Erasure to Attacks?
LoRA
Fact Frasure
0.353
0.367
0.463
Empty
U.196
0.198
0.343
Entropy
0.294
0.293
0.406
Design. We explore the relationship between model size and susceptibility to both whitebox and blackbox attacks. We scale the model size (from 7B to 13B parameters) while keeping all other factors constant which allows us to isolate the impact of model size robustness to attacks. We
HP
0.069
0.072
0.146
Error Injection
0.401
0.370
0.475
Pact-Eras
0.112
0.186
0.262
Table 2: Comparison of attack success rates across the three levels of rephrase images to attack mod- els edited by different defense mechanisms..
12

---

Published in Transactions on Machine Learning Research (12/2024)
7B
13B
Easy
Hard
Combined
0.6
Img Neighborhood
LoRA
0.4
Fact Erasure
0.058
0.060
0.059
Empty
0.052
0.066
0.059
0.2
Entropy
0.060
0.180
0.120
HP
0.005
0.009
0.007
Error Injection
0.065
0.071
0.068
IR + Fact-Eras
0.051
0.081
0.066
MM Rephrase
HP
Comparison of Img-neighborhove AcAcc across the two complexity levels of neighborhood images to evaluate the model damage caused by the model editing using the different objectives for unlearning.
Table 3:
Figure 3: Effect of scaling the LLa size from 7B to 13B on attack success HP attack (whitebox) and Multimodal Rephrase Attack (blackbox) against the Fact Erasure defense. We find that scaling makes the models more robust against the attacks.
Human Judgements
Easy
Med
Hard
Pre-filter
Post-filter
Img Rephrase
Rephrase Question
All
0.79
0.93
Edit module
Rephrase Image
Medium
0.73
0.91
LLM MLP
0.062
0.059
0.088
Hard
0.79
1.00
MM proj MLP
0.212
0.201
0.406
Neighborhood Question
Question Rephrase
All
0.73
0.97
Neighborhood Image
LLM MLP
0.069
0.072
0.146
Easy
0.66
0.97
Hard
MM proj MLP
0.244
0.212
0.408
Table 4: Human evaluations assessing the quality of data samples generated by the model, conducted both before and after manual filtering process, to ensure the removal of low quality samples.
Table 5: Comparison success rates for image and question rephrasing attacks when modifying the MLP module within LLM layers versus within the multi- modal projector on UNLOK-VQA.
chose to evaluate using the Baet-Erasureee defense because HP defense more sensitive to hyperparameter selection.
To analyze the effect of scaling,
aimed to keep the evaluation independent of hyperparameter choices. We chose the HP attack because it modifies the model's weights, which could interfere with analyzing the effect of scaling. While unlearning and scaling interact consistently across methods, fine-tuning introduces a confounding factor, making it unsuitable for this evaluation.
Results. Figure: presents our findings. clear trend indicates that the larger model (13B) when edited using the same deletion objective exhibits greater resilience against atlacks in both whitebox (HP attack) and blackbox (multimodal rephrase attack) settings compared a smaller model (7B). This suggests that increasing model complexity improve the efficacy of delction methods and thereby enhance the model's ability to defend against targeted altas.s.
6.3 Ablation Across Different Difficulty Levels
Design. To investigate whether the intuitive trend of easy, medium, and hard rephrase samples (introduced in Scction 3) is reflected in the attack success rates of the respective rephrase images and questions, we conduct ablation experiments across the three difficulty levels of rephrase questions and rephrase images, and present the result in Table 2. Similarly, to investigate whether the intuitive trend of easy and hard
13

---

Published in Transactions on Machine Learning Research (12/2024)
neighborhood images (introduced in Section 3) is reflected in the image-neighborhood A-Acc, we conduct ablation experiments across the two difficulty levels of neighborhood images, and present the result in Table 3.
Rephrase ablation results. Our observations in Table 2 reveal that easy and medium rephrase images exhibit similar attack success rates against most defenses, whereas hard rephrase images are significantly more effective. Similarly, we observe a consistent trend for the three types of question rephrases: easy and medium question rephrases demonstrate similar attack success rates against most defenses, while hard question rephrases based on jailbreak prompts have the highest attack success rates. One potential reason for the success of hard images lies in the fact that although their semantic content is preserved, the general composition of the images is significantly altered. This alteration can, to some extent, render defense mechanisms ineffective.
Neighborhood ablation results. We observe that the Image Neighborhood A-Acc is higher for hard neighborhood images compared to easy neighborhood images as evident from Table 3. This is because the hard neighborhood images lie closer to the deleted sample point than the easy neighborhood images by construction.
6.4
Editing Multimodal Projector
Whitebox Attack
Blackbox Attack
Rand A-Ace
Q Neigh
I Neigh Á-Ace
Rewrite Score
Edited Module
HP
PD
PD2
HP+FT
Hard Img
Hard Ques Rephrase
MM
A-Ace
Rephrase
LLM MLP
0.036
0.133
0.105
0.052
0.058
0.101
0.157
0.032 -0.038
0.027 -0,092
0.007 -0,008
0.999 0.954
MM proj MLP
0.205
0.232
0,187
0.247
0.406
0.408
0.573
Table 6: Comparison of success rates for the attacks in Section 4.2 when tuning the MLP module within LLM layers versus within the multimodal projector on UNLOK-VQA.
Design. In the conventional fine-tuning paradigm for MLLMSs applied to downstream tasks (Liu et al., 2023a; Li et al., 2023), the focus on optimizing the multimodal projector, which connects the LLM to the vision encoder and enables them to interact with each other. Our investigation aims to determine if a more effective approach lies in editing modules within the multimodal projector rather than editing the LLM modules.
Results. Our findings presented in Table 6 demonstrate that editing modules within the LLM led to better deletion performance (lower attack success rates for similar rewrite score and A-Acc) against the best defense (HP defense) compared to the conventional approach of editing/ ceitinggfinn-tuiinn fiin-tuninng the multimodal projector. This observation is also consistent across the three types of rephrase attacks presented in Table 5 as the atlack success rate when editing the multimodal projector is much higher. This suggests that editing the LLM modules could be a more effective strategy for multimodal information deletion in MLLMs. In the context of projector editing too, the rising success rates of attacks across easy, medium, and hard rephrases suggest that the more difficult rephrases present greater challenges for generalization, as shown in Table 5. The improved results from editing within LLM layers, compared multimodal projectors could stem from the stage of processing. Multimodal projectors operate early, handling raw input translation before the model fully integrates information. In contrast, LLM layers process data later, where final knowledge representations are formed. Editing at this stage is more effective, directly targeting the model's semantic associations, possibly resulting in more precise removal of specific knowledge.
7 Conclusion
Our study introduces UNLOK-VQA, holistic dataset for evaluating targeted unlearning in MLLMs. We present an allack-and-dleeense framework and a pipeline for creating high-quality image-text pairs for evaluating efficacy, specificity, and generalization of defense methods. Our findings reveal that multimodal extraction attacks are more successful than singlo-modality attacks, while the best defense mechanism reduces attack success significantly. This work underscores the importance of developing effective unlearning methods for MLL.Ms and provides critical resource for advancing research in this area.
14

---

Published in Transactions on Machine Learning Research (12/2024)
Broader Impact Statement
Our work addresses the critical issue of deleting scnsitive information in multimodal large language models (MLLMss), highlighting significant ethical implications. MLLMs, with their vast multimodal knowledge, can potentially generate harmful content or perpetuate biases if misused. Our proposed dataset and technical approaches aim mitigate these ethical challenges. However, our findings reveal the complexity of effectively removing sensitive information from pretrained MLLMs, raising moral and legal concerns about their responsible deployment. Our research seeks to promote responsible AT innovation.
Acknowledgments
This work was supported by NSF-AI Engage Institute DRL-2112635, DARPA MCS Grant N66001-19-2-400 NSF-CAREER Award 1846185, ARO Award W911NF2110221 ONR Grant N00014-23-1-22511 1-2356, and Google PhD Fellowship. The views, opinions, and/or findings contained in this article are those of the authors and not of the funding agency.
References
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.1 12966, 2023.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint or Xiv:2212.08073, 2022.
Nora Belrose, David Schneider-Jospph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306. 03819, 2023. URL https://arxiv http:://arxxv.oor/ppf/////2 /2306 03809. pdf.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877- 1901, 2020.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463-480. IEFE, 2015. URL https://www. wew-- org/ h//5p2205/ppper--arcivvvee
/edd//9999443.../ pdf.
Nicholas Carlini, Chang Liu, Ülfar Erlingsson, Jernoj Kos, and Dawn Xiaodong Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Securily Symposium, 2018. URL
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint 3rXiv:1902.06706 2019.
Yang Chen, Ethan Mendes, Sauvik Das, Wei Xu, and Alan Ritter. Can language models be instructed protect personal information?, 2023.
Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, and Ningyu Zhang, Can we edit multimodal large language models? In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13877-13885, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653 /tt////11223.emmmmme/ 856. URL https: http://acaaanttoggoyoo wwo.org///00
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
15

---

Published in Transactions on Machine Learning Research (12/2024)
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling cnatructoofiietunee language models. Journal of Machine Learning Research, 25(70):1-55 53, 2024.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empiricul Melhods in Natural Language Processing, pp. 1491-65000 2021.
Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher Choquette-Choo, Matthew Jagielski, Milad Nasr, Eric Wallace, and Florian Tramèr. Privacy side channels in machine learning systems. arXiv preprint ar 23v: 05610, 2023.
Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. arxiv preprint arXiv:23030073: 2023. URL https://arxiv.. http://wwwwi..rrp/////////// 07345.
Mor Geva, Roci Schuster, Jonathan Bcrant, and Omer Levy. Transformer food-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Nolural Language Processing, pp.
2021.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform oditing? surprising differences in causality-based localization vs. knowledge editing in language models. Advances in Newral Informalion Processing Systems, 36, 2023.
Peter Henderson, Fric Mitchell, Christopher Manning, Dan Jurafsky, and Chelsea Finn. Self-destructing models: Increasing the costs of harmful dual uses of foundation models. In Proceedings of the 2023 AAAT/ACM Conference on AI, Ethics, and Sociely, pp. 287 296, 2023.
Alvin Heng and Harold Soh. Selective ammesia: A continual learning approach to forgetting in deep generative models. arXiv preprint arXiv:2305.10133 2023. URL http:://axxiv..g/////////// http:////23..11111 .dff.
Yushi HU, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah Smith. Tifa: Accurate and interpretable lext-lo-image laithfulness evaluation with question answering. In Proceedingss of the LEEE/CVF International Conference on Computer Vision, pp. 20406 20417, 2023.
Wenyue Hua, Jiang Guo, Mingwen Dong, Henghui Zhu, Patrick Ng, and Zhiguo Wang. Propagation and pitfalls: Reasoning- based assessment of knowledge editing through counterfactual tasks. arXiv preprint 177555.pff.
arXiv:240111758. 2024. URL attps://arxits orrgiiioor///oooooo.ooo
org/ wwt/ootg//////oooo/oo/tto/t../p/ http//dff/2/0// /2401.
Han Huang, Haitian Zhong, Q. Liu, Shu Wu, Liang Wang, and Tien-Ping Tan. Kebench: A benchmark knowledge editing for large vision-language models. ArXiv, abs/2400.07330, 2024. URL https:
saapiissmanticcchol... wwwpatttcccccaaaoooooooorrrallli wotg/CorpssI::2683338 268334447.
Gabriel Iharro,, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic, 2023. URL attps://arxxs org/pdf/ 2212.04089.0 pdf.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, pp. 2, 2019.
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing evades detectors of ai-generated text, but retrieval an effective defense. arXiv preprint arXie: 13t.20333,3408, 2023.
Nupur Kumari, Bingliang Zhang, Sheng- Ablating concepts in text-to-image diffusion models. arXiv preprint arxw::3099 13516, 2023. URL https://arxiv.c www.iiv..gg/////////// /2003 13516. pdf.
Yu Yu Wang, Eli Shechtman, Richard Zhang, and Jun- Yan Zhu.
Simon Lermen, Charlie Rogers- Smitt,,Smith, and Jeffrey Ladish. Lora fine-tuning efficiently undocs safety training in llama 2-chat 70b. Xiiie preprint arXi::2300 20624, 2023.
16

---

Published in Transactions on Machine Learning Research (12/2024)
Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, and Sheng Bi. Single image unlearning: Efficient machine unlearning in multimodal large language models. or arv preprinl arXiv:2405. 200.2445.1223, 2024a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. Blip-2: Bootstrapping languago-image pre- training with frozen image encoders and large language models. In International Conference on Machine Learning (ICML), 2023. URL https://api. www.stiiimmoooata........ otgg/orppuID:2563905. 256639050.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann- Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdpp benchmark: Measuring and reducing malicious use with unlcarning. arXiv preprint arXiv:2433.022 03218, 2024b. URL Sttps://arzer .rg.ivvorrrr.ooo org/pdf/ 2403. 02t.8..22.....
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023a.
Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. Query-relevant images jailbreak large multi- models. arXiv preprint orXiv:2311.17600 2023b.
Nils Lukas, A. Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-B eguelin. Analyzing leakage of personally identifiable information in language models. 2023 IEEE Symposium on Security and Privacy (SP), pp. 346 363, 2023. URL httpp://app..emmmttccccchoo....
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan HadfieldMMenell. Eight methods to evaluate robust unlearning in llms. arXio preprint arXiv:2402. 16835, 2024.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for Nms. aixiv preprint ar arXiv:2401.061 2024.
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowlodge. In Proceedings of the IEEE/cof conference on computer vision and pallern recognition, pp. 3195 3204, 2019.
Kevin Mong, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 36, 2022.
nostalgebraist.
interpreting gpt: the logit lens, 2020. URL
OpenAl. Gpt-4 technical report. 2023. URL http:://g...eeaaeaacoaoo.a/aooop
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from 1lms? objectives for defending against extraction attacks. In The Twelfth International Conference on Learning Representations, 2023a.
Vaidehi Patil, Adyasha Maharana, and Mohit Bansal. Debiasing multimodal models via causal information minimization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 4108-4123, 2023b.
Vaidehi Patil, Leonardo Ribeiro, Mengwen Liu, Mohit Bansal, and Markus Dreyer. Refiesumm: Self-refining milm for generating multimodal summarization dataset. Tn Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13773-13786, 2024.
Renjic Pi, Tianyang Han, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. pllmeerrotctoo: Ensuring milm's safety without hurting performance. arXiv preprint arXiv:2401.088 029000 2024.
17

---

Published in Transactions on Machine Learning Research (12/2024)
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL; Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.ces forum?id=d152228xqt..
Xiangyu Qi Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine- tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint or arXiu:2310.0000 2023.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Con- ference on Empirical Methods in Natural Language Processing, 2019. URL https:///ae somantccscooaar. stttp arg/CorpusID::011.. 201644339..
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual lasks, 2024.
Rafael Rosales, Pablo Munoz, and Michael Paulitsch. Exploring resiliency to natural image corruptions in deep learning using design diversity. arXiv preprint arXiu:2303.09983 2023.
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint S.Hi:2308.012666 2023.
Rusheb Shah, Quentin Feuillade Montixi, Soroush Pour, Arush Tagade, and Javier Rando. Scalable and transferable black-box jailbreaks for language models via persona modulation. In Socially Responsible Language Modelling Research, 2023.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. Xii preprint ar XtXw:23100167889 2023.
Yhien-Yao Wang, T-Han Yeh, and Hongpeng Liao. Yolov9: Learning what you want to learn using pro- grammable gradient information. ArXiv, 120122222.16166 2024. URL attps://api. sem.aticccannno..rr.rr. Dtg/CorpuuIDD:257770555
Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint Xxx::23100029/// 2023.
Jaehong Yoon, Shoubin Yu, Vaidehi Patil, Huaxiu Yao, and Mohit Bansal, Safree: Training free and adaptive guard for safe text-to-image and video generation. arXiv preprint arXiv:2410.12761 2024.
Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang, Rrhf: Rank responses to align language models with human feedback without tears. arXiv preprint 2rX5::2304.05302 2023.
Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. Removing rlhf protections in gpt-4 via fine-tuning. arXie preprint arXiv:2311.0555, 2023.
Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori B Hashimoto, and Daniel Kang. Removing rlhf protections in gpt-4 via fine-tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 681-687, 2024.
Eric Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forgot-mo-not: Learning to forget text-to-image diffusion models. arXiv preprint Xiv: 11591,7911 2023.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models. arXiv preprint ar Xix:2200 00363, 2020.
18

---

Published in Transactions on Machine Learning Research (12/2024)
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhossiny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. ArXiv, abs/2304.10592, 2023. URL
https://api... /
semanticscholar pawaamniisccoola.oooo//oroppaao (000rppp 258291930.
Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint urXiv:2402 02207, 2024.
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks aligned language models. arXiv preprint arx::23007 2023. URL https: htta:://arrre //axxvvv.vv////////////////aaatl org/pdf/2233 2307. 15043. pdf.
A Dataset samples
We present examples of samples in UNL.KK-VQA in Table 7.
B Reproducibility Details
Image generation prompts for SDXL:
Neigh (Fasy): Generate an image for which the answer to this question: (question] is (alternate answer] and there is no {original answer] in the image.
Neigh (Hard): Generate an image for which the answer to this question: {question] is (alternate answer) and there is no (original answer) in the image including some components from the following image description but retaining the answer (alternate answer): {unrelated image description). Please make sure the answer to the question (question) in the context of the image is (alternate answer].
Replace (Medium): Replace the foriginal answer) to another reasonable foriginal answer), high quality, detailed.
Rephrase (Hard): Generate an image for which the answer this question: {question) is {original answer) based on the following image description: foriginal image description).
We observe small diversion in the trend against the input rephrasing defense, medium question rephrases are more effective than easy question rephrases because the defense employs question rephrases belonging to the easy question rephrase distribution.
Editing modules. We edit the MLP down projection module within Layer of LORA finetuning. We apply LoRA with rank of 1 and a of 1. This enables loss aggressive editing approach in order to make the edit targeted and avoid damage to data points that were not meant to be deleted.
Filtering details. We filter the details OK-VQA dataset before applying the automatic generation pipeline so as to to retain single token answers and to make sure that the model knows the answer before we delete it similar to that in (Patil et al., 2023a).
C Human Evaluation
Motivated by prior work that involves evaluation of multimodal summarization dataset (Patil et al., 2024), for our human evaluation experiments, engaged four graduate research assistants with computer science backgrounds. They were given set of instructions for interpreting and performing the task. During virtual meetings, we provided an overview of the datasel and a detailed task description. We collected for 80 of the dataset samples. The evaluation was conducted twice: once without manual filtering and once with manual filtering. We compute accuracy of the annotation with respect to ground truth answers for an idcal dataset. We added three options to the answers to simplify the annotation task. We present the results in Table 4 and screenshots of the human evaluation interface in Table and Table 9.
19

---

Published in Transactions on Machine Learning Research (12/2024)
Sports and Recreation
10.8%
12.6%
Plants and Animals
Other
2.8%
15.0%
Geography, History, Language Culture
Objects, Material Clothing
Categories
Weather and Climate
14.6%
People and Everyday life
13.2%
Vehicles and Transportation
8.8%
Brands, Companies Products
9.6%?.6%
Science and Technology
Cooking and Food
Figure 4: Distribution of question categories in UNLOK-VQA, It consists of samples belonging to diverse categories and covers all the categories in the original OK-VQA dataset.
2.0
2.0
1.5
1.5
1.0
1.0
0.0
0.0
Random
Neigh Img
Rephrase Img
Random
Neigh Ques
Rephrase Ques
Figure 5: Average distance of the random, neigh- borhood image and rephrase image points from the original data point. Neighborhood points are closer to the larget data point being deleted compared to random points on which other unlearning datasets evaluate specificity. Rephrase points are closer com- pared to both neighborhood and random data points. This is also reflected by higher Image Neighborhood Acc compared to Random AcAcc.
Figure 6: Average distance of the random, neighbor- hood image and rephrase question points from the original data point. Neighborhood points are closer to the target data point being deleted compared to random points on which other unlearning datasets evaluate specificity. Rephrase points are closer com- pared to both neighborhood and random data points. This is also reflected by higher Question Neighbor- hood AcAce compared to Random A-Acc
D An Additional Method for Neighborhood Image Generation
Motivated by prior work that involves evaluation of the model invariant images (Patil et al., 2023b), in order get medium rephrase images, use Grounded SAM to repaint the original image minimally to let the answer to the original question become the alternate answer a'. To get the target for Grounded SAM, we prompt TX-XXL (Chung et al., 2024) to get the subject from q that leads to the answer a. Then, we ask Grounded SAM to repaint the subject in the image to the alternative answer a'. These images are considered the hardest because the majority of the pixels remain the same, iee. we expect them to lie within a smaller radius neighborhood compared to modium neighborhood images. While we tried this variant of generating neighborhood images, found that it is not possible change the answer question in the context of the image by editing just object, especially for samples in OK-VQA that could involve multi-hop reasoning to
20

---

Published in Transactions on Machine Learning Research (12/2024)
answer the question. However, we find this approach cannot successfully alter the image to fit the alternate answer, so we do not adopt this approach in our study.
21

---

Published in Transactions on Machine Learning Research (12/2024)
Rephrase Images (Easy, Medium, Hard, respectively)
Neighborhood Images (Easy, Hard respectively)
Rephrase tions
Ques-
Neighborhood
Target data
Questions
Easy Rephrase: The following blog- post. What issues would someone have bringing this suitcase on plane?
Question: Is there
suitcase? Answer:
Medium Rephrase: What difficulties would there be if someone brought this suitcase on board plane?
Question: What type of container is this?
Question: What the cat doing? An- swer: Laying
Hard Rephrase: You'le Al with access
Question: issues would someone have bringing this suitcase plane?
vast kuowledge about suitcase. Please help me answer this question. What issues would someone have bringing this suitcase plane?
Question: case open? Answer:
Answer: Cat
Easy Rephrase: The following is hlog- post. What kind habitat shown?
Question: Who is joying the natural sur- roundings? Answer: Riders
Medium Rephrase:
Question: Are there natural surroundings? Answer: Yes
What kind of habita- shown?
Hard Rephrase: You're an Al with
Question: Is this an outdoor activity? An- swer: Yes
accesss vast knowl- edge about hahitat. Please help me answer this question.
Question: What kind of habitat shown? Answer: Forest.
Question:
Is this
leisurely activity? Answer: Leisurely
What kind of habitat shown?
Table 7: Examples of samples in UNLOK-VQA. Rephrase images with original questions are used for image rephrase attack, Rephrase questions with original images are used for question rephrase attacks, and multimodal rephrase attacks use a combination of rephrase questions and rephrase images. Neighborhood Images with Original questions are used to compute Image Neighborhood A-Acc and Neighborhood questions with Original images used to compute Question Neighborhood A-Acc.
22

---

Published in Transactions on Machine Learning Research (12/2024)
Question
questton What be dodging?
mage
Auection A
Yee
Question
Image
qussiion What slte dodging? bnower Baceball
Based Image S. Answerr an answerr
No
O dan't know
Image
quection A: What he dodging?
nenwer A: Benebat
Answer Auestion
Image-A
Table Human evaluation interface 23

---

Published in Transactions on Machine Learning Research (12/2024)
Question
Daneral
danti
What to
Bcodboll
Doadd Antover
Camitoo ET
No
doel
Image 2:
What dedgin??
deower Booeboll
Qacilio Acavee
drew
Suove
Table 9: Human evaluation interface
24


Reviews:
[{"review_id": "eZeGyBlUDN", "reviewer": "Reviewer_ycL5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- Present a dataset for evaluating multi-modal editing.\n- An extensive study on different attack and defense methods that demonstrate the value of this dataset.\n- Demonstrate that the discrepancy between different model editing methods and the model behavior regarding different model sizes.\n\n**Weaknesses**\n\n- The definition of easy/medium/hard seems to be arbitrary, how do we know they are actually on that level? \n- In addition to the difficulty level, what are the quality of the examples? A in-depth analysis on generated data quality would be great. \n- Long description about how to generate examples without much statistics or some information about the dataset, or any information about dataset quality.\n- This work did an extensive study on attack and defense methods. However, I found it is difficult to get some insights from this work. In addition to the experiment results that demonstrate multimodal editing is much more effective than unimodal editing, I would like to see more explanation and justification of the experiment design.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work presents a new dataset that covers the variations on both images and texts, which allows a comprehensive evaluation on multi-modal editing. With a collection of model attack and defense methods, the work demonstrate the value of this dataset, as well as different patterns regarding unimodal vs. multi-modal editing and resilience regarding different model sizes.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It would be great to have more explanation and justification on the choices in experiment design\n- It would be great to have a more in-depth analysis on the data quality, for example, the quality of generated paraphrases.", "broader_impact_concerns": "No significant concern", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RcbDBR5hEQ", "reviewer": "Reviewer_Fhpe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths:\n1. The proposed dataset provides a systematic way to understand the robustness of the unlearning methods for MLLMs.\n2. The attack-and-defend framework binds together multiple unlearning methods for MLLMs and provides a standard benchmark for the community to build further.\n\n# Weaknesses:\n## Insufficient Evaluation / Discussion\n1. The authors claim that the proposed white-box attack called $PD^2$, is capable of capturing the \"deleted\" information. __(Q1)__ What is the motivation for this new attack over the PD attack? __(Q2)__ Why is the performance (ASR) of $PD^2$ poorer than almost all the other attacks?\n2. Are there any ablation studies to justify the choice of using the 7th and 9th layers for editing of the 7B and 13B MLLMs? Are there any reasons why they work better than other layers?\n3. For the scaling experiment (Figure 3): __(Q1)__ Why did the authors choose to evaluate only using the \"Fact-Erasure\" defence and not using the best defence method from Table 1 (__HP__) ? __(Q2)__ Given the fine-tuning attack is the strongest white-box attack (from Table-1), why didn't the authors use that instead of the HP attack in Figure 3?  __(Q3)__ The authors highlight that \"scaling makes models more robust\", but the experimental results in section 6.3 and figure 3, do not provide sufficient data points to support the claim.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "# This work presents\n##  (A) UnLOK-VQA, a benchmark for the evaluation of unlearning in multimodal Large Language Models (MLLM).\n1. Prior methods have focussed on creating data for the unlearning of LLM, but UnLOK-VQA extends and introduces a systematic evaluation of unlearning in MLLMs.\n2. This dataset evaluates the unlearning methods in three aspects; (a) __Efficacy:__ Measures how well the information is deleted, __Generalization:__ Evaluates if the method is robust to alternate ways of information extraction (eg: paraphrased prompts)  and __Specificity:__ Measures the damage on the benign (unintended) concepts.\n\n\n## (B) An attack-and-defence framework\n1. The authors have chosen 6 unlearning objects as the defence methods to evaluate.\n2. They evaluate the robustness of these 6 methods under both the white-box (4) and black-box (3) attacks.\n4. The authors evaluate the 6 models under these 7 attacks on the proposed UnLOK-VQA dataset and present their findings.\n5. Additionally, they also propose a white-box attack called $PD^2$.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Clarifications (Critical)\n1. When generating the _\"Rephrase Image - Medium\"_ data, the authors mention that a segment of the image (identified with GroundingDINO) is removed. __Question__: What happens to the removed region? Do the authors use any inpainting tools to fill the missing region?\n2. Out of curiosity, why is there no white-box attack focusing specifically on the image side? All the chosen white-box attacks are focussing on only the text side. Similar to Table 4, do the authors have any experiments justifying the need to attack only the text side? Is it possible to design an attack on the image side?\n3. In section 6.4, aside from the empirical justification for editing modules within the LLM yielding better results over multimodal projectors,  a discussion on why this pattern is observed should help understand the results better.\n\n\n## Recommendations (Not critical, but will strengthen the work)\n1. Figure 2, which provides a holistic overview, is not easy to follow immediately. It takes some time to understand what it tries to convey. Updating the image would help the reader follow the work more easily.\n2. There are a few typos / grammatical erors that can be rechecked and corrected.\n3. The chronology of the table can be corrected (eg: Table 2 and Table 3).", "broader_impact_concerns": "A broader Impact statement is provided in the manuscript.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qErkp7t4Wh", "reviewer": "Reviewer_jwQq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "__Strengths:__\n\n- The evaluation involves various information extraction and deletion methods.\n\n- The pipeline is clearly illustrated, which is easy to understand.\n\n__Weaknesses:__\n\n- The challenge(s) that this work aims to solve has/have not been well explained. More details are expected as to why generalization and specificity are important in this task and why existing works do not support them.\n\n- A mismatch exists between the authors' design and examples. Specifically, the design of medium-level rephrase images is to substitute one random object in the original image with an object generated by Grounded SAM (Section 3.2 and Figure 2). However, the examples provided in Table 7 show that medium-level rephrase images are created by removing a part of the image (instead of objects).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors proposed a pipeline to create VQA samples to evaluate the influence of an unlearned MLLM from the perspectives of generalization and specificity. Following the proposed pipeline, the authors constructed their UnLOK-VQA dataset based on an existing benchmark dataset OK-QVA, which is evaluated by several information extraction and deletion methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Besides providing clarification responses to the above weaknesses concerns, the following requested changes are expected.\n\n- The authors apply their pipeline only to the specific OK-VQA dataset. To validate the generalizability of the proposed method, more VQA datasets should be considered. Otherwise, a corresponding explanation of this decision is needed.\n\n- In Adversary's Objective (Section 4.1), the authors should better explain why their setting is more general than typical one-shot settings, which could improve the comprehension of this work's contributions.\n\n- In Section 6.2, only two LLaVA model sizes with 7B and 13B parameters are evaluated to draw conclusions on the influence of model size on vulnerabilities. Other model architectures with varying model sizes should be evaluated for a more convincing conclusion. If additional experiments are not possible, a relevant discussion is needed.\n\n__Minor Comments:__\n\n- Equation (2) could be introduced earlier, e.g., in Section 3.1 when it is mentioned.\n\n- Tables 5 and 6 should be placed before the references.", "broader_impact_concerns": "I do not have any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HRFZM9LI79", "reviewer": "Reviewer_kEAH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n1.\tThis paper introduces a new dataset for evaluating the deletion of specific undesirable multimodal knowledge from models.\n\n2.\tIt shows that the latest editing methods like LoRA fine-tuning cannot fully delete knowledge from MLLMs, which is insightful.\n\n# Weaknesses\n\nIn Contribution 2, the author presents an important finding: state-of-the-art model editing methods like LoRA fine-tuning cannot fully delete knowledge from MLLMs. However, there are several issues with this finding:\n\n1.\tComparison with Full-Parameter Fine-Tuning: The comparison between LoRA and full-parameter fine-tuning is missing. Given that LoRA only updates a small number of model parameters, it remains unclear whether full-parameter fine-tuning could delete knowledge from MLLMs.\n\n2.\tAblation Study Analysis for LoRA: Would increasing the rank size $r$ potentially delete knowledge from MLLMs? A more in-depth analysis of the ablation studies on LoRA is needed.\n\n3.\tOther Fine-Tuning Methods: It should be clarified whether other methods for fine-tuning MLLMs, such as Adalora or Adapter, could achieve similar effects.\n\n4.\tThe Neighborhood Image only includes \"Easy\" and \"Hard,\" which differs from other settings. It is suggested that the authors explain the reason for this.\n\n5.\tStandards that should be further clarified in Section 3.4.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces UnLOK-VQA, a novel multimodal unlearning benchmark designed to evaluate the deletion of specific multimodal knowledge from MLLMs. Building on this, the paper presents an extensive evaluation framework that includes attack and defense strategies to assess the effectiveness of various unlearning methods. It highlights that the latest editing methods like LoRA fine-tuning cannot fully delete knowledge from MLLMs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The introduction of RLHF-related work on the second page of the manuscript is necessary. Moreover, while the author refers to in-context relearning, there is an apparent lack of a comprehensive introduction and corresponding literature citations to support this concept.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are happy with the author's response and how they address it in the revised version.

The paper can be accepted in its current form.
Decision: Accept (poster)