GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs

## Abstract

Abstract Object hallucination in Multimodal Large Language Models (MLLMs) is a persistent failure mode that causes the model to perceive objects absent in the image.
This weakness of MLLMs is currently studied using static benchmarks with fixed visual scenarios, which preempts the possibility of uncovering model-specific or unanticipated hallucination vulnerabilities.
We introduce GHOST ( G enerating H allucinations via O ptimizing S tealth T okens), a method designed to stress-test MLLMs by actively generating images that induce hallucination. GHOST is fully automatic and requires no human supervision or prior knowledge. It operates by optimizing in the image embedding space to mislead the model while keeping the target object absent, and then guiding a diffusion model conditioned on the embedding to generate natural-looking images. The resulting images remain visually natural and close to the original input, yet introduce subtle misleading cues that cause the model to hallucinate. We evaluate our method across a range of models, including reasoning models like GLM-4.1V-Thinking, and achieve a hallucination success rate exceeding 28%, compared to around 1% in prior data-driven discovery methods. We confirm that the generated images are both high-quality and object-free through quantitative metrics and human evaluation. Also, GHOST uncovers transferable vulnerabilities: images optimized for Qwen2.5-VL induce hallucinations in GPT-4o at a 66.5% rate. Finally, we show that fine-tuning on our images mitigates hallucination, positioning GHOST as both a diagnostic and corrective tool for building more reliable multimodal systems.

## 1 Introduction

Multimodal large language models (MLLMs) (Liu et al., 2024, 2023; Bai et al., 2025a; Team et al., 2025) have demonstrated impressive performance across a wide range of vision and language tasks, including image captioning, visual question answering, and multimodal reasoning. However, they suffer from a potentially critical failure mode: object hallucination, where the model incorrectly predicts the presence of an object that is not present in the image. Such errors raise concerns about the reliability of MLLMs in safety-sensitive applications and underscore the need for systematic methods to stress-test their visual robustness.

Existing evaluations of object hallucination in MLLMs typically rely on static benchmarks, fixed visual scenarios, and curated image sets (Li et al., 2023; Hu et al., 2023; Leng et al., 2024; Lovenia et al., 2023). While useful, these approaches constrain analysis to a fixed set of visual scenarios and fail to expose model-specific vulnerabilities. Moreover, they lack a systematic way to assess which types of images trigger hallucinations, making it difficult to determine whether these errors are isolated or indicative of deeper, structural failure modes. Identifying such patterns is crucial not only for understanding model behavior but also for mitigating these failures.

To address these limitations, we introduce GHOST (Generating Hallucinations via Optimizing Stealth Tokens), a method designed to stress-test MLLMs by synthesizing hallucination-inducing images. Given an input image and a target object, GHOST generates a visually similar image that preserves the absence of the object but causes the model to incorrectly detect it. The edits introduced by GHOST are subtle and semantic, contextual cues that are perceptible to humans but do not introduce the object itself. This is accomplished by optimizing the image’s CLIP embedding to elicit hallucination while regularizing against object insertion, then using it to guide the diffusion model to generate a natural-looking image.

Consider the example in Figure [1](https://arxiv.org/html/2509.25178v1#S1.F1). The original image (left) shows a banana on a plate with no knife present, and multiple MLLMs correctly answer “No” when asked whether a knife is in the picture. However, after GHOST introduces subtle changes, such as modifying the banana’s stem to resemble the edge of a knife, the models flip their answer to “Yes,” hallucinating a knife. These misleading cues are not convincing to human observers, yet they are sufficient to push the models across the decision boundary.

One key innovation in GHOST is its design that decouples the optimization process from the image generator. Prior work on systematic failure image generation either omits feedback from the target model, limiting their ability to capture model-specific blind spots (Zhang et al., 2024; Wu et al., 2024), or uses resource-intensive and slow pipelines (Augustin et al., 2025). In contrast, GHOST introduces a mapper that aligns the visual spaces of the target model and the diffusion model, enabling efficient optimization while preserving compatibility across different image generators and vision-language models.

Figure: Figure 1: (Left) All models correctly answer “No” when asked if there is a knife in the image. (Right) GHOST introduces subtle cues, and all models now hallucinate the presence of a knife.
Refer to caption: figs/FigureAbstractlegend.png

Together, these components make GHOST scalable and effective: for instance, on Qwen2.5-VL (Bai et al., 2025a), GHOST generates 2,816 hallucination-inducing images out of 9,423 initial inputs, a 29% success rate–compared to 0.1% in DASH (Augustin et al., 2025). GHOST also generalizes to reasoning-based MLLMs like GLM-4.1V-Thinking (Team et al., 2025), where it shifts the model’s reasoning trajectory to justify the presence of a nonexistent object (Figure [7](https://arxiv.org/html/2509.25178v1#S5.F7)). The resulting images are natural and high quality, as demonstrated by the FID results in Table [6](https://arxiv.org/html/2509.25178v1#S5.F6). Figure [2](https://arxiv.org/html/2509.25178v1#S1.F2) shows qualitative examples, where each column compares an original image (top) with its GHOST-edited counterpart (bottom). The subtle semantic changes introduced by GHOST cause the model to hallucinate its presence. Human evaluation confirms that GHOST images remain object-free from a human perspective, with 89% of responses agreeing the target object is absent.

GHOST also reveals transferable vulnerabilities: images optimized for one model often cause hallucinations in others. For example, images generated for Qwen2.5-VL induce hallucinations in GPT-4o at a 66.5% rate (Table [3](https://arxiv.org/html/2509.25178v1#S5.T3)). GHOST thus exposes not just model-specific blind spots, but systemic vulnerabilities across MLLMs.

Finally, we demonstrate that GHOST can also aid in mitigation: fine-tuning on GHOST synthetic images improves model robustness on downstream hallucination benchmarks (Section [6.2](https://arxiv.org/html/2509.25178v1#S6.SS2)), highlighting GHOST’s potential as both a diagnostic and corrective tool.

Figure: Figure 2: (Top) Input images, the MLLM does not hallucinate the target object. (Bottom) GHOST images, the MLLM hallucinates the object, despite its absence being clear to a human observer.
Refer to caption: figs/FigureIntrodMultiple.jpg

## 2 Related works

Systematic Failures in MLLMs. Recent works have introduced frameworks to automatically identify failures in MLLMs (Tong et al., 2024a, b; Hosseini et al., 2025). Tong et al. (2024b) highlight MLLMs’ struggles with basic visual understanding, attributing them to limitations in CLIP-based vision encoders. In contrast, Tong et al. (2024a) focuses on failures stemming from the language modality. Hosseini et al. (2025) show that MLLMs often over-rely on spurious visual cues, leading to hallucinations when such cues are present but the object itself is absent. A large body of work has specifically examined object hallucination (Li et al., 2023; Leng et al., 2024; Lovenia et al., 2023; Hu et al., 2023; Wang et al., 2023), typically proposing benchmarks and evaluation protocols to quantify the issue. However, these approaches rely on curated image sets, constraining analysis to a fixed set of visual scenarios. In contrast, our method actively generates failure images, uncovering novel and unanticipated visuals that are unlikely to appear in static datasets.

Discovering Hallucinations through Image Generation. Some prior studies (Wu et al., 2024; Zhang et al., 2024) evaluate MLLMs on synthesized images generated via carefully designed prompts using text-to-image models. While these methods can reveal general failure modes, they do not incorporate feedback from the MLLM itself, limiting their ability to uncover model-specific vulnerabilities or adapt flexibly across models. In contrast, GHOST leverages model feedback and performs targeted optimization in the CLIP image space.

DASH (Augustin et al., 2025) is the most comparable prior work, as it also incorporates MLLM feedback for image generation. However, it operates directly over diffusion latents and requires the MLLM, the diffusion model, the generated image, and an object detector to all remain in the optimization loop, making it both time and resource intensive and forcing the use of a distilled single-step diffusion model. Moreover, DASH ultimately retrieves similar images from the real dataset based on the generated ones. GHOST addresses these limitations with a decoupled and more efficient design. In Section [5](https://arxiv.org/html/2509.25178v1#S5) and Appendix [B](https://arxiv.org/html/2509.25178v1#A2), we provide comparisons with DASH.

Robustness of Diffusion Representations. It is well-established that deep networks trained with empirical risk minimization (ERM) often rely on spurious correlations to make predictions (Sagawa et al., 2019; Kirichenko et al., 2022; Noohdani et al., 2024; Parast et al., 2025). Recent studies have also identified such correlations in CLIP (Radford et al., 2021b) and autoregressive vision-language models (Wang et al., 2024a; Varma et al., 2024; Kim et al., 2023; Ye et al., 2024; Zheng et al., 2024). In contrast, diffusion models have shown greater robustness in their learned representations, as they aim to approximate the underlying data distribution rather than optimizing for a specific downstream task (Li et al., 2024; Luo et al., 2024). Li et al. (2024) demonstrate that U-Net-based latent diffusion models (Rombach et al., 2022) are less prone to shortcut solutions compared to discriminative models. DEEM (Luo et al., 2024) further argues that vision encoders in MLLMs, often based solely on models like CLIP-ViT, can be improved using the generative feedback of diffusion models to align the semantic distributions of the image encoder. They showed this alignment enhances robustness to out-of-distribution inputs and reduces visual hallucinations. Motivated by these insights, we leverage diffusion models to probe vulnerabilities in MLLMs.

Adversarial Attacks on MLLMs. GHOST can be viewed as a form of adversarial generation, but we highlight key distinctions from prior work. Methods like AnyAttack (Zhang et al., 2025a) and AttackVLM (Zhao et al., 2023) operate in pixel space and preserve the overall image semantics, aiming for imperceptible perturbations. In contrast, GHOST inserts semantic-level misleading cues, plausible to humans yet triggering hallucinations in MLLMs. While adversarial examples are sometimes transferable across models, GHOST exhibits a different kind of transfer: cues crafted for one model generalize to others, pointing to shared failure modes and spurious biases. Our approach is closer in spirit to content-based adversarial attacks such as Chen et al. (2023), but differs in both setting (image classifiers vs. MLLMs) and objective (causing hallucination rather than misclassification).

## 3 Problem Setup

We consider an MLLM $\mathcal{M}$ that integrates a vision encoder $\mathcal{V}_{\mathcal{M}}$ with a language model backbone $f_{\mathcal{M}}$. The text input is denoted as $\mathbf{X}_{q}\in\mathcal{L}$, where $\mathcal{L}$ is the space of language, and the image input as $\mathbf{X}_{v}\in\mathcal{I}$, where $\mathcal{I}$ is the space of images. The vision encoder $\mathcal{V}_{\mathcal{M}}:\mathcal{I}\to\mathbb{R}^{N\times d_{\mathcal{M}}}$ maps the image $\mathbf{X}_{v}$ into a sequence of $N$ vision tokens of dimension $d_{\mathcal{M}}$, denoted by $\mathbf{Z}_{v}=\mathcal{V}_{\mathcal{M}}(\mathbf{X}_{v})$. These tokens, together with the text input $\mathbf{X}_{q}$, are passed to the LLM backbone $f_{\mathcal{M}}$, which generates an output sequence $Y=(y_{1},\dots,y_{T})$ autoregressively. Each token $y_{t}$ is sampled from the conditional probability
$p(y_{t}\mid y_{<t},\mathbf{X}_{q},\mathbf{Z}_{v})=f_{\mathcal{M}}(y_{<t},\mathbf{X}_{q},\mathbf{Z}_{v})$.

We also consider a Latent Diffusion Model $\mathcal{G}$, which generates an image $\tilde{\mathbf{X}}_{v}=\mathcal{G}(\mathbf{X}_{v}|\>c)$ given an initial image $\mathbf{X}_{v}$ and a conditioning signal $c$. We employ Stable Diffusion unCLIP (Rombach et al., 2022), where $c$ is an embedding in the CLIP embedding space. Formally, let $\mathcal{V}_{\text{CLIP}}:\mathcal{I}\cup\mathcal{L}\to\mathbb{R}^{d_{\text{CLIP}}}$ denote the CLIP encoder that learned joint representation for text and images. Note that $\mathcal{M}$ and $\mathcal{G}$ use different vision encoders; therefore, we introduce a mapper $\Pi:\mathbb{R}^{d_{\text{CLIP}}}\to\mathbb{R}^{N\times d_{\mathcal{M}}}$ to bridge these embedding spaces.
Given an initial image $\mathbf{X}_{v}$ without the target object $t$, a prompt $\mathbf{X}_{q}$ of the form “Do you see a $t$ in the image?”, and a target token $y^{*}$ (e.g.,“Yes”), we aim to find an embedding $c\in\mathbb{R}^{d_{\text{CLIP}}}$ that satisfies the following constraints:

- 1.
Proximity to the original image: $c$ remains close to the initial embedding $c_{0}=\mathcal{V}_{\text{CLIP}}(\mathbf{X}_{v})$
- 2.
Absence of the target object semantics: $c$ doesn’t encode the target object itself.
- 3.
Inducing hallucination: $\mathcal{M}$ assigns high probability to answering the target token $y^{*}$
$p(y^{*}\mid\mathbf{X}_{q},\Pi(c))\;\geq\;\tau_{\text{yes}}$
(1)
where $\tau_{\text{yes}}$ is a confidence threshold.

We condition the diffusion model on the optimized embedding $c$ to generate an image
$\tilde{\mathbf{X}}_{v}=\mathcal{G}(\mathbf{X}_{v}|\>c)$ and check whether $\mathcal{M}$ hallucinates the target object. Diffusion unCLIP enables this process by allowing the image generation to be conditioned on a CLIP image embedding. Full pipeline and implementation details are provided in Section [4](https://arxiv.org/html/2509.25178v1#S4).

## 4 Method

This section introduces GHOST (Generating Hallucinations via Optimizing Stealth Tokens), a fully automated pipeline for stress-testing object hallucinations in MLLMs. We begin with the implementation of the mapper $\Pi$, then describe our optimization procedure, and finally explain how the diffusion model generates images within the pipeline.

### 4.1 Bridging Embedding Spaces

To incorporate feedback from the MLLM into the generation process, a naive approach would require backpropagating through the entire pipeline – including the MLLM, the generated image, and the diffusion model– which is computationally expensive. To overcome this, we introduce a mapper ($\Pi$) that bridges the embedding spaces of the MLLM and the diffusion model, enabling more efficient optimization. We implement $\Pi$ as an MLP and find that this simple design is sufficient for our purposes. Given an MLLM $\mathcal{M}$ and a dataset $\mathcal{D}$ of images, the MLP is trained with a mean squared error (MSE) objective:

$$ $\mathcal{L}_{\text{align}}=||\Pi(\mathcal{V}_{\text{CLIP}}(\mathbf{X}v))-\mathcal{V}_{\mathcal{M}}(\mathbf{X}_{v})||_{2}^{2},$ (2) $$

where $\mathbf{X}_{v}\in\mathcal{D}$. The training setup is illustrated in Figure [3](https://arxiv.org/html/2509.25178v1#S4.F3)b. For more details see Appendix [C](https://arxiv.org/html/2509.25178v1#A3).

### 4.2 Optimization Objective

Figure: Figure 3: a) Overview of GHOST. We optimize only the CLIP embedding, then condition unCLIP on it, see (c) for decoding details. b) Training setup for the MLP, which aligns CLIP embeddings with the MLLM vision encoder using an MSE loss. c) A partially noised latent of the original image is denoised conditioned on the optimized embedding.
Refer to caption: figs/MainFigureCondition.jpg

With the mapper $\Pi$ trained, we now introduce the attack. Given a target object $t$ and an initial image $\mathbf{X}_{v}$ that do not contain $t$, we optimize the CLIP embedding $z=\mathcal{V}_{\text{CLIP}}(\mathbf{X}_{v})$ so that it tends to satisfy the conditions from Section [3](https://arxiv.org/html/2509.25178v1#S3).

Proximity to the original image: We regularize $c$ to remain close to the embedding of the initial real image. This prevents excessive drift and preserves high-level semantics. Formally, let $c_{0}$ be the CLIP embedding of the original image, we impose an $\ell_{2}$ penalty:

$$ $\mathcal{L}_{reg}=\bigl\|c-c_{0}\bigr\|_{2}^{2}$ (3) $$

Absence of the target object semantics: To discourage $c$ from encoding the target object directly, we penalize its similarity to the CLIP embedding of the object’s textual descriptions. Rather than relying on a single prompt, we define a set of textual templates $\mathcal{T}_{clip}$ (e.g., “a photo of a $t$”) to represent the target object (See Appendix [E.2](https://arxiv.org/html/2509.25178v1#A5.SS2) for more details). We compute the expected cosine similarity between $c$ and the CLIP embeddings of these templates:

$$ $\mathcal{L}_{\text{clip}}=\mathbb{E}_{\textbf{T}_{q}\sim\mathcal{T}_{\text{clip}}}\left[\cos(c,\mathcal{V}_{\text{CLIP}}(\mathbf{T}_{q}))\right]$ (4) $$

Inducing hallucination:
To mislead the model to perceive the target object, we see the probability that the LLM assigns to the target token $y^{*}$. Specifically, given a query prompt $\mathbf{X}_{q}$ of the form “Do you see a $t$ in the image?”, we define the loss as:

$$ $\mathcal{L}_{\text{adv}}=-\log p\!\left(y^{\star}\;\middle|\;\mathbf{X}_{q},\,\Pi(c)\right)$ (5) $$

To prevent overfitting to a single phrasing, we construct a diverse set of semantically equivalent query templates and randomly sample $\mathbf{X}_{q}$ from this set at each optimization step (The templates are provided in Appendix[E.4](https://arxiv.org/html/2509.25178v1#A5.SS4)).

Our joint objective combines all components:

$$ $\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{adv}}+\lambda_{clip}\,\mathcal{L}_{\text{clip}}+\lambda_{reg}\,\mathcal{L}_{\text{reg}},$ (6) $$

where $\lambda_{clip}$, $\lambda_{reg}$ are hyperparameters that control the relative importance of these terms. We minimize this loss using the AdamW (Loshchilov and Hutter, 2017) optimizer. An overview of this pipeline is illustrated in Figure [3](https://arxiv.org/html/2509.25178v1#S4.F3)a, and ablation studies are provided in Section [6](https://arxiv.org/html/2509.25178v1#S6).

After each optimization step, we check whether the model’s probability for the target token exceeds the threshold confidence $\tau_{yes}$. We perform at most $M$ optimization steps per image. If the probability does not surpass $\tau_{\text{yes}}$ within $M$ steps, we discard the result and move to the next image. Otherwise, once the threshold is met, we use the current embedding $z$ to generate a candidate image via the diffusion model, described in the next section.

### 4.3 Guided Diffusion

As shown in Figure [3](https://arxiv.org/html/2509.25178v1#S4.F3)c, instead of starting from pure noise, we begin the reverse process from a partially noised latent of the original image. This design encourages the final image to preserve the high-level structure of the input while still allowing for subtle semantic shifts that can induce hallucination. Concretely, we encode the original image $\mathbf{X}_{v}$ into its VAE latent representation, apply forward diffusion for $t$ steps to obtain $z_{t}$, and then condition the reverse denoising on $c$. The noise level $t$ controls the trade-off between preserving the original image structure and allowing space for misleading cues. To account for diffusion stochasticity, we allow up to $N=4$ generation attempts per image. More details are provided in Appendix [D](https://arxiv.org/html/2509.25178v1#A4).

We use an open-vocabulary object detector, OWLv2 (Minderer et al., 2024), as a filtering step to verify that the generated image does not actually contain the target object. If OWLv2 detects the target object, we conservatively discard the sample. Only when the model hallucinates the object and OWLv2 confirms its absence do we consider the hallucination-inducing process successful. Figure [4](https://arxiv.org/html/2509.25178v1#S4.F4) provides a qualitative example: as we optimize the CLIP embedding to increase the model’s confidence in answering “Yes,” the images gradually incorporate misleading visual features, in this case, vase-like structures. Note that this figure is illustrative only; we do not generate images at each step of optimization.

Figure: Figure 4: Optimization steps toward hallucination. We show the model’s Yes/No probabilities for the optimized embedding at each step. Images are then generated by conditioning diffusion on that embedding. As “Yes” confidence increases, misleading cues (e.g., vase-like structures) emerge. Samples flagged by OWLv2 are discarded.
Refer to caption: figs/GRADUALNew.jpg

## 5 Experiments

In this section, we present both quantitative and qualitative results of GHOST on three open-source MLLMs. We also evaluate the transferability of GHOST-generated images across a diverse set of additional models.

### 5.1 Results

Models. We evaluate GHOST on three open-source MLLMs accessed via HuggingFace: Qwen2.5-VL-7B-Instruct (Bai et al., 2025a), LLaVA-v1.6-Mistral-7B (Liu et al., 2024), and GLM-4.1V-Thinking (Team et al., 2025).

Dataset. We select 10 target object classes from the COCO dataset (Lin et al., 2014). For each class, we begin by excluding all images containing the target object based on COCO annotations. Among the remaining negatives, we select the top 1,000 images with the highest CLIP similarity to the object name. This sorting identifies visually diverse yet semantically related contexts where hallucination is more likely to occur. In Appendix [E.1](https://arxiv.org/html/2509.25178v1#A5.SS1), we also evaluate GHOST without CLIP sorting and show that our results remain consistent. To isolate GHOST-induced failures, we discard any image where the MLLM already predicts the target object before optimization.

Results. We present the GHOST results in Table [5](https://arxiv.org/html/2509.25178v1#S5.F5). For each model, “Samples” refers to the number of images (out of the initial 10,000 COCO images) in which the model does not perceive the target object. We then report how many hallucination-inducing images GHOST successfully generates, followed by the success rate (i.e., number of hallucinations divided by the number of samples). We also include results from DASH (Augustin et al., 2025), applied to the COCO dataset. Despite operating on a much smaller image pool, GHOST discovers orders of magnitude more hallucination-inducing samples. For instance, GHOST identifies 2,816 successful cases for Qwen2.5-VL, while DASH-LLM and DASH-OPT combined find only 99. It is important to note that DASH was originally designed to search over ReLAION-5B (LAION, 2024), a massive web-scale dataset. Its goal is not to maximize hallucination count, but to uncover naturally occurring failure cases across large-scale image corpora. Our comparison on COCO is intended to provide a point of reference against an existing method that shares a similar objective: identifying systematic hallucination-inducing inputs. Further details about our DASH experiments are provided in Appendix [B](https://arxiv.org/html/2509.25178v1#A2).

We also break down GHOST’s performance by object class in Figure [5](https://arxiv.org/html/2509.25178v1#S5.F5). GHOST discovers a substantial number of hallucination cases for all classes. More details on the class-wise results of our method are provided in Appendix [F](https://arxiv.org/html/2509.25178v1#A6).

Figure: Table 1: GHOST and DASH (Augustin et al., 2025) results on COCO. “Samples” reflects the size of the input pool each method operates over.
Refer to caption: figs/qwengensam.png

Image Quality. To evaluate the quality of the samples generated by GHOST, we conduct both qualitative and quantitative analyses. Figure [6](https://arxiv.org/html/2509.25178v1#S5.F6) presents visual comparisons between GHOST and two baseline diffusion methods: Stable Diffusion v2.1 (Rombach et al., 2022), where we prompt the model using the COCO caption of the image, and Stable Diffusion unCLIP, where the diffusion is conditioned on the CLIP embedding of the image. GHOST preserves the semantics of the original image while introducing minimal visual artifacts. Table [6](https://arxiv.org/html/2509.25178v1#S5.F6) reports Fréchet Inception Distance, FID scores (Heusel et al., 2017), evaluating each method along two axes: (i) realism, measured by FID against COCO validation images, and (ii) semantic fidelity, measured by FID against the initial images used in diffusion. While GHOST achieves comparable realism to baseline methods, it outperforms them in semantic preservation, confirming its advantage in maintaining visual identity. We observe higher FID scores for GLM hallucinations, likely due to the model’s reasoning nature. Since we optimize a “thinking” token rather than the final answer directly, the optimization path is less aligned with the hallucination objective, making it harder to induce consistent visual drift. Also more qualitative samples are provided in Appendix [F](https://arxiv.org/html/2509.25178v1#A6).

Figure: Figure 6: Qualitative comparison of generated images.
Refer to caption: figs/FigureFIDresized.jpg

Reasoning Model. We evaluate GHOST on GLM-4.1V-Thinking (Team et al., 2025), a multimodal model trained with a reasoning-centric framework. The model structures its responses in two stages: it first generates a reasoning trace enclosed within <think>…</think> tokens, followed by a final answer within <answer>…</answer>. To adapt GHOST to this format, we use the probability of the token "Yes" at the first decoding step after the <think> token. This allows us to keep the optimization objective and runtime consistent with other models. We find that, despite not explicitly optimizing for the final answer, this is sufficient to induce hallucinated "Yes" responses in the final output. A qualitative example is shown in Figure [7](https://arxiv.org/html/2509.25178v1#S5.F7), where GHOST edits shift the model’s reasoning trajectory to justify the presence of a nonexistent object.

Figure: Figure 7: Applying GHOST to a reasoning model (GLM-4.1V-Thinking).
Refer to caption: figs/GLMThinkingFig.jpg

Human Evaluation. To evaluate whether GHOST images preserve the absence of the target object from a human perspective, we asked 40 peers to review the images, resulting in over 3,000 total votes. They were shown the final GHOST images and asked whether they could see the target object. On average, 89% of responses for images optimized w.r.t LLaVA-v1.6 and 86.3% for Qwen2.5-VL indicated that the object was not present. For reference, DASH (Augustin et al., 2025) reports that 5.2% of their images contained the object and 7.8% were ambiguous, implying that 87% of their samples were perceived as object-free. Additional details are provided in Appendix [A](https://arxiv.org/html/2509.25178v1#A1).

### 5.2 Transferability

We evaluate how images generated by GHOST on one model transfer to others. In addition to the three source models, we include two larger open-source models: LLaMA3.2 11B (Meta, 2024) and Aya 32B (Dash et al., 2025), as well as two closed-source models: GPT-4o (OpenAI, 2024) and Gemini2.5-Flash (Comanici et al., 2025). As shown in Table [3](https://arxiv.org/html/2509.25178v1#S5.T3), images optimized on Qwen2.5-VL achieve a 66.5% hallucination success rate on GPT-4o. These results suggest that different models share common failure patterns. More details are provided in Appendix [F](https://arxiv.org/html/2509.25178v1#A6).

**Table 3: Transferability of hallucinations across models. Each row corresponds to the model used, and each column shows the success rate (%) on a target model.**
| Model | Qwen2.5-VL | LLaVA-v1.6 | GLM4.1V | GPT-4o | Aya | LLaMA3.2 | Gemini |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Qwen2.5-VL | – | 62.2 | 72.0 | 66.5 | 71.1 | 65.8 | 58.6 |
| LLaVA-v1.6 | 52.6 | – | 50.5 | 50.5 | 54.4 | 49.7 | 42.8 |
| GLM4.1V | 63.2 | 57.1 | – | 63.8 | 67.6 | 69.1 | 53.8 |

## 6 Ablation Studies

### 6.1 Effect of Hyperparameters

Figure: Figure 8: Larger $\lambda_{\text{clip}}$ leads to less objects.
Refer to caption: figs/ablationclip2.png

Effect of $\tau$.
We study how the confidence threshold $\tau$ impacts GHOST’s success by varying $\tau\in\{0.5,0.6,0.7,0.8,0.9\}$ on 400 images from two objects (200 images each), while keeping the number of optimization steps fixed at 100 and using the same learning rate. A higher $\tau$ requires the optimized embedding to achieve higher $p(y^{*}\mid\mathbf{X}_{q},\Pi(c))$, making optimization harder. As expected, this reduces the number of images that meet the threshold within the allowed steps (see Appendix [E.7](https://arxiv.org/html/2509.25178v1#A5.SS7)). However, as shown in Figure [9](https://arxiv.org/html/2509.25178v1#S6.F9), images generated from embeddings optimized with higher $\tau$ are more likely to induce hallucination. These results confirm that the confidence score $p(y^{*}\mid\mathbf{X}_{q},\Pi(c))$ serves as a meaningful proxy for the model’s belief in object presence, even though the optimization is decoupled from the diffusion process.

Effect of $\lambda_{\text{clip}}$. We evaluate the impact of $\lambda_{\text{clip}}$ using the same setup. As shown in Figure [9](https://arxiv.org/html/2509.25178v1#S6.F9), increasing $\lambda_{\text{clip}}$ reduces the object detection rate per generated image. This confirms that stronger penalties on CLIP similarity discourage the diffusion model from generating images containing the object. Additional results are provided in Appendix [E.6](https://arxiv.org/html/2509.25178v1#A5.SS6).

Effect of $\lambda_{reg}$. As shown in Appendix [E.5](https://arxiv.org/html/2509.25178v1#A5.SS5), increasing $\lambda_{reg}\in\{1.0,1.5,2.0\}$ on GLM samples lowers FID scores, indicating that this term helps preserve visual realism.

### 6.2 Mitigation

As a small toy setup, we fine-tune Qwen2.5-VL using GHOST images. Following DASH (Augustin et al., 2025), we include both positive and negative samples to avoid forgetting of object concepts. However, unlike DASH, which uses real COCO images as positives, we generate both positives and negatives synthetically using Diffusion unCLIP to ensure consistency in visual style. This prevents the model from simply learning to reject all synthetic content.

Table [4](https://arxiv.org/html/2509.25178v1#S6.T4) shows that fine-tuning improves robustness to hallucination-inducing images generated by other models (LLaVA and GLM), despite never seeing them during training. We also observe improvements on the POPE (Li et al., 2023) benchmark, while DASH reports no gains on POPE. Finally, performance on general tasks such as VQAv2 and COCO captioning remains nearly unchanged, suggesting that fine-tuning likely preserves general capabilities.

**Table 4: Fine-tuning Qwen2.5-VL on GHOST images improves robustness to hallucination.**
|  | GHOST-LLaVA | GHOST-GLM | POPE | VQAv2 | Caption |
| --- | --- | --- | --- | --- | --- |
| Baseline | 52.6 | 63.1 | 88.5 | 73.5 | 89.5 |
| Finetuned | 7.0 | 7.3 | 90.7 | 73.4 | 89.4 |

## 7 Conclusion

We introduce GHOST, a pipeline for generating images that induce hallucinations in MLLMs by inserting minimal cues. Our method leverages diffusion models to craft these cues, demonstrating that their vision space aligns well with the visual representations used by MLLMs. Beyond a single model, GHOST reveals cross-modal failure modes that are not detectable by prior works. We hope GHOST can help and inspire future work toward building more robust and reliable multimodal systems.

## Acknowledgments

This project was supported in part by a grant from an NSF CAREER AWARD 1942230, the ONR PECASE grant N00014-25-1-2378, ARO’s Early Career Program Award 310902-00001, Army Grant No. W911NF2120076, the NSF award CCF2212458, NSF Award No. 2229885 (NSF Institute for Trustworthy AI in Law and Society, TRAILS), a MURI grant 14262683, DARPA AIQ DARPA AIQ grant HR00112590066 and an award from meta 314593-00001. Naveed Akhtar is a recipient of the Australian Research
Council Discovery Early Career Researcher Award (project
# DE230101058) funded by the Australian Government. This work is also partially supported by Google Research Scholar Program Award.

## References

- Augustin et al. [2025]
Maximilian Augustin, Yannic Neuhaus, and Matthias Hein.
Dash: Detection and assessment of systematic hallucinations of vlms.
In *ICCV*, 2025.
- Azizmalayeri et al. [2022]
Mohammad Azizmalayeri, Arshia Soltani Moakhar, Arman Zarei, Reihaneh Zohrabi, Mohammad Manzuri, and Mohammad Hossein Rohban.
Your out-of-distribution detection method is not robust!
*Advances in Neural Information Processing Systems*, 35:4887–4901, 2022.
- Bai et al. [2025a]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin.
Qwen2.5-vl technical report.
*arXiv preprint arXiv:2502.13923*, 2025a.
- Bai et al. [2025b]
Tianyi Bai, Yuxuan Fan, Jiantao Qiu, Fupeng Sun, Jiayi Song, Junlin Han, Zichen Liu, Conghui He, Wentao Zhang, and Binhang Yuan.
Hallucination at a glance: Controlled visual edits and fine-grained multimodal learning.
*arXiv preprint arXiv:2506.07227*, 2025b.
- Chen et al. [2023]
Zhaoyu Chen, Bo Li, Shuang Wu, Kaixun Jiang, Shouhong Ding, and Wenqiang Zhang.
Content-based unrestricted adversarial attack.
*Advances in Neural Information Processing Systems*, 36:51719–51733, 2023.
- Comanici et al. [2025]
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al.
Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.
*arXiv preprint arXiv:2507.06261*, 2025.
- Dash et al. [2025]
Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian, Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller-Morales, et al.
Aya vision: Advancing the frontier of multilingual multimodality.
*arXiv preprint arXiv:2505.08751*, 2025.
- Heusel et al. [2017]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.
Gans trained by a two time-scale update rule converge to a local nash equilibrium.
*Advances in neural information processing systems*, 30, 2017.
- Ho et al. [2020]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models, 2020.
URL [https://arxiv.org/abs/2006.11239](https://arxiv.org/abs/2006.11239).
- Hosseini et al. [2025]
Parsa Hosseini, Sumit Nawathe, Mazda Moayeri, Sriram Balasubramanian, and Soheil Feizi.
Seeing what’s not there: Spurious correlation in multimodal llms.
*arXiv preprint arXiv:2503.08884*, 2025.
- Hu et al. [2022]
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.
LoRA: Low-rank adaptation of large language models.
In *International Conference on Learning Representations*, 2022.
URL [https://openreview.net/forum?id=nZeVKeeFYf9](https://openreview.net/forum?id=nZeVKeeFYf9).
- Hu et al. [2023]
Hongyu Hu, Jiyuan Zhang, Minyi Zhao, and Zhenbang Sun.
Ciem: Contrastive instruction evaluation method for better instruction tuning.
*arXiv preprint arXiv:2309.02301*, 2023.
- Kim et al. [2023]
Jae Myung Kim, A Koepke, Cordelia Schmid, and Zeynep Akata.
Exposing and mitigating spurious correlations for cross-modal retrieval.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 2585–2595, 2023.
- Kirichenko et al. [2022]
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson.
Last layer re-training is sufficient for robustness to spurious correlations.
*arXiv preprint arXiv:2204.02937*, 2022.
- LAION [2024]
LAION.
Releasing re-laion-5b: transparent iteration on laion-5b with additional safety fixes.
[https://laion.ai/blog/relaion-5b/](https://laion.ai/blog/relaion-5b/), 2024.
Accessed: 30 aug, 2024.
- Leng et al. [2024]
Sicong Leng, Yun Xing, Zesen Cheng, Yang Zhou, Hang Zhang, Xin Li, Deli Zhao, Shijian Lu, Chunyan Miao, and Lidong Bing.
The curse of multi-modalities: Evaluating hallucinations of large multimodal models across language, visual, and audio.
*arXiv preprint arXiv:2410.12787*, 2024.
- Li et al. [2024]
Alexander Cong Li, Ananya Kumar, and Deepak Pathak.
Generative classifiers avoid shortcut solutions.
In *The Thirteenth International Conference on Learning Representations*, 2024.
- Li et al. [2023]
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen.
Evaluating object hallucination in large vision-language models.
*arXiv preprint arXiv:2305.10355*, 2023.
- Lin et al. [2014]
Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.
Microsoft COCO: common objects in context.
In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, *Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V*, volume 8693 of *Lecture Notes in Computer Science*, pages 740–755. Springer, 2014.
doi: 10.1007/978-3-319-10602-1\_48.
URL [https://doi.org/10.1007/978-3-319-10602-1_48](https://doi.org/10.1007/978-3-319-10602-1_48).
- Liu et al. [2023]
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.
Visual instruction tuning.
In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.
URL [http://papers.nips.cc/paper_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html](http://papers.nips.cc/paper_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html).
- Liu et al. [2024]
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee.
Llava-next: Improved reasoning, ocr, and world knowledge, January 2024.
URL [https://llava-vl.github.io/blog/2024-01-30-llava-next/](https://llava-vl.github.io/blog/2024-01-30-llava-next/).
- Loshchilov and Hutter [2017]
Ilya Loshchilov and Frank Hutter.
Decoupled weight decay regularization.
*arXiv preprint arXiv:1711.05101*, 2017.
- Lovenia et al. [2023]
Holy Lovenia, Wenliang Dai, Samuel Cahyawijaya, Ziwei Ji, and Pascale Fung.
Negative object presence evaluation (nope) to measure object hallucination in vision-language models.
*arXiv preprint arXiv:2310.05338*, 2023.
- Luo et al. [2024]
Run Luo, Yunshui Li, Longze Chen, Wanwei He, Ting-En Lin, Ziqiang Liu, Lei Zhang, Zikai Song, Xiaobo Xia, Tongliang Liu, et al.
Deem: Diffusion models serve as the eyes of large language models for image perception.
*arXiv preprint arXiv:2405.15232*, 2024.
- Meta [2024]
Meta.
Llama-3.2-11b-vision-instruct, 2024.
URL [https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct).
Accessed: 2025-01-14.
- Minderer et al. [2024]
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby.
Scaling open-vocabulary object detection, 2024.
URL [https://arxiv.org/abs/2306.09683](https://arxiv.org/abs/2306.09683).
- Mirzaei et al. [2025]
Hossein Mirzaei, Mohammad Jafari, Hamid Reza Dehbashi, Ali Ansari, Sepehr Ghobadi, Masoud Hadi, Arshia Soltani Moakhar, Mohammad Azizmalayeri, Mahdieh Soleymani Baghshah, and Mohammad Hossein Rohban.
Rodeo: Robust outlier detection via exposing adaptive out-of-distribution samples.
*arXiv preprint arXiv:2501.16971*, 2025.
- Moakhar et al. [2024]
Arshia Soltani Moakhar, Eugenia Iofinova, Elias Frantar, and Dan Alistarh.
Spade: Sparsity-guided debugging for deep neural networks, 2024.
URL [https://arxiv.org/abs/2310.04519](https://arxiv.org/abs/2310.04519).
- Noohdani et al. [2024]
Fahimeh Hosseini Noohdani, Parsa Hosseini, Aryan Yazdan Parast, Hamidreza Yaghoubi Araghi, and Mahdieh Soleymani Baghshah.
Decompose-and-compose: A compositional approach to mitigating spurious correlation.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 27662–27671, 2024.
- OpenAI [2024]
OpenAI.
Gpt-4 technical report, 2024.
URL [https://arxiv.org/abs/2303.08774](https://arxiv.org/abs/2303.08774).
- Parast et al. [2025]
Aryan Yazdan Parast, Basim Azam, and Naveed Akhtar.
Ddb: Diffusion driven balancing to address spurious correlations, 2025.
URL [https://arxiv.org/abs/2503.17226](https://arxiv.org/abs/2503.17226).
- Radford et al. [2021a]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.
Learning transferable visual models from natural language supervision, 2021a.
URL [https://arxiv.org/abs/2103.00020](https://arxiv.org/abs/2103.00020).
- Radford et al. [2021b]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International conference on machine learning*, pages 8748–8763. PMLR, 2021b.
- Rombach et al. [2022]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 10684–10695, June 2022.
- Sagawa et al. [2019]
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang.
Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.
*arXiv preprint arXiv:1911.08731*, 2019.
- Sohl-Dickstein et al. [2015]
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli.
Deep unsupervised learning using nonequilibrium thermodynamics, 2015.
URL [https://arxiv.org/abs/1503.03585](https://arxiv.org/abs/1503.03585).
- Team et al. [2025]
V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang, Da Yin, Fan Yang, Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang, Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong, Shiyu Huang, Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng, Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong, Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang, Yadong Xue, Yanfeng Wang, Yanzi Wang, Yifan An, Yifan Du, Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li,
Minlie Huang, Yuxiao Dong, and Jie Tang.
Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning, 2025.
URL [https://arxiv.org/abs/2507.01006](https://arxiv.org/abs/2507.01006).
- Tong et al. [2024a]
Shengbang Tong, Erik Jones, and Jacob Steinhardt.
Mass-producing failures of multimodal systems with language models.
*Advances in Neural Information Processing Systems*, 36, 2024a.
- Tong et al. [2024b]
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie.
Eyes wide shut? exploring the visual shortcomings of multimodal llms.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 9568–9578, 2024b.
- Varma et al. [2024]
Maya Varma, Jean-Benoit Delbrouck, Zhihong Chen, Akshay Chaudhari, and Curtis Langlotz.
Ravl: Discovering and mitigating spurious correlations in fine-tuned vision-language models.
*arXiv preprint arXiv:2411.04097*, 2024.
- Wang et al. [2023]
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang.
An llm-free multi-dimensional benchmark for mllms hallucination evaluation.
*CoRR*, 2023.
- Wang et al. [2024a]
Qizhou Wang, Yong Lin, Yongqiang Chen, Ludwig Schmidt, Bo Han, and Tong Zhang.
Do CLIP models always generalize better than imagenet models?
In *The Thirty-eighth Annual Conference on Neural Information Processing Systems*, 2024a.
URL [https://openreview.net/forum?id=wWyumwEYV8](https://openreview.net/forum?id=wWyumwEYV8).
- Wang et al. [2024b]
Zhecan Wang, Garrett Bingham, Adams Wei Yu, Quoc V Le, Thang Luong, and Golnaz Ghiasi.
Haloquest: A visual hallucination dataset for advancing multimodal reasoning.
In *European Conference on Computer Vision*, pages 288–304. Springer, 2024b.
- Wu et al. [2024]
Xiyang Wu, Tianrui Guan, Dianqi Li, Shuaiyi Huang, Xiaoyu Liu, Xijun Wang, Ruiqi Xian, Abhinav Shrivastava, Furong Huang, Jordan Lee Boyd-Graber, et al.
Autohallusion: Automatic generation of hallucination benchmarks for vision-language models.
*arXiv preprint arXiv:2406.10900*, 2024.
- Wu et al. [2025]
Yuanchen Wu, Lu Zhang, Hang Yao, Junlong Du, Ke Yan, Shouhong Ding, Yunsheng Wu, and Xiaoqiang Li.
Antidote: A unified framework for mitigating lvlm hallucinations in counterfactual presupposition and object perception.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pages 14646–14656, 2025.
- Ye et al. [2024]
Wenqian Ye, Guangtao Zheng, Yunsheng Ma, Xu Cao, Bolin Lai, James M Rehg, and Aidong Zhang.
Mm-spubench: Towards better understanding of spurious biases in multimodal llms.
*arXiv preprint arXiv:2406.17126*, 2024.
- Zhang et al. [2024]
Chenshuang Zhang, Fei Pan, Junmo Kim, In So Kweon, and Chengzhi Mao.
Imagenet-d: Benchmarking neural network robustness on diffusion synthetic object.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21752–21762, 2024.
- Zhang et al. [2025a]
Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Chen Yunhao, Jitao Sang, and Dit-Yan Yeung.
Anyattack: Towards large-scale self-supervised adversarial attacks on vision-language models.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2025a.
- Zhang et al. [2025b]
Jusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang, and Keze Wang.
Cf-vlm: Counterfactual vision-language fine-tuning.
*arXiv preprint arXiv:2506.17267*, 2025b.
- Zhang* et al. [2020]
Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi.
Bertscore: Evaluating text generation with bert.
In *International Conference on Learning Representations*, 2020.
URL [https://openreview.net/forum?id=SkeHuCVFDr](https://openreview.net/forum?id=SkeHuCVFDr).
- Zhao et al. [2023]
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin.
On evaluating adversarial robustness of large vision-language models.
In *Thirty-seventh Conference on Neural Information Processing Systems*, 2023.
- Zheng et al. [2024]
Guangtao Zheng, Wenqian Ye, and Aidong Zhang.
Benchmarking spurious bias in few-shot image classifiers.
In *European Conference on Computer Vision*, pages 346–364. Springer, 2024.

## Appendix A Human Evaluation

To evaluate whether GHOST-generated images preserve the absence of the target object from a human perspective, we design a simple interface and collect annotations from peers. For both LLaVA and Qwen2.5-VL, we randomly sample 50 successful GHOST images per object across 10 object categories. To validate annotator reliability, we include control images that are known to contain the target object. These control images are generated using the same Diffusion unCLIP model, but conditioned on positive samples, ensuring the object is clearly present. Examples are shown in Figure [18](https://arxiv.org/html/2509.25178v1#A7.F18). We include 50 control images per object.

Each annotator is shown a set of approximately 100 randomly selected images (comprising 20% control images, 40% GHOST–Qwen, and 40% GHOST–LLaVA) and asked “Is there a [target object] in this image?” with binary response options: “Yes” or “No.” Before beginning the evaluation, annotators completed a brief training phase (Figure [10](https://arxiv.org/html/2509.25178v1#A1.F10)) that displayed a few labeled examples to calibrate their expectations.

In total, we collect 1,590 votes on LLaVA images, 1,607 on Qwen images, and 1,056 on control images, from 40 unique participants. Aggregate results are shown in Table [5](https://arxiv.org/html/2509.25178v1#A1.T5), reporting the percentage of “Yes” responses in each group. Figure [11](https://arxiv.org/html/2509.25178v1#A1.F11) further breaks down the human yes-rate by object class. We used the human evaluation setup from Moakhar et al. (2024).

**Table 5: Human yes rate on control and GHOST-generated images.**
| Set of Images: | Control Images | GHOST–LLaVA | GHOST–Qwen |
| --- | --- | --- | --- |
| Human Yes Rate | 91.2% | 11.0% | 13.7% |

Figure: Figure 10: Human Evaluation Training Phase
Refer to caption: figs/training_humaneval.jpg

Figure: Figure 11: Human Yes-Rate by object class
Refer to caption: figs/human_classwise.png

## Appendix B DASH

We reproduce DASH results with Qwen2.5 and LLaVA. For LLaVA, we only evaluate DASH-LLM, a variant of DASH that relies solely on language model feedback. We omit DASH-OPT as it is too memory-intensive, even a batch size of 1 exceeds the limits of 80GB GPUs. For Qwen2.5, we evaluate both DASH-LLM and DASH-OPT. In DASH-OPT, we tune three parameters: step size, gradient clipping, and the VLM loss weight. In DASH-OPT, we tune three parameters: step size, gradient clipping, and the VLM loss weight. Specifically, we search step size and gradient clip values in $[0.1,0.2,0.4,0.8]$, and the VLM loss weight in $[1,2,4,6,8]$. As our evaluation metric, we count the number of images that do not contain the target object according to an object detector but still mislead the MLLM into hallucinating its presence. We are unable to use ReLION-5B, as its KNN search service is private. Additionally, computing CLIP embeddings over the full dataset is computationally infeasible for us. Instead, we use the COCO dataset for our experiments. The corresponding results are shown below.

**Table 6: The number of successful image found by DASH on COCO**
| Method | Number of Found Images |
| --- | --- |
| DASH-opt (qwen2.5) | 42 |
| DASH-LLM (qwen2.5) | 57 |
| DASH-LLM (llava) | 153 |

**Table 7: Number of successful image found by DASH on COCO by class name**
| Class/Method | DASH-opt (qwen2.5) | DASH-LLM (qwen2.5) | DASH-LLM (llava) |
| --- | --- | --- | --- |
| traffic light | 0 | 0 | 0 |
| carrot | 1 | 0 | 10 |
| knife | 0 | 2 | 14 |
| clock | 13 | 11 | 7 |
| toilet | 0 | 0 | 0 |
| boat | 2 | 0 | 0 |
| suitcase | 3 | 1 | 11 |
| bottle | 20 | 42 | 80 |
| vase | 3 | 0 | 23 |
| bus | 0 | 1 | 8 |

**Table 8: Successful images found by DASH on COCO by class name**
| Class/Method | DASH-opt (qwen2.5) | DASH-LLM (qwen2.5) | DASH-LLM (llava) |
| --- | --- | --- | --- |
| clock |  |  |  |
| suitcase |  |  |  |
| bottle |  |  |  |

## Appendix C Mapper

### C.1 Mapper Training

To bridge the embedding spaces, we introduce a mapper $\Pi$ that projects the CLS token from the CLIP vision encoder space $\mathcal{V}_{\text{CLIP}}$ into the vision encoder space of the target multimodal model. We design a simple multi-layer perceptron (MLP) to achieve this mapping. Specifically, the mapper takes the CLIP CLS token $\mathbf{z}_{\text{CLIP}}\in\mathbb{R}^{d_{\text{CLIP}}}$ as input and outputs a sequence of $N$ tokens, each of dimension $d_{\mathcal{M}}$, suitable for the target model:

$$ $\Pi:\mathbb{R}^{d_{\text{CLIP}}}\;\to\;\mathbb{R}^{N\times d_{\mathcal{M}}}.$ $$

Concretely, we first broadcast the CLS token across $N$ positions, obtaining

$$ $\mathbf{Z}_{\text{rep}}=[\mathbf{z}_{\text{CLIP}},\ldots,\mathbf{z}_{\text{CLIP}}]\in\mathbb{R}^{N\times d_{\text{CLIP}}}.$ $$

In parallel, we maintain a set of learnable context tokens

$$ $\mathbf{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{N}\},\quad\mathbf{e}_{i}\in\mathbb{R}^{d_{\mathcal{M}}},$ $$

which provide token-specific priors for the mapping. For each position $i$, we concatenate the repeated CLS token with its corresponding learnable context token:

$$ $\mathbf{h}_{i}=[\mathbf{z}_{\text{CLIP}};\mathbf{e}_{i}]\in\mathbb{R}^{d_{\text{CLIP}}+d_{\mathcal{M}}}.$ $$

The sequence $\mathbf{H}=\{\mathbf{h}_{1},\ldots,\mathbf{h}_{N}\}$ is then passed through an MLP with two hidden layers of size $d_{h}$, each followed by a GELU activation:

$$ $\mathbf{y}_{i}=\text{MLP}(\mathbf{h}_{i}),\quad\mathbf{Y}=\{\mathbf{y}_{1},\ldots,\mathbf{y}_{N}\}\in\mathbb{R}^{N\times d_{\mathcal{M}}}.$ $$

The learnable context embeddings $\mathbf{E}$ serve as token-specific conditioning vectors.
Although the MLP parameters are shared across all positions, the concatenation
$[\mathbf{z}_{\text{CLIP}};\mathbf{e}_{i}]$ ensures that each target token $i$ is shaped by both the
global CLIP semantics and its unique context embedding. For training the mapper, we used the AdamW optimizer with cosine annealing learning rate scheduling. The details of the training process and hyperparameters are summarized in Table [10](https://arxiv.org/html/2509.25178v1#A3.T10).

### C.2 Model Selection.

For model selection, we randomly sampled 100 images containing a given object class and 100 images without it from the COCO training set. We then evaluated the accuracy of the MLLM using the bridged embeddings obtained from the mapper $\Pi$, and the same prompts used in the main method (e.g. Do you see the object in the image). We considered six object classes: *vase*, *boat*, *bird*, *giraffe*, *car*. These classes were selected to cover a range of object characteristics: small objects such as *remote*, which require fine-grained detail to be detected from the mapper, and large objects such as *giraffe*, which are comparatively easier for the MLLM to recognize.
In Table [9](https://arxiv.org/html/2509.25178v1#A3.T9), we report the accuracy obtained with different settings of context dimension and hidden dimension. The final configuration selected for our experiments is highlighted in bold. We chose this setting by balancing accuracy with model size and efficiency, ensuring that the mapper achieves strong performance without incurring unnecessary computational cost.

**Table 9: Ablation on hidden dimension and context dimension for the projector, evaluated on LLaVA and Qwen. Numbers report accuracy (%). The final configuration selected for our experiments is highlighted in bold.**
|  | Hidden Dim | 512 | 1024 | 2048 | 4096 |
| --- | --- | --- | --- | --- | --- |
| LLaVA | Context = 6144 | 70.17 | 72.17 | 72.75 | 73.67 |
| Context = 4096 | 70.83 | 72.25 | 72.50 | 72.67 |  |
| Qwen | Context = 6144 | 69.50 | 71.08 | 73.50 | 73.92 |
| Context = 4096 | 68.08 | 71.67 | 73.92 | 72.75 |  |

**Table 10: Training hyperparameters for each model.**
| Model | LR | Epochs | Batch Size | Weight Decay | Scheduler | Warmup Steps |
| --- | --- | --- | --- | --- | --- | --- |
| GLM | 2e-4 | 10 | 32 | 0.01 | Cosine ($T_{\max}=10$) | 1000 |
| LLaVA | 2e-4 | 10 | 64 | 0.01 | Cosine ($T_{\max}=10$) | 1000 |
| Qwen | 2e-4 | 10 | 32 | 0.01 | Cosine ($T_{\max}=10$) | 1000 |

### C.3 Mapper Evaluation

Inspired by (Liu et al., 2023), we assess the reconstruction capability of the MLP connector $f$ using GPT-4 as a judge.
For each input image $x$, we first obtain the response of the target MLLM (e.g., LLaVA) when conditioned on the true image embedding $\mathcal{V}_{\mathcal{M}})$.
We then provide this response, together with the ground-truth COCO object annotations, to GPT-4, which rates the consistency between the response and the annotations.
Next, we replace the image input with the projected embedding $\Pi(\mathcal{V}_{\text{clip}}(x))$, obtain the MLLM response, and request GPT-4 to rate it in the same way.
The relative score between the two ratings serves as a measure of the mapper’s ability to reconstruct the semantics of the image.
We define the relative score as the ratio between the GPT rating of the projected-embedding response and that of the real-image response which quantifies how much semantic information from the original image embedding is retained after projection.
Further details on this experiment and the evaluation prompt are provided in the appendix.
For this study, we randomly selected 100 images from the COCO dataset.
The results on LLaVA-v1.6, Qwen2.5-VL, and GLM are reported in Table [11](https://arxiv.org/html/2509.25178v1#A3.T11).
As shown, the projector preserves most of the semantic content of the original image while substantially reducing computational cost.

**Table 11: Evaluation of Mapper using GPT-4 as judge. Scores are reported for MLLM responses conditioned on the real image and on the reconstructed embedding.**
| Source Model | Input Type | Basic Score | Relative Score |
| --- | --- | --- | --- |
| Qwen2.5-VL | Real Images | 89.6 | 76.0 |
| Reconstructed | 68.1 |  |  |
| LLaVA-v1.6 | Real Images | 84.9 | 54.5 |
| Reconstructed | 46.3 |  |  |
| GLM4.1-Thinking | Real Images | 94.0 | 75.6 |
| Reconstructed | 71.1 |  |  |

## Appendix D Latent Diffusion Models

Diffusion probabilistic models (Sohl-Dickstein et al., 2015; Ho et al., 2020) generate samples that approximate the data distribution through a parameterized Markov chain.
The core idea is to gradually corrupt a clean sample with Gaussian noise (forward process), and then learn to reverse this process (backward process) to recover clean data.

Latent Diffusion Models (LDMs) (Rombach et al., 2022) perform this procedure in a compressed latent space.
Let $z_{0}$ denote the latent representation of a clean image obtained via a VAE encoder ($\mathcal{E}_{vae}$.)
The forward noising process defines a distribution

$$ $q(z_{t}\mid z_{0})=\mathcal{N}\!\bigl(z_{t};\,\sqrt{\bar{\alpha}_{t}}\,z_{0},\,(1-\bar{\alpha}_{t})I\bigr),$ $$

where $t\in\{1,\ldots,T\}$ is the diffusion timestep, $\alpha_{t}$ are variance schedule parameters, and $\bar{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$.
Thus, a latent $z_{t}$ can be obtained directly by applying the scheduler to $z_{0}$, progressively injecting noise as $t$ increases.

The reverse process learns to iteratively denoise, starting from $z_{T}\sim\mathcal{N}(0,I)$ or, more generally, from a noisy latent $z_{t}$:

$$ $z_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}\left(z_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\,\epsilon_{\theta}(z_{t},t,C)\right)+\sigma_{t}\,\mathbf{z},$ $$

where $\epsilon_{\theta}$ is a UNet denoiser trained on large-scale data, $C$ is an optional conditioning signal (e.g., text or image embeddings), and $\mathbf{z}\sim\mathcal{N}(0,I)$.
Finally, the clean latent $z_{0}$ is decoded through the VAE decoder $\mathcal{D}$ to obtain the generated sample in pixel space.

In this work, we employ the Stable Diffusion unCLIP model (Rombach et al., 2022), which conditions the denoising process on CLIP image embeddings (Radford et al., 2021a).
Given an image $x$ and a CLIP image encoder $\mathcal{E}_{\text{clip}}$, the model generates a new sample conditioned on $\mathcal{E}_{\text{clip}}(x)$, ensuring that the output preserves the high-level semantics of $x$.

## Appendix E GHOST Optimization

### E.1 GHOST Without CLIP Sorting

For evaluating the performance of GHOST without CLIP-based sorting, we conduct an experiment on four classes: “Boat”, “Vase”, “Knife”, and “Bottle”. For each class, we randomly select 200 samples that do not have the target object from their categories based on COCO annotations. When we select the pictures without sorting, the initial images are farther from the target object semantically, so more changes are needed to induce hallucination. To overcome this issue, we use noise level $t=15$ (in contrast to our standard setting where the noise level is $t=30$) as a hyperparameter to add more noise to the image and be able to change the image further. We also select $\tau=0.6$ to soften the constraint of optimization; all the other hyperparameters are the same as the standard setting. As reported in Table [12](https://arxiv.org/html/2509.25178v1#A5.T12), the success rate is comparable to our method in the sorting setting. Qualitative samples with $t=15$ as the noise level are also shown in Figure [12](https://arxiv.org/html/2509.25178v1#A5.F12). As shown in Figure [12](https://arxiv.org/html/2509.25178v1#A5.F12), when the noise level is decreased, more changes are applied to the initial images to induce hallucination.

Figure: Figure 12: Qualitative samples from GHOST on Qwen2.5 VL 7B, with noise level $t=15$. The top row shows the original images, and the bottom row shows samples generated with GHOST.
Refer to caption: figs/AppendixSORt.jpg

**Table 12: Class-wise results with success rate and number of successful samples in $t=15$ noise level setting.**
| Class | Images Optimized | Success Rate | Successful Samples |
| --- | --- | --- | --- |
| Vase | 198 | 0.32 | 63 |
| Knife | 198 | 0.46 | 91 |
| Boat | 200 | 0.24 | 48 |
| Bottle | 188 | 0.57 | 107 |
| Total / Mean | 784 | 0.40 | 309 |

### E.2 Text representation

The $\mathcal{L}_{\text{clip}}$ term in our objective function (Equation [4](https://arxiv.org/html/2509.25178v1#S4.E4)) prevents the optimized image embedding $z$ from directly encoding the semantics of the target object. To make this regularization robust, we use an enhanced, compositional text representation of the target object, rather than a single word textual description. This representation helps push the optimized image embedding away from a comprehensive semantic understanding of the object, encouraging the MLLM to hallucinate based on subtle visual cues instead of object generation.

Our compositional text representation for a target object $t$ is constructed from three distinct sources, whose CLIP embeddings are then combined via a weighted average:

- 1.
Direct Object Description: We start with a straightforward descriptive sentence, such as “A photo of a class_name”. This provides a simple, unambiguous representation of the object. Let its CLIP embedding be $E_{D}$.
- 2.
Generic Contextual Templates: We utilize a set of generic templates to capture various linguistic contexts. These templates are universal across all target objects and are listed in Listing [1](https://arxiv.org/html/2509.25178v1#LST1). For a given target object $t$, these templates are filled to form specific phrases (e.g., “A scene featuring a vase”). Up to $N_{GT}=4$ such template-based phrases are selected, and their CLIP embeddings are denoted as $E_{GT,j}$ for $j=1,\dots,N_{GT}$.
- 3.
Mined COCO Captions: We augment the representation by identifying and embedding actual captions from the COCO training dataset that explicitly contain the target object. Incorporating real-world descriptive language captures nuanced ways humans refer to objects in visual contexts. Up to $N_{CC}=5$ such mined captions are selected, and their CLIP embeddings are denoted as $E_{CC,k}$ for $k=1,\dots,N_{CC}$.

These CLIP text embeddings are combined into a single ‘compositional embedding’ ($E_{\text{comp}}$) using a weighted average. The formula is:

$$ $E_{\text{comp}}=w_{D}E_{D}+\sum_{j=1}^{N_{GT}}\frac{w_{GT}}{N_{GT}}E_{GT,j}+\sum_{k=1}^{N_{CC}}\frac{w_{CC}}{N_{CC}}E_{CC,k}$ (7) $$

where $w_{D}=0.3$, $w_{GT}=0.4$, and $w_{CC}=0.3$ are the base weights for the direct description, generic templates, and COCO captions, respectively. The weights for generic templates and COCO captions are distributed evenly among their respective constituent embeddings. This robust $E_{\text{comp}}$ is then used as the specific text query $\mathbf{T}_{q}$ for the $\mathcal{L}_{\text{clip}}$ loss term, defined as $\cos(z,\mathcal{V}_{\text{CLIP}}(E_{\text{comp}}))$. By minimizing this loss, we encourage $z$ to be dissimilar to $E_{\text{comp}}$, thereby preventing the optimized image embedding from inadvertently encoding the actual presence of the target object.

Figure: Listing 1: Generic contextual templates used for constructing the enhanced text representation. The placeholder ‘class_name’ is replaced with the target object name during embedding generation.

### E.3 Hyperparameters

We summarize the attack hyperparameters for each victim model in Table [13](https://arxiv.org/html/2509.25178v1#A5.T13), while the corresponding model identifiers are listed separately in Table [14](https://arxiv.org/html/2509.25178v1#A5.T14). We tuned the hyperparameters based on our evaluation metrics, including image quality (FID score), attack success rate, and human evaluation, which all are discussed in the main paper.

**Table 13: Attack hyperparameters for each victim model.**
| Model | lr | total steps | $\tau$. | N | Guidance Scale | $\lambda_{clip}$ | $\lambda_{reg}$ | OD thr. | t | num_inf. |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Qwen | 0.1 | 100 | 0.8 | 4 | 5.0 | 15.0 | 10.0 | 0.5 | 30 | 50 |
| LLaVA | 0.1 | 100 | 0.8 | 4 | 5.0 | 15.0 | 10.0 | 0.5 | 30 | 50 |
| GLM | 0.2 | 125 | 0.5 | 5 | 5.0 | 0.5 | 1.5 | 0.5 | 30 | 50 |

**Table 14: Model identifiers.**
| Model | ID / Name |
| --- | --- |
| Qwen | Qwen/Qwen2.5-VL-7B-Instruct |
| LLaVA | llava-hf/llava-v1.6-mistral-7b-hf |
| GLM | THUDM/GLM-4.1V-9B-Thinking |

### E.4 Prompts

At each step of the optimization, we randomly select a prompt from our template set to avoid overfitting to a specific prompt. The prompts are designed to query the presence of the target object in a binary (Yes/No) format. This randomization ensures that the optimization does not exploit superficial linguistic patterns but instead focuses on inducing the desired hallucination. The complete set of prompt templates is provided in Listing [2](https://arxiv.org/html/2509.25178v1#LST2).

Figure: Listing 2: Binary question templates used for querying the presence of the target object. The placeholder ‘obj’ is replaced with the object name (e.g., “boat”, “vase”) during optimization.

### E.5 Effect of λ r ​ e ​ g \lambda_{reg}

To assess the effect of $\lambda_{reg}$, we apply GHOST to the GLM4.1 Thinking model with $\lambda_{reg}\in\{1.0,1.5,2.0\}$. Since the FID score is sensitive to the number of samples, for each pair of $\lambda_{reg}$ values we select only the samples that are successful in both settings, and then compute the FID score against both the COCO validation set and the original images. As reported in Table [15](https://arxiv.org/html/2509.25178v1#A5.T15), increasing $\lambda_{reg}$ consistently makes the generated images more similar to the original ones, leading to improved realism, as reflected by lower FID scores.

### E.6 Effect of λ c ​ l ​ i ​ p \lambda_{clip}

Increasing $\lambda_{\text{clip}}$ encourages semantic separation between the guiding embedding $z$ and the target-object semantics. We implement GHOST on two target objects (“traffic light" and “Boat", 200 images each) on Qwen2.5-VL with $\lambda_{\text{clip}}\in\{5,10,15,20\}$ and $N{=}4$ samples per image to evaluate the effect of it. As shown in Fig. [14](https://arxiv.org/html/2509.25178v1#A5.F14), larger $\lambda_{\text{clip}}$ consistently reduces the count of OD-filtered samples, indicating that the regularization term achieves its intended effect by suppressing target-object semantics in the generated images. From $\lambda_{\text{clip}}=5$ to $15$, stronger suppression of target-object cues makes it harder for the MLLM to answer “yes”, so more samples are generated and also a larger share of generated samples fail to trigger hallucination.
At $\lambda_{\text{clip}}=20$, this term becomes over-weighted: fewer runs satisfy $p(y^{*}\mid\mathbf{X}_{q},\Pi(z))\geq\tau_{\text{yes}}$, and more runs hit the maximum (100) steps of optimization (e.g., $10\rightarrow 20$)

### E.7 Effect of τ \tau

We implement GHOST with two target objects (200 images each) on LLaVA-1.6, varying $\tau\in\{0.5,0.6,0.7,0.8,0.9\}$ and using $N{=}4$ generations per image. As shown in Fig. [14](https://arxiv.org/html/2509.25178v1#A5.F14), increasing $\tau$ tightens the optimization constraints, and the number of generated samples consistently decreases. At lower $\tau$, the embedding contains weaker target-object cues, so more generations fail to elicit a “yes” from the MLLM. For $\tau\in\{0.5,0.6,0.7\}$, the ratio of total to unsuccessful generations is similar, and the number of successful samples is nearly unchanged.

Figure: Figure 13: The effect of $\lambda_{\text{clip}}$.
Refer to caption: figs/lambdaclip.png

**Table 15: Ablation study on $\lambda_{\text{reg}}$. FID scores computed on the intersected set of generated images, with respect to the original images and the COCO validation set. The lower FID in each pair is shown in bold.**
| Gen. Data | $\lambda_{\text{reg}}$ | Orig | Val |
| --- | --- | --- | --- |
| (1.0, 1.5) | 1.0 | 127.46 | 145.05 |
|  | 1.5 | 125.11 | 144.11 |
| (1.0, 2.0) | 1.0 | 148.85 | 169.74 |
|  | 2.0 | 138.67 | 163.84 |
| (1.5, 2.0) | 1.5 | 144.25 | 168.20 |
|  | 2.0 | 135.79 | 163.54 |

## Appendix F GHOST Additional Results

In this section, we provide extended qualitative and quantitative results to complement the main paper. Figures [15](https://arxiv.org/html/2509.25178v1#A6.F15), [16](https://arxiv.org/html/2509.25178v1#A6.F16), and [17](https://arxiv.org/html/2509.25178v1#A6.F17) present qualitative examples of GHOST applied to Qwen2.5, LLava-1.6, and GLM-4.1Thinking, respectively, showing the original images alongside GHOST-generated samples and the corresponding model responses. Tables [16](https://arxiv.org/html/2509.25178v1#A6.T16), [17](https://arxiv.org/html/2509.25178v1#A6.T17), and [18](https://arxiv.org/html/2509.25178v1#A6.T18) report detailed sample statistics for each object category across the three victim models, including the number of generated, filtered, and successful samples. Finally, Tables [20](https://arxiv.org/html/2509.25178v1#A6.T20), [20](https://arxiv.org/html/2509.25178v1#A6.T20), and [21](https://arxiv.org/html/2509.25178v1#A6.T21) provide class-wise transferability results, highlighting how GHOST-induced hallucinations on one victim model transfer to others.

Figure: Figure 15: Qualitative samples from GHOST on Qwen2.5 as the victim model, along with Qwen’s responses to each image. For each pair, the top image shows the original input, while the bottom image shows the corresponding sample generated by GHOST.
Refer to caption: figs/QualitativeAppendixQwen.jpg

Figure: Figure 16: Qualitative samples from GHOST on LLava1.6 as the victim model, along with LLava’s responses to each image. For each pair, the top image shows the original input, while the bottom image shows the corresponding sample generated by GHOST.
Refer to caption: figs/QualitativeAppendixLLava.jpg

Figure: Figure 17: Qualitative samples from GHOST on GLM4.1Thinking as the victim model, along with GLM’s thinking phase in each image. For each pair, the top image shows the original input, while the bottom image shows the corresponding sample generated by GHOST.
Refer to caption: figs/QualitativeAppendixGLM.jpg

**Table 16: Sample statistics for different objects with Qwen2.5 as the victim model.**
| Object | Samples Considered | Image Generated | Image Filtered | Successful Samples |
| --- | --- | --- | --- | --- |
| Boat | 915 | 3141 | 191 | 258 |
| Bottle | 881 | 2746 | 21 | 383 |
| Bus | 891 | 2995 | 311 | 244 |
| Carrot | 956 | 3353 | 132 | 228 |
| Clock | 962 | 3473 | 392 | 175 |
| Knife | 966 | 2368 | 114 | 353 |
| Suitcase | 949 | 3067 | 234 | 367 |
| Toilet | 969 | 3368 | 817 | 289 |
| Traffic light | 992 | 3291 | 424 | 312 |
| Vase | 942 | 3279 | 101 | 207 |

**Table 17: Sample statistics for different objects with LLlava-1.6 as the victim model.**
| Object | Samples Considered | Image Generated | Image Filtered | Successful Samples |
| --- | --- | --- | --- | --- |
| Boat | 863 | 2804 | 63 | 270 |
| Bottle | 763 | 1558 | 4 | 221 |
| Bus | 786 | 2221 | 168 | 178 |
| Carrot | 948 | 3041 | 101 | 218 |
| Clock | 927 | 2588 | 198 | 298 |
| Knife | 843 | 1092 | 29 | 188 |
| Suitcase | 885 | 1644 | 100 | 364 |
| Toilet | 945 | 3081 | 543 | 268 |
| Traffic light | 967 | 2508 | 160 | 241 |
| Vase | 859 | 1499 | 33 | 216 |

**Table 18: Sample statistics for different objects with GLM-4.1 Thinking as the victim model.**
| Object | Samples Considered | Image Generated | Image Filtered | Successful Samples |
| --- | --- | --- | --- | --- |
| Boat | 847 | 2032 | 202 | 305 |
| Bottle | 710 | 1346 | 53 | 348 |
| Bus | 764 | 2102 | 373 | 321 |
| Carrot | 939 | 3130 | 76 | 272 |
| Clock | 937 | 3146 | 297 | 290 |
| Knife | 968 | 2644 | 80 | 271 |
| Suitcase | 937 | 2322 | 276 | 328 |
| Toilet | 942 | 1932 | 486 | 217 |
| Traffic light | 979 | 2294 | 192 | 176 |
| Vase | 866 | 2137 | 141 | 286 |

**Table 19: Class-wise transferability with Qwen2.5 as the victim model.**
| Object | LLaMA | LLava-1.6 | GLM |
| --- | --- | --- | --- |
| Boat | 67.8 | 68.2 | 79.7 |
| Bottle | 67.1 | 66.1 | 85.4 |
| Bus | 73.0 | 49.6 | 85.4 |
| Carrot | 52.2 | 48.3 | 64.7 |
| Clock | 57.7 | 70.3 | 75.6 |
| Knife | 64.0 | 64.9 | 52.7 |
| Suitcase | 69.2 | 70.3 | 58.7 |
| Toilet | 73.4 | 52.9 | 73.5 |
| Traffic light | 67.7 | 64.4 | 78.1 |
| Vase | 58.9 | 61.8 | 71.7 |

**Table 21: Class-wise transferability results with GLM-4.1Thinking as the victim model.**
| Object | LLaMA3.2 | Qwen2.5 | LLava1.6 |
| --- | --- | --- | --- |
| Boat | 75.7 | 67.0 | 66.0 |
| Bottle | 75.6 | 72.5 | 62.8 |
| Bus | 62.6 | 42.7 | 35.8 |
| Carrot | 45.2 | 52.2 | 31.6 |
| Clock | 56.1 | 56.1 | 65.9 |
| Knife | 84.5 | 69.0 | 68.6 |
| Suitcase | 82.2 | 70.9 | 68.1 |
| Toilet | 69.4 | 68.9 | 48.4 |
| Traffic light | 71.7 | 72.8 | 53.9 |
| Vase | 64.6 | 63.2 | 64.9 |

## Appendix G Finetuning

A growing body of work has shown that models robustness can be substantially improved through fine-tuning on synthetically generated or edited images (Augustin et al., 2025; Parast et al., 2025; Wang et al., 2024b; Wu et al., 2025; Azizmalayeri et al., 2022; Zhang et al., 2025b; Mirzaei et al., 2025; Bai et al., 2025b).

### G.1 Fine-tuning protocol.

We fine-tune Qwen2.5VL-7B using images generated by our GHOST procedure (targeting Qwen2.5VL -7B) to assess whether such counterfactual samples improve model performance. We include this as a proof-of-concept demonstration, with scaling to larger datasets and all classes reserved for future work. We adopt LoRA (Hu et al., 2022) for parameter-efficient adaptation; the exact hyperparameters are listed in Table [22](https://arxiv.org/html/2509.25178v1#A7.T22).

### G.2 Dataset construction.

We constructed a balanced fine-tuning set by pairing each *negative* sample (a GHOST-generated image) with one *positive* sample. Positives were synthesized with Stable Diffusion unCLI P(SD unCLIP) (Rombach et al., 2022), with images which contain the object as input. We intentionally avoided using real COCO images as positives to prevent shortcut learning (i.e., the degenerate rule “synthetic$\rightarrow$negative, real$\rightarrow$positive”).

To increase label fidelity for the synthesized positives, we relied on the COCO annotations , and filtered candidate generations by ranking their CLIP embeddings against a compositional text embedding of the target object (Sec. [E.2](https://arxiv.org/html/2509.25178v1#A5.SS2)), keeping the highest-scoring images to give inputs to SD unCLIP model. This process improves the likelihood that the positive set truly depicts the intended object while keeping the data domain consistent (synthetic vs. synthetic) across classes. Examples of positive samples generated are shown in Figure [18](https://arxiv.org/html/2509.25178v1#A7.F18).

### G.3 Model selection

We use POPE on the COCO training split as our checkpointing criterion. After each epoch, we compute the POPE *Random* setting on 1,000 samples and retain the checkpoint with the highest F1 score.

### G.4 Model Evaluation

We evaluate the fine-tuned model on our generated images from other models (victim models) and on POPE using the COCO validation set. Specifically, we use the POPE *Adversarial*, *Random*, and *Popular* settings with maximum samples of $n=3000$ samples. The results are available in the main paper. To evaluate how much fine-tuning impacted the overall performance of the MLLM, we evaluated the model on VQA and captioning tasks. For the VQA task, we leverage the small VQAv2 dataset from the Hugging Face implementation(^1^11https://huggingface.co/datasets/merve/vqav2-small). The dataset consists of an image–question pair and a single-word ground-truth answer. We compute accuracy for both the base model and the fine-tuned model on this task to assess the impact of fine-tuning. For the captioning task, we leverage COCO validation images and captions as ground truth. We generate captions for all COCO validation images with the base model and the fine-tuned model using the prompt: “Write a short caption for the given image.” Then we compute BERTScore (Zhang* et al., 2020) using the generated captions and COCO annotations as ground truth.

**Table 22: Fine-tuning hyperparameters and LoRA configuration.**
| LoRA configuration |  |
| --- | --- |
| Rank ($r$) | 8 |
| Alpha | 32 |
| Dropout | 0.05 |
| Training setup |  |
| Optimizer | Adam |
| Learning rate | $5\times 10^{-6}$ |
| Epochs | 15 |
| Batch size | 16 |
| Warmup ratio | 0.10 |
| Scheduler | Cosine with warmup |

Figure: Figure 18: Positive samples generated with Stable Diffusion unCLIP (Rombach et al., 2022)
Refer to caption: figs/unCLIP.jpg

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: What detector threshold did you use?

Did you try an ensemble of detectors?

What are the default hyperparameters for the main results?

What are the prompts and decoding settings for closed models?

For reasoning models, when does a “Yes” in thinking flip to “No” in the final answer?
soundness: 3
strengths: - The pipeline is simple, modular and well motivated. 

- The success rates are strong across many MLLMs. The transfer rate to closed models is also high.

- Preliminary mitigation results via fine-tuning suggest utility beyond red-teaming.
summary: The paper introduces GHOST. It first optimizes a CLIP image embedding. It keeps the image close to the source. It pushes away from the target object. It also raises the model’s chance of saying “Yes” to the object query. A small mapper links CLIP space to the vision tokens of the MLLM. After optimization, the method uses unCLIP to render the final image. The images do not contain the object, yet many MLLMs claim the object is present. The method beats a prior approach by a large margin. The authors also use the images to fine-tune models and reduce hallucinations.
weaknesses: - The pipline heavly relianced on a single detector (OWLv2); possible false-negatives not quantified; no ensemble verification.

- Limited baseline coverage beyond DASH; no ablations versus strong pixel-space attacks adapted to MLLMs. It should include more public hallucination benchmarks.

- The closed-model settings are not fully described. The compute cost and runtime are not reported. Please ddd default hyperparameters in the main text.

- Minors writing issues (Also in Appendix)

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: * The method regularizes the generated image to remain close to the original image (Equation 3). From an attack perspective, it is unclear why maintaining similarity to the original image is important, since the primary goal is to induce hallucination in the model. A clearer justification for this design choice would be valuable.

* It would be interesting to see how the proposed method performs on image domains other than COCO, as this would test the generalization and robustness of GHOST across more diverse visual distributions.
soundness: 3
strengths: * The paper proposes an effective model-specific attack that generates adversarial images capable of inducing hallucinations in a target MLLM regarding a given target object in the original image.

* The generated images demonstrate strong transferability, degrading the performance of other models beyond the specific target model. This suggests that the proposed approach reveals not only model-specific weaknesses but also shared vulnerabilities across models.
summary: The paper introduces GHOST, a gradient-free method that generates images designed to induce hallucinations in text-to-image and multimodal models, and establishes a benchmark for measuring hallucination robustness. Experiments show that GHOST achieves high attack success rates and strong transferability across multiple open-source models, revealing both model-specific weaknesses and shared vulnerabilities. The authors also explore simple mitigation strategies, such as fine-tuning, which help reduce but not fully eliminate the hallucination effect, demonstrating GHOST’s potential as a tool for evaluating and improving model robustness.
weaknesses: * The proposed method is limited to inducing misperception of object existence and does not address other types of hallucination, such as attribute or relationship hallucinations. This limitation is reflected in Table 4, where improvements are observed mainly on POPE—a benchmark focused on object-centric VQA—while no significant gains are found on VQAv2 or Caption benchmarks. This implies that the method may not generalize to mitigating broader hallucination types.

* The adversarially generated images (e.g., Figures 1, 2, 4, and 6) appear somewhat unnatural. This raises concerns about the practical impact of improving robustness against such images in real-world applications that primarily involve natural images. From an impact perspective, it would strengthen the work if the human evaluation included not only whether the target object was preserved or removed, but also whether the generated images looked natural to human observers. The current reliance on FID scores is insufficient to fully assess perceptual naturalness.

* The proposed method requires multiple optimization steps per image, which may reduce its efficiency. A quantitative analysis of this efficiency—such as reporting the hit ratio (i.e., success rate among optimized samples)—would be helpful.

* The approach involves numerous hyperparameters (e.g., thresholds and control factors), which implies that substantial model-specific tuning may be required to achieve optimal attack performance. This limits its scalability and reproducibility.

* Minor issue: The paper could improve notation consistency—for example, the fonts for λ_clip and λ_reg differ in several equations.

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 2
questions: * Could you clarify why *human-perceptible* modifications were chosen as a design goal? What advantages do these perceptible cues provide over imperceptible perturbations in evaluating hallucination?  
    
* Why was **SD unCLIP** selected as the semantic preservation backbone instead of other diffusion variants? Table 2 shows inconsistent FID results across models, suggesting that SD unCLIP may not always guarantee semantic fidelity  
    
* In Table 1, the results may not be entirely fair since GHOST and DASH were designed with different objectives. How could the two methods be compared in a more balanced and fair manner?
soundness: 2
strengths: * Unlike prior benchmarks relying on static datasets,  this approach dynamically generates images that preserve the original semantics while triggering hallucinations, providing a fully automated way.  
    
* The framework is well-designed and clearly presented. Each component is justified, and the ablation studies effectively demonstrate their contributions.  
    
* The experiments are extensive and evaluate on a diverse set of MLLMs using various metrics.
summary: This paper introduces a new method for inducing object hallucination in MLLMs. 
To address this issue, the paper utilizes an image generation model to automatically create images that preserve the semantic meaning of the original inputs while still being able to trigger hallucinations.
Through extensive experiments, the method demonstrates a significantly higher hallucination success rate compared to previous approaches across multiple models, confirming its effectiveness. 
Additionally, the paper shows that images generated for one model can induce hallucinations in other models, and that fine-tuning with generated data helps mitigate hallucination, highlighting its dual role as both a diagnostic and corrective tool.
weaknesses: Major Weakness

* **Insufficient Motivation for Human-Perceptible Modifications:** While GHOST introduces “human-perceptible” changes, the paper does not explain why human perceptibility is a necessary design choice. This aspect distinguishes GHOST from adversarial methods but is presented without motivation.  
    
* **Unclear Justification for Using SD unCLIP as the Semantic Preservation Backbone:** The paper adopts SD unCLIP as the image generation backbone to preserve semantic content from the original image, yet it does not clearly justify why this model was chosen over other diffusion variants.   
  Moreover, Table 2 shows inconsistent FID scores across models: While SD unCLIP achieves low FID values for some settings, in other cases its FID is high. This raises questions about whether SD unCLIP is a reliable method for maintaining semantic similarity.

* **Fairness of Comparison with DASH:** Table 1 compares GHOST with DASH, yet the evaluation settings differ significantly. Although both methods share the same goal of inducing hallucinations via false-positive responses, DASH is fundamentally designed for large-scale, open-world datasets (e.g., ReLAION-5B) to detect *systematic hallucination patterns* across diverse object classes. In contrast, GHOST operates on a COCO-based subset. Given this difference in scale and design philosophy, directly comparing hallucination success rates on COCO may not be fair. 

Minor Weakness

* **Clarity in Figure 3**  
  * The black dashed arrow is ambiguous; please clarify what operation or information flow it represents.

* **Typos and Formatting Issues**  
  * Line 235–236: In Equation (2), the two terms involving X\_v​ appear inconsistent.  
      
  * Appendix H.2: There seems to be an empty placeholder for *unCLIP*.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: Q1. What is the average time to generate a hallucination‑inducing image, and how does this scale with image resolution and the number of target objects? Could the process be parallelized or approximated to reduce cost?

Q2. Have you tested GHOST with alternative diffusion architectures (e.g., SDXL, DiT) or other vision encoders (e.g., EVA CLIP)? Would the mapper still align the embedding spaces effectively?

Q3. How sensitive are the results to the choice of OWLv2? Could false negatives in detection inflate success rates? Did you manually inspect filtered samples to ensure the target object was truly absent?
soundness: 3
strengths: S1. GHOST introduces a principled pipeline to generate hallucination‑inducing images.

S2. Images optimized for one model induce hallucinations in others, highlighting shared failure modes across architectures.

S3. Fine‑tuning on GHOST images reduces hallucinations and improves POPE scores while retaining performance on VQAv2 and captioning.
summary: The paper proposes GHOST (Generating Hallucinations via Optimizing Stealth Tokens), a pipeline for stress‑testing multimodal large language models (MLLMs) by automatically generating images that induce object hallucinations. Starting from an input image and a target object absent from that image, GHOST optimizes the image’s CLIP embedding to increase the MLLM’s probability of answering “yes” to a query about the target’s presence, while regularizing the embedding to remain close to the original and discouraging direct encoding of the object’s semantics. To avoid backpropagating through the diffusion model and the MLLM, the authors train a mapper (a simple MLP) that aligns the CLIP vision embedding space with the MLLM’s vision encoder, enabling efficient gradient updates. Once the optimized embedding reaches a confidence threshold, an unCLIP diffusion model generates an image conditioned on this embedding, starting from a partially noised latent of the original image to preserve overall structure. An open‑vocabulary detector filters out images where the target object actually appears, ensuring that only hallucinations are counted.
weaknesses: W1. Experiments focus on 10 COCO object classes. The method may perform differently on a broader set of objects, particularly those with abstract or ambiguous semantics. The human evaluation sample is limited to 40 volunteers, which may not capture diverse perceptions.

W2. The OWLv2 detector filters out samples where the target object is present, but open‑vocabulary detectors are not perfect; subtle insertions might slip through, biasing success rates.

W3. Fine‑tuning is demonstrated only on Qwen2.5‑VL with synthetic positives and negatives. More robust experiments (e.g., across models or with real images) and analysis of downstream safety implications would strengthen the claim that GHOST can act as a corrective tool.

W4. quantify the per‑image runtime or GPU requirements; scaling to many objects or higher resolutions could be prohibitive.

W5. GHOST hinges on the availability of CLIP vision embeddings and an unCLIP diffusion model. It is unclear whether other vision encoders or diffusion architectures would offer similar alignment.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1.GHOST's attack generation involves an adversarial loss tailored to a specific victim model . Given the observed transferability, could a potentially universal hallucination dataset be constructed by optimizing against a set of diverse open-source models? Could such a dataset be effectively used for evaluating and perhaps even finetuning other models to improve general hallucination robustness?

2.The approach of finetuning on GHOST images, adversarially generated images causing hallucination, conceptually resembles methods like PerturboLLaVA, which finetunes on perturbed text to mitigate hallucinations. Could the mathematical framework or explanation provided by PerturboLLaVA be adapted to theoretically explain why finetuning on GHOST's visually perturbed, hallucination-inducing images improves the model's resistance to object hallucination?
soundness: 3
strengths: 1.The work constructs a benchmark for VLM hallucination vulnerability from a novel perspective by optimizing visual embeddings, demonstrating effective hallucination-inducing attacks against sota open-source models. Furthermore, it shows that hallucinations induced on open-source models can effectively transfer to sota closed-source VLMs, suggesting shared vulnerabilities.

2.The paper demonstrates that finetuning models using the data generated by GHOST can reduce object hallucination, showing improvements on both the GHOST-generated adversarial images and established hallucination benchmarks like POPE.
summary: This paper introduces GHOST, a method for automatically generating images designed to induce object hallucinations in VLM. Unlike static benchmarks , GHOST actively synthesizes images by optimizing a CLIP image embedding to elicit a hallucinated response while simultaneously regularizing against inserting the object's semantics and staying close to an original image embedding . The authors demonstrate GHOST's effectiveness in inducing hallucinations across various MLLMs, including reasoning models, and show that these hallucinations are highly transferable, even to closed-source models like GPT-4o. Finally, preliminary experiments suggest that finetuning an MLLM on GHOST-generated images can improve its robustness against hallucinations on GHOST's own generated data and the POPE benchmark.
weaknesses: 1.A significant concern is the potential distribution gap between the images decoded from the optimized visual embeddings and natural images. For example, the right image in Figure 1 appears somewhat unnatural . It is questionable whether using such potentially out-of-distribution images constitutes a fair or meaningful way to attack and evaluate the robustness of MLLMs against real-world hallucinations.

2.The experiments validating the mitigation effect of finetuning on GHOST data seem insufficient. While results on POPE and GHOST's own data are shown, the evaluation would be more convincing if it included results on other recognized hallucination benchmarks such as CHAIR or HallusionBench to demonstrate broader robustness improvements.


Decision: Accept (Poster)